With the breakdown of coherent Kondo scattering with rising temperature, the Fermi surface (FS) in rare-earth Kondo lattices is expected to transition from the large FS counting the 4f moments, which form entangled quasiparticles with strongly enhanced effective masses with the conduction states, to the small FS with decoupled and localized 4f moments. A direct observation of this transition with temperature has, however, remained elusive, because conventional probes of the FS in Kondo lattices require high magnetic fields, which might reconstruct the FS or cannot work reliably at elevated temperatures. Using high-resolution Compton scattering, we overcome these limitations and show that the FS topology in the prototypical Kondo lattice YbRh2Si2 undergoes pronounced changes between 14 and 300 K in zero magnetic field. In particular, we present clear evidence for a largely restored small FS at room temperature. Our results suggest the relevant energy scale of the complex Kondo crossover phenomenon in YbRh2Si2.