An ensemble of quantum dot excitons may be used for coherent information manipulation. Due to the ensemble inhomogeneity any optical information retrieval occurs in the form of a photon echo. We show that the inhomogeneity can lead to a significant deviation from the conventional echo timing sequence. Variation of the area of the initial rotation pulse, which generates excitons in a dot subensemble only, reveals this complex picture of photon echo formation. We observe a retarded echo for π/2 pulses, while for 3π/2 the echo is advanced in time as evidenced through monitoring the Rabi oscillations in the time-resolved photon echo amplitude from (In,Ga)As/GaAs self-assembled quantum dot structures and confirmed by detailed calculations.