We report on positive trion (X⁺) spin relaxation in self-assembled InAs/GaAs quantum dots. The hyper-fine interaction between the electron and nuclear spins acts as a randomly oriented magnetic field for the electron spin, giving rise to ensemble dephasing and thus to partial relaxation in less than 1 ns. Time-resolved photoluminescence polarization measurements carried out on an ensemble of InAs/GaAs quantum dots, charged with excess holes by proximity p-doping, reveal this peculiar dynamics. The fast polarization decrease can be efficiently suppressed by screening the hyperfine interaction with a longitudinal magnetic field of 100 mT. The quenching of this hyperfine-induced relaxation is also evidenced for a single charge-tunable quantum dot in the regime of X⁺ photoluminescence and excited with modulated polarization to prevent dynamic nuclear polarization.