We investigate the magneto-optical properties of excitons bound to single stacking faults in high-purity GaAs. We find that the two-dimensional stacking fault potential binds an exciton composed of an electron and a heavy hole, and we confirm a vanishing in-plane hole g-factor, consistent with the atomic-scale symmetry of the system. The unprecedented homogeneity of the stacking-fault potential leads to ultranarrow photoluminescence emission lines (with a full width at half-maximum 80μeV) and reveals a large magnetic nonreciprocity effect that originates from the magneto-Stark effect for mobile excitons. These measurements unambiguously determine the direction and magnitude of the giant electric dipole moment (e×10nm) of the stacking-fault exciton, making stacking faults a promising new platform to study interacting excitonic gases.