The Rayleigh backscattering produced by an incident fundamental mode in a multimode step-index optical fiber was analyzed using a recently developed diffraction technique. The complete set of backward propagating modes, both radial and azimuthal, was determined and regarded. For this type of fiber, normalized mode functions were constructed in an explicit form, thus providing a unified power scale to characterize the relationships between various excited modes. The dependencies of the mode excitation efficiencies on the technical parameters of the fiber and the frequency of the launched radiation were studied. A comparison of the mode excitation efficiencies was performed with those in the fiber with quadratic refractive index profiles. (C) 2017 Optical Society of America