The paper presents a nonconventional approach to processing the interference signals of the optical coherence tomography (OCT) that allows for multiple improvements of the longitudinal spatial resolution of an image obtained. This technique is based on using the root-MUSIC spectral estimation algorithm. The parameters of this algorithm ensuring the accurate determination of the reflector coordinates within a sample have been found. In the course of the theoretical and numerical assessments carried out, the requirements for providing the superresolution by the root-MUSIC algorithm were defined and formulated. In order to evaluate the effectiveness of the proposed method experimentally, the cross-section geometric parameters of multimode and single-mode optical fibers were measured. The obtained results confirmed the validity of the approach that was put forward. © Erovenko Z. A., Markvart A. A., Petrov A. V., Bisyarin M. A., Liokumovich L. B., Ushakov N. A., 2025. Published by Peter the Great St. Petersburg Polytechnic University. This is an open access article under the CC BY-NC 4.0 license (https://creativecommons.org/licenses/by-nc/4.0/)