This paper presents the results of experimental studies and digital methods of recording holograms using spatial light modulators. The use of holographic interference methods to measure nanoscale irregularity is described for the case of a nanostep (70 nm high and 30 mkm wide). Examples of using the method to increase sensitivity are given to measure the nanostep and the hook method in a digital way. The probability of implementing the method for correcting internal system distortions in the interferometer is proved. It is shown that the use of holographic interference methods with digital processing of holograms provides an increase in the sensitivity of interferograms without losing basic information on the object and without increasing measurement accuracy