Stars of spectral types O and B produce neutron stars (NSs) after supernova explosions. Most of NSs are strongly magnetised including normal radio pulsars with B~10^12 G and magnetars with B~10^14 G. A fraction of 7-12 per cent of massive stars are also magnetised with B~10^3 G and some are weakly magnetised with B~1 G. It was suggested that magnetic fields of NSs could be the fossil remnants of magnetic fields of their progenitors. This work is dedicated to study this hypothesis. First, we gather all modern precise measurements of surface magnetic fields in O, B and A stars. Second, we estimate parameters for log-normal distribution of magnetic fields in B stars and found mu_B = 2.83 ± 0.1 log10(G), sigma_B = 0.65 ± 0.09 for strongly magnetised and mu_B = 0.14 ± 0.5 log10(G), sigma_B = 0.7_-0.27^+0.57 for weakly magnetised. Third, we assume that the magnetic field of pulsars and magnetars have 2.7 DEX difference in magnetic fields and magnetars represent 10 per cent of all young NSs and run population synthesis. We found that it is impossible to simultaneously reproduce pulsars and magnetars populations if the difference in their magnetic fields is 2.7 DEX. Therefore, we conclude that the simple fossil origin of the magnetic field is not viable for NSs.