Machine learning belongs to the crossroad of cybernetics (control science) and computer science. It is attracting recently an overwhelming interest, both of professionals and of the general public. In the talk a brief overview of the historical development of the machine learning field with a focus on the development of mathematical apparatus in its first decades is provided. A number of little-known facts published in hard to reach sources are presented.