On Machine Learning In Regression Analysis

Svetlana Nikolaevna Leora, Sergey Michaylovich Ermakov

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review


As is known, the task of constructing a regression function from observed data is ofgreat practical importance. In the case of additive error of observations at points whosecoordinates are given without errors, we have:yj=f(Xj) +εj, wherej= 1,...,Nis the observation number,yj– the observed value,Xj= (x1j,...,xsj) is the point atwhich the observation took place,εjis the observation error. It is also assumedEεj= 0.The task is to define a function f that is usually considered to be given parametrically,f(X) =f(X,U), whereUare unknown parameters. The problem has obvious connectionswith problems of approximation of functions.The report discusses one of the possible approaches using the idea of machine learning.It is based on the approximation problem for some functionf. LetAbe a linear operatoracting in a linear normed spaceF. IfA∗is an adjoint operator toA, the functionsφjandψjofFsatisfy the conditions:AA∗φj=s2jφj,AA∗φj=s2jψj,j= 1,...,r(A), thenamong all m-dimensional (m≤r(A)) operatorsAmthe operator ̃Am=∑j=1,msj(·,φj)ψjminimizes the norm of||A−Am||.IfK=I−Ais an operator, such thatKf= 0, and in this equality we replaceAwith itsapproximation ̃Am, then we get an approximation tofin the formfˆ=∑j=1,msj(f,φj)ψj.The idea of ””learning” is as follows. LetK(θ) – the parametric family of operators.Using sampled values off, we find an operatorK0=K(θ0),θ0= arg min||K(θ)||, wereθbelongs toθ. AssumingA=I−K0, we find the corresponding functionsφjandψjandconstruct an approximationfˆ for the appropriatem.Real algorithms use spaces of functions defined at discrete points. A well-known particularcase of using this approach is SVD time series analysis. Based on this approach, thereare different generalizations of this analysis. Some examples of generalization are givenin the report.
Original languageEnglish
Title of host publication10th International Workshop on Simulation and Statistics
Subtitle of host publicationWorkshop booklet
Place of PublicationSalzburg
PublisherUniversitat Salzburg
StatePublished - Sep 2019
Event10th International Workshop on Simulation and Statistics
- Salzburg, Australia
Duration: 2 Sep 20196 Sep 2019


Conference10th International Workshop on Simulation and Statistics
Internet address

Scopus subject areas

  • Mathematics(all)

Cite this