A linear algebraic system is solved by the Monte Carlo method generating a vector stochastic series. The expectation of a stochastic series coincides with the Neumann series presenting the solution of a linear algebraic system. An analytical form of the covariation matrix of this series is obtained, and this matrix is used to estimate the exactness of the system solution. The sufficient conditions for the boundedness of the covariation matrix are found. From these conditions, it follows the stochastic stability of the algorithm using the Monte Carlo method. The number of iterations is found, which provides for the given exactness of solution with the large enough probability. The numerical examples for systems of the order 3 and of the order 100 are presented.