The paper considers the possibility of using neural networks in solving differential equations and systems. The paper investigates the influence of various basic neural network topologies, various optimizers, activation functions, fast access links and some heuristic methods on the quality of neural network training and the accuracy of solving the differential equation.