The paper aims to analyze the performance of Neural ODE model based solutions for time-series prediction. The original paper from 2018 gave the opportunity for continuous-time modeling of long-time processes. Since then, lots of different modifications were proposed, like ODE-RNN and p-node ODE-RNN, augmented neural ODE, Hamiltonian Neural Networks, Neural Controlled differential equations, etc. All these modifications try to modify the form of ODE lying behind the NN layers. However, the performance of these approaches is still strongly influenced by the choice of numerical solvers, with conventional methods often compromising computational efficiency, stability, or precision. This paper introduces the Taylor Mapping (TM) solver, a numerical integrator that leverages polynomial approximations of the ODE general solution for Neural ODEs. The TM-solver adaptively truncates higher-order terms to balance accuracy and computational cost. What is more, the TM-solver explicitly constructs interpretable maps to reveal local dynamics and can be integrated with automatic differentiation frameworks for efficient training. Evaluations show that the TM-solver improves Neural ODEs by increasing prediction accuracy for synthetic data and real-world datasets.