Autonomous driving systems include modules of several levels. Thanks to deep learning architectures at the moment technologies in most of the levels have high accuracy. It is important to notice that currently in autonomous driving systems for many tasks classical methods of supervised learning are no longer applicable. In this paper we are interested in a specific problem, that is to control a car to move along a given reference trajectory using reinforcement learning algorithms. In control theory, this problem is called an optimal control problem for moving along the reference trajectory. Airsim environment is used to simulate a moving car for a fixed period of time without obstacles. The purpose of our research is to determine the best reinforcement learning algorithm for a formulated problem among state-of-the-art algorithms such as DDPG, PPO, SAC, DQN and others. As a result of the conducted training and testing, it was revealed that the best algorithm for this problem is A2C.