Nowadays mathematical methods of modeling and optimization are extensively used in many fields of science and technology. Development of specialized software for various applications are of more and more importance. A special class of the problems attracting attention of numerous researches is represented by the problems associated with the beam dynamics formation in accelerators. The paper deals with optimization problems of charged particle beam in accelerators. The theory of the optimal beam dynamics formation in both accelerating and focusing structures is suggested. Different mathematical control models describing beam dynamics are presented. Especially we consider the problems connected with taking into account of charged particles interaction. The optimization methods are developed for the presented functionals. They are used for the solving of various beam dynamics problems in different accelerating and focusing structures. It should be noted that the problems under consideration are closely connected with the trajectory ensembles (bundles) control. The problems of the trajectory ensembles control have been recently developed by various authors. This is largely due to the interest in the nonstandard problems of the optimal control theory having a variety of applications. Such problems are often treated as the problems under incomplete information on the initial date and external influences.