Dynamics of the Entropy and F-Divergence Minimization Processes Based on the Speed-Gradient Principle

Dmitry S. Shalymov, Alexander L. Fradkov

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearch


The maximum and minimum entropy states are intensively investigated nowadays because of numerous applications in a various fields of science. Nevertheless the dynamics of a system that tends to minimize its relative entropy is still not well investigated. We propose a new equations describing dynamics of non-stationary processes that minimize f-divergence which is a class of generalized information distances. We use the Speed-Gradient principle originated in the control theory. The uniqueness of the limit probability distribution under the mass conservation and energy conservation constraints is examined. The proposed equations allow to forecast the dynamics of complex non-equilibrium systems. New dynamic equations for the Kullback–Liebler relative entropy, the Tsallis relative entropy, the Burg entropy, the Cressie-Read entropy and other are obtained as a special cases of provided general results.
Original languageUndefined
Title of host publication2016 IEEE Conference on Norbert Wiener in the 21st Century
StatePublished - 2016
Externally publishedYes

Cite this