The paper presents the results of perceptual experiments (by humans) and automatic recognition of the emotional states of children with Down syndrome (DS) by video, audio and text modalities. The participants of the study were 35 children with DS aged 5–16 years, and 30 adults – the participants of the perceptual experiment. Automatic analysis of facial expression by video was performed using FaceReader software runs on the Microsoft Azure cloud platform and convolutional neural network. Automatic recognition of the emotional states of children by speech was carried out using a recurrent neural network. Specifically for this project, we did not apply any additional transfer learning or fine-tuning as our goal was to investigate how the generic models perform for children with DS. The results of perceptual experiments showed that adults recognize the emotional states of children with DS by video better than by audio. Automatic classification of children’s emotional states by facial expression revealed better results for joy and neutral states than for sadness and anger; by audio the best results were shown for the neutral state, by the texts of children’s speech - for joy, the state of sadness was not recognized automatically. The study revealed the possibility of using the available software for classifying the neutral state and the state of joy, i.e. states with neutral and positive valence, and the need to develop an approach to determine the state of sadness and anger.