TY - JOUR

T1 - Artionyms and machine learning: auto naming of the paintings

AU - Алтынова, Анна Юрьевна

AU - Колычева, Валерия Андреевна

AU - Григорьев, Дмитрий Алексеевич

AU - Семенов , Александр

PY - 2024/12/1

Y1 - 2024/12/1

N2 - Image captioning is a question of great interest in a wide range of applications. In the art market there is a particularly acute shortage of specialized machine learning methods for accelerated and at the same time in-depth study of often too specific aspects of art. One of the main difficulties is caused by ambiguous names of art works, as well as clarifying (in practice, often complicating understanding and perception) signatures of the authors to them. Although previous research has established that captioning of photos can be done with high efficacy, there is little published data about generation of captions for artistic paintings. In this research, we utilize a transformer architecture to generate an artionym for a given painting in author’s manner. We describe the model and report its performance on different art styles. We assess the model performance with an expert evaluation and image captioning metrics, and then discuss their capacity to analyze art-related names.

AB - Image captioning is a question of great interest in a wide range of applications. In the art market there is a particularly acute shortage of specialized machine learning methods for accelerated and at the same time in-depth study of often too specific aspects of art. One of the main difficulties is caused by ambiguous names of art works, as well as clarifying (in practice, often complicating understanding and perception) signatures of the authors to them. Although previous research has established that captioning of photos can be done with high efficacy, there is little published data about generation of captions for artistic paintings. In this research, we utilize a transformer architecture to generate an artionym for a given painting in author’s manner. We describe the model and report its performance on different art styles. We assess the model performance with an expert evaluation and image captioning metrics, and then discuss their capacity to analyze art-related names.

KW - Image captioning

KW - Machine learning in art

KW - Text generation

KW - Transformers

KW - Visual art

UR - https://www.mendeley.com/catalogue/6beeff31-cc40-3b12-8194-d7776aa4c006/

U2 - 10.11591/ijai.v13.i4.pp4445-4452

DO - 10.11591/ijai.v13.i4.pp4445-4452

M3 - Article

VL - 13

SP - 4445

EP - 4452

JO - IAES International Journal of Artificial Intelligence

JF - IAES International Journal of Artificial Intelligence

SN - 2089-4872

IS - 4

ER -