TY - JOUR

T1 - Автоматизация распознавания сложной текстовой CAPTCHA с применением условной генеративно-состязательной нейронной сети

AU - Zadorozhnyy, A.S.

AU - Korepanova, A.A.

AU - Abramov, M.V.

AU - Sabrekov, A.A.

N1 - Export Date: 11 March 2024 Адрес для корреспонденции: Abramov, M.V.; Saint Petersburg Federal Research Center of the Russian Academy of SciencesRussian Federation; эл. почта: mva@dscs.pro Текст о финансировании 1: The work was carried out within the framework of the project under the state assignment of SPC RAS SPIIRAS no. FFZF-2022-0003. Пристатейные ссылки: Korepanova, A.A., Bushmelev, F.V., Sabrekov, A.A., Node.js parsing technologies in the task of aggregating information and evaluating the parameters of cargo routes by extracting data from open sources (2021) Computer Tools in Education Journal, (3), pp. 41-56. , https://doi.org/10.32603/2071-2340-2021-3-41-56, Корепанова А.А., Бушмелев Ф.В., Сабреков А.А. Технологии парсинга на Node.js в задаче агрегации сведений и оценки пара-метров грузовых маршрутов посредством извлечения данных из открытых источников Компьютерные инструменты в образо-вании. 2021. 3. C. 41–56. https://doi.org (in Russian); Zi, Y., Gao, H., Cheng, Z., Liu, Y., An end-to-end attack on text CAPTCHAs (2019) IEEE Transactions on Information Forensics and Security, 15, pp. 753-766. , https://doi.org/10.1109/TIFS.2019.2928622, P. https://doi.org/10.1109/TIFS.2019.2928622 Zi Y., Gao H., Cheng Z., Liu Y. An end-to-end attack on text CAPTCHAs. IEEE Transactions on Information Forensics and Security, 2019, 15, 753–766; Noury, Z., Rezaei, M., Deep-CAPTCHA: A deep learning based CAPTCHA solver for vulnerability assessment (2020) ERN: Neural Networks & Related Topics (Topic), , https://doi.org/10.2139/ssrn.3633354, https://doi.org/10.2139 ssrn.3633354 Noury Z., Rezaei M. Deep-CAPTCHA: A deep learning based CAPTCHA solver for vulnerability assessment. ERN: Neural Networks & Related Topics (Topic), 2020; Sahil Ahmed, S., Anand, K.M., Convolution neural network-based CAPTCHA recognition for indic languages (2021) Advances in Intelligent Systems and Computing, 1407, pp. 493-502. , P. https://doi.org/10.1007/978-981-16-0171-2_46 Sahil Ahmed S., Anand K.M. Convolution neural network-based CAPTCHA recognition for indic languages. Advances in Intelligent Systems and Computing, 2021, 1407, 493–502. https://doi.org/10.1007/978-981-16-0171-2_46; Lu, S., Huang, K., Meraj, T., Rauf, H.T., A novel CAPTCHA solver framework using deep skipping Convolutional Neural Networks (2022) PeerJ Computer Science, 8, p. e879. , P. https://doi.org/10.7717 peerj-cs.879 Lu S., Huang K., Meraj T., Rauf H.T. A novel CAPTCHA solver framework using deep skipping Convolutional Neural Networks. PeerJ Computer Science, 2022, 8, e879. https://doi.org; Wang, Z., Shi, P., CAPTCHA recognition method based on CNN with focal loss (2021) Complexity, 2021, p. 6641329. , P. https://doi.org/10.1155/2021/6641329 Wang Z., Shi P. CAPTCHA recognition method based on CNN with focal loss. Complexity, 2021, 2021, 6641329. https://doi.org; Chen, J., Luo, X., Zhu, L., Zhang, Q., Gan, Y., Handwritten CAPTCHA recognizer: a text CAPTCHA breaking method based on style transfer network (2023) Multimedia Tools and Applications, 82 (9), pp. 13025-13043. , https://doi.org/10.1007/s11042-021-11485-9ChenJ.,LuoX.,ZhuL.,ZhangQ.,GanY, N P. Handwritten CAPTCHA recognizer: a text CAPTCHA breaking method based on style transfer network. Multimedia Tools and Applications, 2023, 82 9, 13025–13043. https://doi.org; Bostik, O., Horak, K., Kratochvila, L., Zemcik, T., Bilik, S., Semisupervised deep learning approach to break common CAPTCHAs (2021) Neural Computing and Applications, 33 (20), pp. 13333-13343. , https://doi.org/10.1007/s00521-021-05957-0, N P. https://doi.org/10.1007/s00521-021-05957-0 Bostik O., Horak K., Kratochvila L., Zemcik T., Bilik S. Semisupervised deep learning approach to break common CAPTCHAs. Neural Computing and Applications, 2021, 33 20, 13333 13343; Le, T.A., Baydin, A.G., Zinkov, R., Wood, F., Using synthetic data to train neural networks is model-based reasoning (2017) Proc. of the 2017 International Joint Conference on Neural Networks (IJCNN), pp. 3514-3521. , https://doi.org/10.1109/IJCNN.2017.7966298, P. https://doi.org/10.1109/IJCNN.2017.7966298 Le T.A., Baydin A.G., Zinkov R., Wood F. Using synthetic data to train neural networks is model-based reasoning. Proc. of the 2017 International Joint Conference on Neural Networks (IJCNN), 2017, 3514–3521; Wang, Y., Wei, Y., Zhang, M., Liu, Y., Wang, B., Make complex captchas simple: a fast text CAPTCHA solver based on a small number of samples (2021) Information Sciences, 578, pp. 181-194. , P. https: doi.org/10.1016/j.ins.2021.07.040 Wang Y., Wei Y., Zhang M., Liu Y., Wang B. Make complex captchas simple: a fast text CAPTCHA solver based on a smal of samples. Information Sciences, 2021, 578, 181–194. https: doi.org; Li, C., Chen, X., Wang, H., Wang, P., Zhang, Y., Wang, W., End-to-end attack on text-based CAPTCHAs based on cycle-consistent generative adversarial network (2021) Neurocomputing, 433, pp. 223-236. , P. https://doi.org/10.1016/j.neucom.2020.11.057 Li C., Chen X., Wang H., Wang P., Zhang Y., Wang W. End-to-end attack on text-based CAPTCHAs based on cycle-consistent generative adversarial network. Neurocomputing, 2021, 433, 223–236. https://doi.org/10.1016/j.neucom.2020.11.057; Simonyan, K., Zisserman, A., Very deep convolutional networks for large-scale image recognition (2014), arXiv. arXiv: 1409.1556. https: doi.org/10.48550/arXiv.1409.1556 Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv, 2014, arXiv: 1409.1556. https: doi.org/10.48550/arXiv.1409.1556; Hartigan, J.A., Wong, M.A., Algorithm AS 136: A k-means clustering algorithm (1979) Journal of the Royal Statistical Society. Series C (Applied Statistics), 28 (1), pp. 100-108. , https://doi.org/10.2307/2346830, N P. https://doi.org/10.2307/2346830 Hartigan J.A., Wong M.A. Algorithm AS 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 1979, 28 1, 100–108; Khan, A., Sohail, A., Zahoora, U., Qureshi, A.S., A survey of the recent architectures of deep convolutional neural networks (2020) Artificial Intelligence Review, 53 (8), pp. 5455-5516. , https://doi.org/10.1007/s10462-020-09825-6, N P. https://doi.org/10.1007/s10462-020-09825-6 Khan A., Sohail A., Zahoora U., Qureshi A.S. A survey of the recent architectures of deep convolutional neural networks. Artificial Intelligence Review, 2020, 53 8, 5455–5516; Oliseenko, V., Abramov, M., Identification of user profiles in online social networks: a combined approach with face recognition (2021) Journal of Physics: Conference Series, 1864, p. 012119. , https://doi.org/10.1088/1742-6596/1864/1/012119, P. Oliseenko Abramov M. Identification of user profiles in online social networks: a combined approach with face recognition. Journal of Physics: Conference Series, 2021, 1864, 012119. https: doi.org; Bushmelev, F., Khlobystova, A., Abramov, M., Livshits, L., Deep machine learning techniques in the problem of estimating the expression of psychological characteristics of a social media user (2023) Studies in Systems, Decision and Control, 457, pp. 315-324. , https://doi.org/10.1007/978-3-031-22938-1_22, P. https://doi.org/10.1007/978-3-031-22938-1_22 Bushmelev F., Khlobystova A., Abramov M., Livshits L. Deep machine learning techniques in the problem of estimating the expression of psychological characteristics of a social media user. Studies in Systems, Decision and Control, 2023, 457, 315 324; Shafiq, M., Gu, Z., Deep residual learning for image recognition: a survey (2022) Applied Sciences, 12 (18), p. 8972. , https://doi.org/10.3390/app12188972, N P. https://doi.org/10.3390/app12188972 Shafiq M., Gu Z. Deep residual learning for image recognition: a survey. Applied Sciences, 2022, 12 18, 8972; Hossen, M.I., Hei, X., A low-cost attack against the hcaptcha system (2021) Proc. of the 2021 IEEE Security and Privacy Workshops (SPW), pp. 422-431. , https://doi.org/10.1109/SPW53761.2021.00061, P. https://doi.org/10.1109/SPW53761.2021.00061 Hossen M.I., Hei X. A low-cost attack against the hcaptcha system. Proc. of the 2021 IEEE Security and Privacy Workshops (SPW), 2021, 422–431; Kapoor, A., Shah, R., Bhuva, R., Pandit, T., Understanding inception network architecture for image classification: Technical Report (2020), https://doi.org/10.13140/RG.2.2.16212.35204, Kapoor A., Shah R., Bhuva R., Pandit T. Understanding inception network architecture for image classification: Technical Report, 2020. https://doi.org; Mittal, S., Kaushik, P., Hashmi, S., Kumar, K., Robust real time breaking of image CAPTCHAs using inception v3 model (2018) Proc. of the 2018 Eleventh International Conference on Contemporary Computing (IC3), pp. 1-5. , https://doi.org/10.1109/IC3.2018.8530607, P. https://doi.org/10.1109/IC3.2018.8530607 Mittal S., Kaushik P., Hashmi S., Kumar K. Robust real time breaking of image CAPTCHAs using inception v3 model. Proc. of the 2018 Eleventh International Conference on Contemporary Computing (IC3), 2018, 1–5; Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Sh, Ozair, Courville, A., Bengio, Y., Generative Adversarial Networks (2020) Communications of the ACM, 63 (11), pp. 139-144. , https://doi.org/10.1145/3422622, N P. https://doi.org/10.1145/3422622 Goodfellow I., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair Sh., Courville A., Bengio Y. Generative Adversarial Networks. Communications of the ACM, 2020, 63 11, 139–144; Mirza, M., Osindero, S., Conditional generative Adversarial Nets (2014), arXiv. arXiv: 1411.1784. https://doi.org/10.48550 arXiv.1411.1784 Mirza M., Osindero S. Conditional generative Adversarial Nets. arXiv, 2014, arXiv: 1411.1784. https://doi.org/10.48550 arXiv.1411.1784; Krizhevsky, A., Sutskever, I., Hinton, G.E., ImageNet classification with deep convolutional neural networks (2017) Communications of the ACM, 60 (6), pp. 84-90. , https://doi.org/10.1145/3065386, N P. https://doi.org/10.1145/3065386 Krizhevsky A., Sutskever I., Hinton G.E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60 6, 84–90; Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q., Densely connected convolutional networks (2017) Proc. of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261-2269. , https://doi.org/10.1109/CVPR.2017.243, P. https://doi.org/10.1109/CVPR.2017.243 Huang G., Liu Z., Van Der Maaten L., Weinberger K.Q. Densely connected convolutional networks. Proc. of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, 2261–2269; Ronneberger, O., Fischer, P., Brox, T., U-Net: Convolutional networks for biomedical image segmentation (2015) Lecture Notes in Computer Science, 9351, pp. 234-241. , https://doi.org/10.1007/978-3-319-24574-4_28, P. https://doi.org/10.1007/978-3-319-24574-4_28 Ronneberger O., Fischer P., Brox T. U-Net: Convolutional networks for biomedical image segmentation. Lecture Notes in Computer Science, 2015, 9351, 234–241; Chollet, F., Xception: Deep learning with depthwise separable convolutions (2017) Proc. of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1800-1807. , https://doi.org/10.1109/CVPR.2017.195, P. https: doi.org/10.1109/CVPR.2017.195 Chollet F. Xception: Deep learning with depthwise separable convolutions. Proc. of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, 1800–1807; He, K., Zhang, X., Ren, S., Sun, J., Deep residual learning for image recognition (2016) Proc. of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778. , https://doi.org/10.1109/CVPR.2016.90, P. https://doi.org/10.1109/CVPR.2016.90 He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition. Proc. of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 770–778; Vyatkin, A., Tulupyev, A., Automation of consistency checking of ideals of conjuncts with truth probability estimates. Information Security of Russian Regions (ISRR-2021) (2021) Proc. of the XII St. Petersburg Interregional Conference, pp. 330-332. , Вяткин А.А., Тулупьев А.Л. Автоматизация проверки непроти-воречивости идеалов конъюнктов с оценками вероятности истин-ности Информационная безопасность регионов России (ИБРР-2021): материалы XII Санкт-Петербургской межрегиональной конференции. 2021. C. 330–332. (in Russian); Vyatkin, A., Abramov, M., Kharitonov, N., Tulupyev, A., Application of tertiary structure of algebraic bayesian network in the problem of a posteriori inference (2023) Bulletin of the South Ural State University. Series “Computational Mathematics and Computer Science, 12 (1), pp. 61-88. , https://doi.org/10.14529/cmse230104, Вяткин А.А., Абрамов М.В., Харитонов Н.А., Тулупьев А.Л. Применение третичной структуры алгебраической байесовской сети в задаче апостериорного вывода Вестник Южно-Уральского го сударственного университет а. С ерия: Вычислительная математика и информатика. 2023. Т. 12. 1. C. 61–88. https://doi.org (in Russian); Vyatkin, A., Kharitonov, N., Tulupyev, A., Application of algebraic bayesian networks in handwritten character recognition (2022) Regional Informatics and Information Security. Proc. of the Anniversary XVIII St. Petersburg International Conference, pp. 538-542. , Вяткин А.А., Харитонов Н.А., Тулупьев А.Л. Применение алге-браических байесовских сетей в задаче распознавания рукопис-ных символов Региональная информатика и информационная безопасность: сборник трудов Юбилейной XVIII Санкт-Петербургской международной конференции. 2022. C. 538–542. (in Russian)

PY - 2024/2/1

Y1 - 2024/2/1

N2 - With the rapid development of Iiitemet teclmologies. the problems of network security continue to worsen. So. one of the most common methods of maintaining security and preventing malicious attacks is CAPTCHA (ftilly automated public Tuiiiig test). CAPTCHA most often consists of some kind of security code, to bypass which it is necessaiy to perfbnn a simple task, such as entering a word displayed in an image, solving a basic aritlunetic equation. etc. However, the most widely used type of CAPTCHA is still the text type. In the recent years, the development of computer vision and. in particular, neinal netvirorks has contributed to a decrease in the resistance to hacking of text CAPTCHA. However. the security and resistance to recognition of complex CAPTCHA containing a lot of noise and distortion is still insufficiently studied. This study examines CAPTCHA, the distinctive feature of which is the use of a large number of different distortions, and each individual image uses its own different set of distortions, that is why even the human eye camiot always recognize what is depicted in the photo. The puipose of this work is to assess the security of sites using the CAPTCHA text type by testing theiiresistance to an automated solution. This testing will be used for the subsequent development of recommendations for improving the effectiveness of protection mechanisms. The result of the work is an implemented synthetic generator and discriniiiiator of the CGAN architectiu e. as well as a decoder program, which is a trained convolutional neural network that solves this type of CAPTCHA. The recognition accuracy of the model constnicted in the article was 63 % on an initially veiy limited data set. which shows the infbnnation security risks that sites using a similar type of CAPTCHA can cany. © Задорожный А.С., Корепанова А.А., Абрамов М.В., Сабреков А.А., 2024.

AB - With the rapid development of Iiitemet teclmologies. the problems of network security continue to worsen. So. one of the most common methods of maintaining security and preventing malicious attacks is CAPTCHA (ftilly automated public Tuiiiig test). CAPTCHA most often consists of some kind of security code, to bypass which it is necessaiy to perfbnn a simple task, such as entering a word displayed in an image, solving a basic aritlunetic equation. etc. However, the most widely used type of CAPTCHA is still the text type. In the recent years, the development of computer vision and. in particular, neinal netvirorks has contributed to a decrease in the resistance to hacking of text CAPTCHA. However. the security and resistance to recognition of complex CAPTCHA containing a lot of noise and distortion is still insufficiently studied. This study examines CAPTCHA, the distinctive feature of which is the use of a large number of different distortions, and each individual image uses its own different set of distortions, that is why even the human eye camiot always recognize what is depicted in the photo. The puipose of this work is to assess the security of sites using the CAPTCHA text type by testing theiiresistance to an automated solution. This testing will be used for the subsequent development of recommendations for improving the effectiveness of protection mechanisms. The result of the work is an implemented synthetic generator and discriniiiiator of the CGAN architectiu e. as well as a decoder program, which is a trained convolutional neural network that solves this type of CAPTCHA. The recognition accuracy of the model constnicted in the article was 63 % on an initially veiy limited data set. which shows the infbnnation security risks that sites using a similar type of CAPTCHA can cany. © Задорожный А.С., Корепанова А.А., Абрамов М.В., Сабреков А.А., 2024.

KW - CGAN

KW - CNN

KW - conditional generative adversarial network

KW - deep learning

KW - information security

KW - text-based CAPTCHAs

UR - https://www.mendeley.com/catalogue/1b1252ff-3d9a-3565-9a74-544cb572862f/

U2 - 10.17586/2226-1494-2024-24-1-90-100

DO - 10.17586/2226-1494-2024-24-1-90-100

M3 - статья

VL - 24

SP - 90

EP - 100

JO - Scientific and Technical Journal of Information Technologies, Mechanics and Optics

JF - Scientific and Technical Journal of Information Technologies, Mechanics and Optics

SN - 2226-1494

IS - 1

ER -