Information published on Internet resources constitutes an important data source for the operation of many modern services and is used in data collection, analysis, monitoring, business intelligence, and other tasks. However, the absence of APIs, the high variability of websites, and frequent changes in site structure significantly complicate development and require continuous parser maintenance. In recent years, Large Language Models (LLMs) have demonstrated high effectiveness in text analysis and program code generation tasks, opening new possibilities for automating data analysis from web resources. This paper proposes an architecture for an adaptive web data extraction system that extends the capabilities of traditional web parsing methods through the integration and adaptation of LLMs. The solution supports two operating modes: generation of Python scripts for page parsing and structuring information from page text. To optimize LLM token costs for parser generation, a semantic caching mechanism is introduced, enabling the reuse of previously generated parsers based on the retrieval of similar queries in vector space. An experimental evaluation of the system was conducted using the LiveWeb-IE benchmark. The results showed that the proposed system extracts query-relevant data with 90% accuracy. Additional caching reduces the response time for a single query by more than an order of magnitude.
Original languageEnglish
Title of host publicationProceedings of 2026 29th International Conference on Soft Computing and Measurements, SCM 2026
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages231-233
Number of pages3
ISBN (Electronic)979-8-3195-4898-6
ISBN (Print)979-8-3195-4899-3
DOIs
StatePublished - 1 May 2026
EventXXIX Международная конференция по мягким вычислениям и измерениям (SCM-2026) - Санкт-Петербургский государственный электротехнический университет «ЛЭТИ» им. В.И. Ульянова (Ленина), Санкт-Петербург, Russian Federation
Duration: 20 May 2026 → 22 May 2026
Conference number: 29
https://scm.etu.ru/2026/ru/

Conference

ConferenceXXIX Международная конференция по мягким вычислениям и измерениям (SCM-2026)
Abbreviated titleSCM'2026
Country/TerritoryRussian Federation
CityСанкт-Петербург
Period20/05/26 → 22/05/26
Internet address

    Research areas

  • Modeling, Timing, Large language models, Measurement, Printing, Codes, HTML, Costing, Costs, large language models, web parsing, automated data analysis, semantic caching, code generation

    Scopus subject areas

  • Artificial Intelligence

ID: 160342511