Automatic classification of error types in solutions to programming assignments at online learning platform

Artyom Lobanov, Timofey Bryksin, Alexey Shpilman

Research outputpeer-review

Abstract

Online programming courses are becoming more and more popular, but they still have significant drawbacks when compared to the traditional education system, e.g., the lack of feedback. In this study, we apply machine learning methods to improve the feedback of automated verification systems for programming assignments. We propose an approach that provides an insight on how to fix the code for a given incorrect submission. To achieve this, we detect frequent error types by clustering previously submitted incorrect solutions, label these clusters and use this labeled dataset to identify the type of an error in a new submission. We examine and compare several approaches to the detection of frequent error types and to the assignment of clusters to new submissions. The proposed method is evaluated on a dataset provided by a popular online learning platform.

Original languageEnglish
Title of host publicationArtificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings
EditorsSeiji Isotani, Peter Hastings, Amy Ogan, Bruce McLaren, Rose Luckin, Eva Millán
PublisherSpringer
Pages174-178
Number of pages5
ISBN (Print)9783030232061
DOIs
Publication statusPublished - 1 Jan 2019
Event20th International Conference on Artificial Intelligence in Education, AIED 2019 - Chicago
Duration: 25 Jun 201929 Jun 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11626 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference20th International Conference on Artificial Intelligence in Education, AIED 2019
CountryUnited States
CityChicago
Period25/06/1929/06/19

Fingerprint

Online Learning
Assignment
Programming
Feedback
Learning systems
Labels
Machine Learning
Education
Clustering

Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Lobanov, A., Bryksin, T., & Shpilman, A. (2019). Automatic classification of error types in solutions to programming assignments at online learning platform. In S. Isotani, P. Hastings, A. Ogan, B. McLaren, R. Luckin, & E. Millán (Eds.), Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings (pp. 174-178). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11626 LNAI). Springer. https://doi.org/10.1007/978-3-030-23207-8_33
Lobanov, Artyom ; Bryksin, Timofey ; Shpilman, Alexey. / Automatic classification of error types in solutions to programming assignments at online learning platform. Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings. editor / Seiji Isotani ; Peter Hastings ; Amy Ogan ; Bruce McLaren ; Rose Luckin ; Eva Millán. Springer, 2019. pp. 174-178 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{e807e1d0a78b48118b832ad80e5627b3,
title = "Automatic classification of error types in solutions to programming assignments at online learning platform",
abstract = "Online programming courses are becoming more and more popular, but they still have significant drawbacks when compared to the traditional education system, e.g., the lack of feedback. In this study, we apply machine learning methods to improve the feedback of automated verification systems for programming assignments. We propose an approach that provides an insight on how to fix the code for a given incorrect submission. To achieve this, we detect frequent error types by clustering previously submitted incorrect solutions, label these clusters and use this labeled dataset to identify the type of an error in a new submission. We examine and compare several approaches to the detection of frequent error types and to the assignment of clusters to new submissions. The proposed method is evaluated on a dataset provided by a popular online learning platform.",
keywords = "Automatic evaluation, Classification, Clustering, MOOC, Programming",
author = "Artyom Lobanov and Timofey Bryksin and Alexey Shpilman",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-23207-8_33",
language = "English",
isbn = "9783030232061",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer",
pages = "174--178",
editor = "Seiji Isotani and Peter Hastings and Amy Ogan and Bruce McLaren and Rose Luckin and Eva Mill{\'a}n",
booktitle = "Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings",
address = "Germany",

}

Lobanov, A, Bryksin, T & Shpilman, A 2019, Automatic classification of error types in solutions to programming assignments at online learning platform. in S Isotani, P Hastings, A Ogan, B McLaren, R Luckin & E Millán (eds), Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11626 LNAI, Springer, pp. 174-178, Chicago, 25/06/19. https://doi.org/10.1007/978-3-030-23207-8_33

Automatic classification of error types in solutions to programming assignments at online learning platform. / Lobanov, Artyom; Bryksin, Timofey; Shpilman, Alexey.

Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings. ed. / Seiji Isotani; Peter Hastings; Amy Ogan; Bruce McLaren; Rose Luckin; Eva Millán. Springer, 2019. p. 174-178 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11626 LNAI).

Research outputpeer-review

TY - GEN

T1 - Automatic classification of error types in solutions to programming assignments at online learning platform

AU - Lobanov, Artyom

AU - Bryksin, Timofey

AU - Shpilman, Alexey

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Online programming courses are becoming more and more popular, but they still have significant drawbacks when compared to the traditional education system, e.g., the lack of feedback. In this study, we apply machine learning methods to improve the feedback of automated verification systems for programming assignments. We propose an approach that provides an insight on how to fix the code for a given incorrect submission. To achieve this, we detect frequent error types by clustering previously submitted incorrect solutions, label these clusters and use this labeled dataset to identify the type of an error in a new submission. We examine and compare several approaches to the detection of frequent error types and to the assignment of clusters to new submissions. The proposed method is evaluated on a dataset provided by a popular online learning platform.

AB - Online programming courses are becoming more and more popular, but they still have significant drawbacks when compared to the traditional education system, e.g., the lack of feedback. In this study, we apply machine learning methods to improve the feedback of automated verification systems for programming assignments. We propose an approach that provides an insight on how to fix the code for a given incorrect submission. To achieve this, we detect frequent error types by clustering previously submitted incorrect solutions, label these clusters and use this labeled dataset to identify the type of an error in a new submission. We examine and compare several approaches to the detection of frequent error types and to the assignment of clusters to new submissions. The proposed method is evaluated on a dataset provided by a popular online learning platform.

KW - Automatic evaluation

KW - Classification

KW - Clustering

KW - MOOC

KW - Programming

UR - http://www.scopus.com/inward/record.url?scp=85068320560&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-23207-8_33

DO - 10.1007/978-3-030-23207-8_33

M3 - Conference contribution

AN - SCOPUS:85068320560

SN - 9783030232061

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 174

EP - 178

BT - Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings

A2 - Isotani, Seiji

A2 - Hastings, Peter

A2 - Ogan, Amy

A2 - McLaren, Bruce

A2 - Luckin, Rose

A2 - Millán, Eva

PB - Springer

ER -

Lobanov A, Bryksin T, Shpilman A. Automatic classification of error types in solutions to programming assignments at online learning platform. In Isotani S, Hastings P, Ogan A, McLaren B, Luckin R, Millán E, editors, Artificial Intelligence in Education - 20th International Conference, AIED 2019, Proceedings. Springer. 2019. p. 174-178. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-23207-8_33