Text and Program Analysis (TAPAS)

This is the home page of the TAPAS consortium which concentrates on developing methods and tools for free text and program text analysis.

TAPAS is a collaborative project between the Laboratory of Software Technology at the Helsinki University of Technology (TKK) and the Department of Computer Science at the University of Joensuu (UJ). The aim is to develop novel assessment methods by combining the strengths of the educational technology research groups from the two universities. The main goals are:

• to develope advanced methods and tools for general purpose text evaluation,
• to apply and tune the new methods to evaluate texts in programming education, and
• to evaluate the new methods with large groups of students.

The first part, development of free text analysis methods, will be carried out at UJ. That includes methods for content and structural analysis of texts, semi-automatic features such as automated feedback and summary generation, and versatile measures to evaluate the quality of text, such as structure, coherence and cohesion. Applying these methods to programming texts will be done in both universities. Goals of this part include linking textual descriptions (specifications, definitions, documentation) to graphical specifications (e.g. UML diagrams) and program code structures, developing methods for analyzing semi-formal representations, such as pseudo-code, and developing feedback mechanisms for both students and teachers. The final evaluation part will take place at TKK on courses with hundreds of students. The performance of the new system will be compared to human evaluations, the effect on student submission quality will be measured, a demographic or cluster analysis will be carried out, and changes in teachers' workload will be analyzed.

The scientific output of the project will include improved methodology for free text analysis, including detecting coverage of relevant topics in essays, automated summary generation and structural analysis of texts. Methodology concerning programming texts include analysis methods of pseudo-code descriptions. The outcomes of the evaluation studies will indicate how formative automatic feedback for students as well as assessment aids for teachers could be applied in practice. All scientific results will be reported on papers in international workshops, conferences and journals.

The practical results of the project will include general purpose tools for automatic analysis and feedback on free text as well as texts related to programming. They will be tested extensively on large scale courses, which means they are readily available for all participants of the research project and external partners.

Background