The Web Intelligence - Classification Challenge
We are pleased to announce the outcome of the Classification Challenge – the second round of European Statistics Awards on web intelligence.
In June 2024, Eurostat launched the second round of competitions – the Classification Challenge - in the field of web intelligence with the primary goal of unveiling innovative methodologies and valuable data resources that could improve the production of European statistics.
We thank all participants for having contributed to the success of this second round through their submissions.
The evaluation of the Web Intelligence Classification Challenge is now complete, and we are happy to announce the winners of the Accuracy, Reusability and Innovativity Awards!
Congratulations to all the winners!
Web Intelligence – 2nd round – the classification of occupations for online job advertisements challenge
The European Statistics Awards Programme Web Intelligence competitions aim at stimulating innovation when retrieving data from the world wide web for producing European statistics. The Classification Challenge focused on developing approaches that learn how to assign a class label (from a known taxonomy) to job advertisements from a given dataset. Classification is a necessary condition to produce high quality statistics from online job advertisements as it ensures granular analysis of in-demand skills, occupations, and regional labour market variations, enables the timely tracking of shifts in employer demand, and ensures overall reliability of labour market indicators.
The competition dataset contained 26 000 online job advertisements, retrieved from around 400 websites active in the European Union. The source of the original dataset is the European Web Intelligence Hub (WIH), wherein around 200 million online job advertisements have been collected and classified since July 2018.
The participants had to provide documented scripts in either R or Python that would allow the identification of occupation for a specific job advertisement. They had to address a number of challenges including developing classification approaches within a multilingual dataset by applying cross-linguality techniques. Handling cross-linguality is a specifically important task since job advertisers may publish job advertisements in different languages on different job portals. The Classification Challenge focused on the coding of scripts, the development of methodologies and algorithms, applied in order to develop classification approaches and its robustness for generic use, on identically structured arbitrarily chosen datasets.
The Classification Challenge was launched in June 2024, with a final deadline for submissions of classifications in September 2024 and documentation in October 2024.
A total of 69 teams, comprised of 137 individuals from 17 countries, signed up for this challenge, with 42 teams following through by submitting solutions for evaluation. The results of the evaluation are announced below.
The participants were competing for three types of awards:
- Accuracy – aimed at reflecting the correct classification of job advertisements from the provided multilingual data, evaluated against a gold standard.
- Reusability – aimed at rewarding submissions which show great potential to be scaled up to European statistics production. The Reusability award is intended to support the most thoroughly described and documented solutions, with the most scalable, well described and open approach.
- Innovativity – aimed at rewarding submissions which show the most originality in their approach. The Innovativity award is intended to encourage cutting-edge solutions that go beyond current best methods.
Accuracy Award winners
| Place | Prize | Team name | Team members | Country |
|---|---|---|---|---|
| 1st place | 10 000 EUR | TNO_AI_LAB |
Calvin Ge Gino Kalkman Xavier Sa Castro Pinho Sadegh Shahmohammadi |
Netherlands |
| 2nd place | 5 000 EUR | FVNWL |
Hong-Hanh Nguyen-Le Van-Tuan Tran Quang-Tien Tran Thang-Long Nguyen-Ho |
Ireland |
| 3rd place | 3 000 EUR | TheClassifiers |
Jannic Cutura Dimitris Petridis Georgios Kanellos |
Germany |
Reusability Award winners
| Place | Prize | Team name | Team members | Country |
|---|---|---|---|---|
| 1st place | 10 000 EUR | TheClassifiers |
Jannic Cutura Dimitris Petridis Georgios Kanellos |
Germany |
| 2nd place | 5 000 EUR | Classy |
Lucas Ng Bin Shen Jeremy Lim Wei Yang Roger Erh Kang Jin |
Singapore |
| 3rd place | 3 000 EUR | TNO_AI_LAB |
Calvin Ge Gino Kalkman Xavier Sa Castro Pinho Sadegh Shahmohammadi |
Netherlands |
Innovativity Award winners
| Place | Prize | Team name | Team members | Country |
|---|---|---|---|---|
| 1st place | 5 000 EUR | TheClassifiers |
Jannic Cutura Dimitris Petridis Georgios Kanellos |
Germany |
| 2nd place | 3 000 EUR | TNO_AI_LAB |
Calvin Ge Gino Kalkman Xavier Sa Castro Pinho Sadegh Shahmohammadi |
Netherlands |
| 3rd place | 1 000 EUR | FVNWL |
Hong-Hanh Nguyen-Le Van-Tuan Tran Quang-Tien Tran Thang-Long Nguyen-Ho |
Ireland |
Since the Reusability and Innovativity awards both required thoroughly described and documented solutions we are happy to share the solutions by all winners:
Solutions of the Reusability Award winners
- TheClassifiers who placed 1st for both reusability and innovativity as well as 3rd for accuracy are sharing their solution in the following GitHub repository.
- TNO_AI_LAB who placed 1st for accuracy, 2nd for innovativity and 3rd for reusability are sharing their solution in the following GitHub repository.
- FVNWL who placed 2nd for accuracy and 3rd for innovativity are sharing their solution in the following GitHub repository.
- Classy who placed 2nd for reusability are sharing their solution in the following GitHub repository.