Crowdsensing-based Road Damage Detection Challenge (CRDDC2022)

Overview

Latest updates

  • August 4, 2022: Main Task (for phase 3) has been announced. The data will be released on August 12th, 2022.
  • August 4, 2022: Results for Data Contribution phase has been declared. Congratulations to the winners!
  • July 1, 2022: The deadline for phase 1 submissions is extended to July 20, 2022.
  • May 30, 2022: Web site of the challenge opens, the task is revealed.

Description

Considering that roads have a direct and significant impact on people's lives, maintenance and management of roads need to be done exhaustively from time to time. However, a lack of financial resources makes many local governments unable to conduct sufficient inspections on time. Some municipalities automate road damage detection by using high-performance sensors. Nevertheless, the high cost of these sensors makes it infeasible to use them at the country level owing to the vast area of roads to be inspected

Therefore, there arises a need for a system that makes it easy to assess the road conditions and identify the damage to the road surface at a low cost. Recognizing this need, the first Road Damage Detection Challenge was organized as an IEEE Big Data Cup in 2018 to evaluate the contemporary methods working towards the same goal.

Another challenge (Global Road Damage Detection Challenge, GRDDC’2020) was organized in 2020 to address the requirements of multiple countries. GRDDC utilized data from India, Japan, and Czech Republic. More details may be accessed at (https://ieeexplore.ieee.org/abstract/document/9377790)

This year, another challenge is here with new opportunities. In previous challenges, the participants were restricted to using only the challenge dataset to train the models. This year, this constraint has been eliminated by allowing participants to develop/propose their datasets. The central theme of the challenge this year is "Crowdsensing," – which refers to the crowdsourcing of sensor data collected as per the choice of contributors.

Anyone having the dataset may register as a data contributor, data recommender, or information provider by participating in the first two phases of the challenge. After a suitability analysis, the selected datasets will be added officially to the CRDDC'2022 dataset and made available to the participants for phase 3.

Successful implementation of this challenge would open new possibilities where just the smartphones and drive cameras would be enough for road inspections, not for a single country but all countries worldwide.

The Task

The challenge will be organized in four phases with different tasks for the participants. The details are as follows.

Phase 1: Data Solicitation: The participants need to submit a report of their proposed road damage dataset (cracks, potholes, or any other damage categories) by providing the details through the corresponding forms. Following categories of participants are considered:

  1. Data Contributors (Data owners who wish to contribute their data to CRDDC)
  2. Information Providers (Data owners who wish to participate by providing only the information related to their dataset, keeping the data private)
  3. Data Recommenders (Researchers/Academicians/Freelancers.... who wants to recommend the inclusion of others' datasets to CRDDC)
Please refer to Submission section to access the forms.

Phase 2: Data Shortlisting: Based on the reports submitted in phase 1, the shortlisted teams need to submit a link to their proposed dataset. After verification, the winners (data contributors) of phase 1 will be announced.

The solicited datasets will be added to the Challenge datasets for the next phase of the challenge.

Phase 3: Main Task: Use any combination of the CRDDC data and train road damage detection and classification models considering Objective 1, Objective 2 or both:

  • Objective 1: Maximize accuracy (F1-Score) for individual countries (India, Japan, US, Norway)
  • Objective 2: Maximize accuracy (F1-Score) for multiple countries (India, Japan, Czech, US, Norway, China).

The participants may propose models targetting one or multiple countries, concerning Objective 1 and 2. For instance - Model A for Country 1, Model B for Country 2, Model C for Countries 3 & 4, or a single model for all countries may be proposed. Average of score for all the leaderboards will be used to determine the winners.
Further, the challenge imposes no restrictions on the type of submission and welcome all the novel algorithms, techniques that are currently under review, and methods that have already been published. A summary of the models proposed by the participants of previous Big Data Cup (GRDDC'2020) may be accessed at (https://ieeexplore.ieee.org/abstract/document/9377790).

Phase 4: Report and Source Code: In phase 4, the participants need to submit a report of their proposed solution along with the source code. The submission will be used to finalize the winners

Note: After the competition phase is completed, a link for submitting the accompanying academic paper will be provided to the top 10 participants (may be increased based on the quality of submissions) as ranked by the public/private leaderboard weighting. For more details, please refer to the Submissions section.

Useful Links related to previous Road Damage Detection Challenge (GRDDC’2020):

  1. GitHub Website for latest news and updates
  2. ResearchGate Project for the latest publication updates
  3. GRDDC Summary Paper – IEEE Conference Proceedings
  4. GRDDC Summary Paper – Preprint – Open Access
  5. GRDDC Workshop – Challenge Overview Video
  6. GRDDC Workshop – Image Gallery()
  7. Data Article for RDD2020 (Open Access)
  8. RDD2020 Dataset on Mendeley
  9. Other Related Articles( Article 1: RDD2020,   Article 2: RDD2019,   Article 3: RDD2018)

Citations

If you use the resources (data, content,…), please cite the following:

  1. Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Mraz, A., Kashiyama, T., & Sekimoto, Y. (2021). Deep learning-based road damage detection and classification for multiple countries. Automation in Construction, 132, 103935. 10.1016/j.autcon.2021.103935
  2. Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., & Sekimoto, Y. (2021). RDD2020: An annotated image dataset for Automatic Road Damage Detection using Deep Learning. Data in Brief, 107133. <https://doi.org/10.1016/j.dib.2021.107133>.
  3. Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Omata, H., Kashiyama, T., & Sekimoto,Y. (2020). Global Road Damage Detection: State-of-the-art Solutions. IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 2020, pp. 5533-5539, doi: 10.1109/BigData50022.2020.9377790.
  4. Maeda, H., Kashiyama, T., Sekimoto, Y., Seto, T., & Omata, H. (2021). Generative adversarial network for road damage detection. Computer‐Aided Civil and Infrastructure Engineering, 36(1), 47-60.

Important Dates

This competition will continue to be on the site.

Tentative Timeline:

Phase Description Timeline
Phase 1 Data Solicitation May 30, 2022 – July 20, 2022
Phase 2 Data Shortlisting & verification July 1, 2022 – July 31, 2022
Phase 3 Main Task Execution August 12, 2022 – September 20, 2022
Phase 4 Report Submission September 21, 2022 - September 30, 2022

**** It may be noted that the participants who do not have any data to contribute may choose to participate directly in Phase 3 or may suggest the inclusion of data available from others to participate in Phase I (as Data Recommenders). For those already having their data (private or public), it would be an added benefit to release the data through the CRDDC since the data would reach a global audience through a well-established platform. ****

Important dates: (Updated)

Date Description
May 30, 2022 Start of the competition, Website of the challenge opens, the task is revealed
July 20, 2022 Extended Deadline to submit data/information for Phase I (Data Solicitation)
July 31, 2022 Extended Deadline for Phase II
(Data Verification of teams shortlisted in Phase I)
August 4, 2022 Main task announcement (Phase III starts)
August 12, 2022 Training Data becomes avaiable.
August 20, 2022 Submissions for Phase 3 starts.
Sept 25, 2022 Deadline for submitting the source code & the solutions,
End of the competition
Oct 5, 2022 Announcement of winning teams,
Sending invitations for submitting papers for the special track at the IEEE BigData 2022 conference.
Oct 25, 2022 Deadline for submitting invited papers
Nov 10, 2022 Notification of Paper acceptance
Nov 20, 2022 Camera-ready of accepted papers due
Dec 17-20, 2022 The IEEE BigData 2022 conference (special track date for CRDDC: TBA)