Important dates
Registration Open: June 13, 2024
Training Data Online: June 13, 2024
Test Data Release: July 13, 2024
Challenge Submission Deadline: July 31, 2024
Results Announcement: August 7, 2024
Challenge Reports Deadline: August 22, 2024
Camera-Ready Papers Due: September 10, 2024
ICPR 2024 dates: December 01-05, 2024
Dataset
MWIRSTD: A Novel Mid-Wave Infrared Small Target Detection Dataset
MWIRSTD is a unique and comprehensive dataset for small target detection in mid-wave infrared (MWIR) imagery, comprising 14 video sequences with approximately 1053 images and annotated targets of three distinct classes of small objects. Captured using cooled MWIR imagers, this dataset offers a realistic opportunity for researchers to develop and evaluate state-of-the-art methods for small object detection in MWIR scenes.
The Significance of Small Targets in MWIR Imagery
Small targets in MWIR imagery are crucial for various applications, including surveillance, object tracking, and threat detection. The ability to detect and classify small targets in MWIR scenes is essential for effective decision-making in these domains. However, small targets pose significant challenges due to their limited spatial resolution, low contrast, and susceptibility to noise and clutter. The MWIRSTD dataset addresses these challenges by providing a diverse range of small targets in realistic MWIR environments.
Dataset Collection and Preprocessing
The MWIRSTD dataset was collected using a cooled MWIR imager installed on a mountain, with environmental conditions ranging from clear to foggy. The imager captured video sequences at a frame rate of 25 frames per second, with a slant distance between the object and the imager ranging from 300 meters to 950 meters. The video sequences were preprocessed by converting the analog output to digital format using a frame grabber card, and resizing the frames to 509 × 655 pixels to exclude unnecessary information.
Ground Truth and Annotations
The ground truth data includes labels for moving ground vehicles, cracker rockets, and debris resulting from these rockets. The dataset consists of three classes of annotations: cracker rockets, debris, and other moving targets. The annotations are provided in the form of semantic maps, with detailed descriptions of the content in each of the 14 recorded sequences.
For more information, please refer to the dataset paper: https://arxiv.org/html/2406.08063v1.
To download the dataset and take part in the competition please, fill the registration form.
Evaluation Metric
MWIRSTD Evaluation Metric:
The MWIRSTD evaluation metric is designed to assess the performance of small target detection algorithms in mid-wave infrared (MWIR) imagery. The metric consists of three components:
- Intersection over Union (IoU): Measures the overlap between the predicted bounding box and the ground truth bounding box.
IoU = (Area of Overlap) / (Area of Union)
where Area of Overlap is the intersection of the predicted and ground truth bounding boxes, and Area of Union is the union of the predicted and ground truth bounding boxes.
- Probability of Detection (PD): Measures the ratio of correctly detected pixels as the target to the total number of pixels detected as a target.
PD = (Number of Correctly Detected Pixels) / (Total Number of Pixels Detected as Target)
- False Alarm Rate (FAR): Measures the ratio of false positive pixels to the total number of pixels detected as a target.
FAR = (Number of False Positive Pixels) / (Total Number of Pixels Detected as Target)
Final Score:
The final score is a weighted sum of the three components:
Final Score = 0.4 * IoU + 0.3 * PD + 0.3 * (1 – FAR)
The weights are chosen to emphasize the importance of detection accuracy, while also considering precision and false alarm rate.
Participation Link
Registration will be done through the below link.
Please upload signed EULA form on the Google form link as given above. Click on the below link to download the EULA form:
Once you complete the registration, you will receive a mail from the organizers containing link to download the dataset. If you do not receive the dataset link and participation confirmation, kindly write a mail to avinres2@gmail.com or wicv2024@visualclpl.com.
Challenge Rules
The challenge is open for participants from industry and academia, who wants to make a contribution to the field of Infrared Computer vision. Below we outline the challenge rules.
- Each participant team can include up to a maximum of 5 people from one or more affiliations. For the sake of fairness to smaller research groups, we will not allow bigger teams to participate as a single team.
- One person can only participate on one team. Mentors included. No exceptions.
- Winners teams will have certificates listing the names of their members in the exact format and order as they were registered.
- We will only allow registrations to be updated until the date of the release of the testing dataset. Changes allowed before this date include adding or removing members, correcting typos on names, as well as updating the order of the team members which will be used for the certificate in the case of winners.
- Participants can register to participate in individual tasks only. However, winners will be selected based on their task coverage. We will be using a scoring system.
- Participants will be allowed to use only datasets provided for training.
- Submissions will be done as per the instructions given on the website.
- CODE submission will be required.
- Participants must provide a description of the methods used to produce the results submitted. In the final competition paper, we will summarize these descriptions when we describe the submitted systems. We reserve the right to disqualify submissions that do not provide a sufficiently detailed description of their system.
- To be fair to all participants, any deadline extensions given will apply to all participants, not just to individual research groups who might request them.