Evaluation Metrics

Detection Precision : evaluating the precision of 3D object detection with PASCAL criteria.
Easy:
- Minimum bounding box height: 40 Px
- Maximum occlusion level: fully visible
- Truncation level: non-truncated
Moderate:
- Minimum bounding box height: 25 Px
- Maximum occlusion level: partly occluded
- Truncation level: no limitation
Hard:
- Minimum bounding box height: 25 Px
- Maximum occlusion level: difficult to see
- Truncation level: no limitation
Communication cost: evaluating the amount of data used from infrastructure-side with number of bits.

1. Vehicle-Infrastructure Collaborative 3D Object Detection Benchmark

The 3D object detectors are trained on DAIR-V2X-V and part of DAIR-V2X-C (20481 images and 20481 point clouds). Results are validated on the rest of DAIR-V2X-C.

Note that DAIR-V2X is currently at the invitation-only stage. Benchmark results will be announced when DAIR-V2X is available for public download.

2. Infrastructure-side 3D Object Detection Benchmark

The 3D object detectors are trained on half of DAIR-V2X-I (5042 images and 5042 point clouds). Results are validated on the rest half of DAIR-V2X-I (5042 images and 5042 point clouds).

Note that DAIR-V2X is currently at the invitation-only stage. Benchmark results will be announced when DAIR-V2X is available for public download.

3. Vehicle-side 3D Object Detection Benchmark

The 3D object detectors are trained on part of DAIR-V2X-V (12325 images and 12325 point clouds). Results are validated on the rest of DAIR-V2X-V (10000 images and 10000 point clouds).

Note that DAIR-V2X is currently at the invitation-only stage. Benchmark results will be announced when DAIR-V2X is available for public download.