🛣️ Pavement Distress Explainability Dashboard

A research-grade computer vision system for road damage detection — extended with a Grad-CAM explainability layer that surfaces why the model makes predictions, not just what it predicts. Built on YOLOv8s fine-tuned on the RDD2022 Japan dataset and deployed as a 4-page interactive Streamlit dashboard.

Upgraded from a standard YOLO detection pipeline by adding Grad-CAM hooks into the backbone, per-class confidence histograms, confusion matrix breakdown by distress type, and an upload-and-explain interface — turning a demo CV project into research-grade work.

---

📸 Dashboard Preview

Overview	Grad-CAM Explorer
Dataset stats, class cards, model metrics	Side-by-side heatmap gallery for 20 test images

Analytics	Upload & Predict
Confusion matrix + confidence distributions	Upload any road photo → instant detection + heatmap

---

🔍 Explainability Insights (What Grad-CAM Revealed)

Three systematic patterns surfaced after training:

1. Confidence calibration gap on D00 longitudinal cracks The model detects longitudinal cracks at conf=0.15 but misses them at conf=0.25 — meaning it localizes the crack correctly but underestimates its own certainty. This is a calibration problem, not a feature blindness problem.

2. Diffuse attention on D10 transverse cracks Grad-CAM heatmaps show the model spreading attention across the full road surface for transverse cracks (avg confidence 0.40) instead of concentrating on crack texture — consistent with having fewer training instances than D20.

3. Domain gap sensitivity Model trained on Japanese dashcam footage at road level. Steeper camera angles (common in US infrastructure inspection photos) shift heatmap attention away from the pavement surface, reducing confidence by ~10–15%.

---

📊 Model Performance

Trained for 30 epochs on 6,320 images, evaluated on 790 test images.

Class	Precision	Recall	F1	Avg Confidence
D00 — Longitudinal Crack	0.838	0.668	0.744	0.458
D10 — Transverse Crack	0.764	0.693	0.727	0.400
D20 — Alligator Crack	0.941	0.817	0.875	0.500
D40 — Pothole	0.865	0.780	0.821	0.496
Overall mAP50	0.566	0.549	—	0.568

---

🏗️ Project Structure

pavement_explainability/
├── app/
│   └── dashboard.py              # Streamlit dashboard — 4 pages
├── utils/
│   ├── convert_voc_to_yolo.py    # Pascal VOC XML → YOLO .txt converter
│   ├── train.py                  # YOLOv8s fine-tuning script
│   ├── gradcam_inference.py      # Grad-CAM hooks + batch heatmap generation
│   └── analytics.py              # Confusion matrix, histograms, metrics JSON
├── data/
│   ├── pavement.yaml             # YOLO dataset config (4 classes)
│   ├── images/                   # train / val / test splits
│   └── labels/                   # YOLO format .txt labels
├── models/
│   └── pavement_best.pt          # Best checkpoint (not tracked in git)
├── gradcam_outputs/              # Generated heatmap PNGs
├── requirements.txt
└── README.md

---

🚀 Quick Start

1. Clone and set up environment

git clone https://github.com/DeekshithaKalluri/pavement-explainability.git
cd pavement-explainability
python3 -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt

2. Download the dataset

Download RDD2022_Japan.zip (~1 GB) and place it in data/.

cd data
unzip RDD2022_Japan.zip

3. Convert dataset format (Pascal VOC → YOLO)

python3 utils/convert_voc_to_yolo.py
# Output: 7,900 labeled images split 80/10/10

4. Train the model

python3 utils/train.py
# ~30 epochs on Apple M4 MPS, ~3 hours
# Best weights saved to models/pavement_best.pt

5. Run analytics

python3 utils/analytics.py
# Generates confusion matrix, confidence histograms, metrics.json

6. Generate Grad-CAM heatmaps

python3 utils/gradcam_inference.py
# Runs on 20 random test images, saves to gradcam_outputs/

7. Launch the dashboard

streamlit run app/dashboard.py
# Opens at http://localhost:8501

---

🧰 Tech Stack

Tool	Version	Purpose
Ultralytics YOLOv8	8.4.56	Object detection
PyTorch	2.12	Model training + gradient computation
pytorch-grad-cam	1.5.5	Grad-CAM backbone hooks
Streamlit	1.58	Interactive dashboard
OpenCV	4.13	Image preprocessing + annotation
scikit-learn	1.8	Confusion matrix
Matplotlib / Seaborn	3.10 / 0.13	Analytics charts

---

📁 Dataset

RDD2022 — Road Damage Detection 2022

• Source: sekilab/RoadDamageDetector
• Subset: Japan only (~1 GB, highest label quality)
• Format: Pascal VOC XML → converted to YOLO .txt
• Classes: D00 Longitudinal · D10 Transverse · D20 Alligator · D40 Pothole
• Split: 6,320 train / 790 val / 790 test (from 10,506 total, 2,606 skipped — no valid labels)

---

💡 Key Design Decisions

Why backbone layer 9? Layer 9 is the last C2f block in YOLOv8's backbone — it holds the richest semantic feature maps before the neck upsampling. Hooking Grad-CAM here gives spatial heatmaps that are meaningful at the object scale (cracks are small, thin structures).

Why separate inference from Grad-CAM? YOLO inference runs in torch.no_grad() for clean detections, then Grad-CAM runs a separate forward pass through a backbone-only wrapper with gradients enabled. This avoids the "tensor does not require grad" error common when hooking directly into YOLO's full forward pass.

Why YOLOv8s over YOLOv8n or YOLOv8m? Small model balances speed (viable on Apple M4 MPS without CUDA) and capacity (enough parameters to learn crack texture patterns). Nano would underfit on 4 fine-grained classes; medium adds 3x training time with marginal gain on this dataset size.

---

📄 License

MIT License — see LICENSE

---

👤 Author

Deekshitha Kalluri GitHub