What Compressed Models Keep
Compress a text-editing diffusion model by step distillation and by pruning, then check whether the students keep the teacher's behaviour, not just its sample quality.
Status: planned · encoder-level precursor in progress
The question
Compressed generators are judged by sample quality and speed. Do they also stay as consistent as their teacher under content-preserving changes, or does compression bring the shortcuts back?
The study
| Model | Role |
|---|---|
| Text-editing diffusion teacher (DiffUTE reproduction) | Reference |
| Step-distilled student | Fewer sampling steps |
| Pruned-UNet student | Smaller network |
Each model gets the standard metrics (OCR accuracy of the edited text, image quality, speed) and a consistency check under content-preserving changes (the kind that broke detectors in BMVC 2026), reported side by side.
Evidence so far
Preliminary work on encoders, not yet on generators: in the edge distillation project, I probed candidate vision teachers for scene-identity leakage and set a Raspberry Pi 5 latency and energy budget.
Related work
- Salimans & Ho, Progressive distillation for fast sampling of diffusion models, ICLR 2022.
- Song et al., Consistency models, ICML 2023.
- Kim et al., BK-SDM: a lightweight, fast, and cheap version of Stable Diffusion, ECCV 2024.
- Ojha et al., What knowledge gets distilled in knowledge distillation?, NeurIPS 2023.
- Hooker et al., Characterising bias in compressed models, arXiv 2020.