Turbo-DDCM: Fast and Flexible Zero-Shot
Diffusion-Based Image Compression

Amit Vaisman1
Guy Ohayon2
Hila Manor1
Michael Elad1
Tomer Michaeli1

1Technion – Israel Institute of Technology 2Flatiron Institute, Simons Foundation

Paper
Code
🤗 Demo
Our method provides reconstructions with equal or better fidelity compared to previous methods, while being much faster.
At the same BPP and runtime, the priority-aware variant (bottom-right) better-serves key regions of choice.
▶ Start Round-Trip Runtime Animation

Abstract

While zero-shot diffusion-based compression methods have seen significant progress in recent years, they remain notoriously slow and computationally demanding. This paper presents an efficient zero-shot diffusion-based compression method that runs substantially faster than existing methods, while maintaining performance that is on par with the state-of-the-art techniques. Our method builds upon the recently proposed Denoising Diffusion Codebook Models (DDCMs) compression scheme. Specifically, DDCM compresses an image by sequentially choosing the diffusion noise vectors from reproducible random codebooks, guiding the denoiser’s output to reconstruct the target image. We modify this framework with Turbo-DDCM, which efficiently combines a large number of noise vectors at each denoising step, thereby significantly reducing the number of required denoising operations. This modification is also coupled with an improved encoding protocol. Furthermore, we introduce two flexible variants of Turbo-DDCM, a priority-aware variant that prioritizes user-specified regions and a distortion-controlled variant that compresses an image based on a target PSNR rather than a target BPP. Comprehensive experiments position Turbo-DDCM as a compelling, practical, and flexible image compression scheme.

Turbo-DDCM Key Properties

Turbo-DDCM relies on a pre-trained latent diffusion generative model, without any need for further training or fine-tuning.
As such, the backbone diffusion model can be replaced flexibly to allow improved future versions.

Turbo-DDCM has a competitive performance with the current state-of-the-art methods in terms of the rate-distortion-perception tradeoff.

compression comparison
Comparison of compression performance on the Kodak24 dataset (images center-cropped to 512x512). The four subplots detail performance against zero-shot diffusion-based methods (plots on the left) and against training-based, fine-tuning-based, and non-neural image compression methods (plots on the right).

Turbo-DDCM is the fastest zero-shot method, achieving up to an order of magnitude speedup over the fastest existing approach.
Our method operates without any custom hardware-specific acceleration (unlike DiffC) and maintaining nearly constant runtime across bitrates (unlike to DDCM and DiffC).

runtimes

Note: PSC is omitted due to its extreme complexity (>300 s/image).

Our method provides a predictable and constant bitrate across images (like to DDCM but unlike DiffC), which can be finely controlled via a single hyperparameter across a wide range of bitrates (like DiffC but unlike DDCM).

Turbo-DDCM Variants

Priority-Aware Variant

This variant enables enhanced reconstruction fidelity in user-selected regions of the target image, with controllable prioritization levels.

Note: All methods except Turbo-DDCM Priority-Aware are non-priority-aware. In the prioritization masks shown for our priority-aware variant, black represents no prioritization, gray indicates medium prioritization, and white denotes high prioritization.

Distortion-Controlled Variant

This variant targets a specific PSNR per image (instead of bitrate), addressing the variable distortion in zero-shot methods at fixed bitrate.

Image 1
All zero-shot methods exhibit high distortion variability across images when evaluated at the same bitrate.
Image 2
Our method brings the actual distortion much closer to the target, reducing RMSE by over 40% for most target PSNRs, compared to a naive approach.

Additional Results

BibTex

Acknowledgements

The format of this webpage was originally made by Matan Kleiner with the help of Hila Manor. The code for the original template can be found here.
Icons are taken from font awesome or from Academicons.