1) Choose the base network: VGG-16 (default, highest quality), AlexNet (faster, mobile), or SqueezeNet (lightest). 2) Outputs of selected convolutional layers of the network (typically 5 layers for VGG) are extracted for both compared images. 3) Activations are unit-normalised per channel per spatial position. 4) Element-wise squared L2 difference of activations is computed. 5) For each layer, differences are multiplied by a learned weight vector w (per-channel, trained on BAPPS). 6) Per-layer differences are averaged over spatial positions (spatial mean). 7) The sum across all layers gives the final LPIPS. Value range: 0 (identical) to ~1 (very different).
Classical image quality metrics such as PSNR and SSIM are simple, shallow, hand-designed functions for classical distortions (JPEG compression, Gaussian noise, blur). They cannot capture human perceptual similarity for modern artifacts (GAN outputs, super-resolution, style transfer) โ two images perceptually identical to a human can have dramatically different PSNR. An objective, reproducible similarity metric calibrated on actual human perception and operating across a broad spectrum of distortions was missing.
VGG-16: best correlation with human ranking, slower (~5-10x slower than SqueezeNet). AlexNet: compromise. SqueezeNet: lightest, for mobile and real-time applications.
"Lin" โ weights w trained on BAPPS (recommended). "Lin-tune" โ additionally fine-tuned backbone. Version without weights (raw features) works but weaker.
Typically 64x64 or 224x224 (ImageNet default). Higher resolution increases memory and time O(n^2), but does not always improve correlation with human perception โ high CNN layers have large receptive fields anyway.
Using LPIPS as a loss term requires differentiability of the whole pipeline (gradients through the CNN backbone). Using only as an evaluation metric can be no-grad (faster).
The backbone expects 3 RGB channels. For grayscale, channel replication suffices. Value range [-1, 1] (ImageNet input normalisation).
LPIPS activates the full CNN forward pass for both images โ all layers, all channels, all spatial positions. No skip-layers or sparse computation.
LPIPS computation is fully parallelisable. CNN forward pass for a batch of image pairs โ identical to classification inference. Layer-wise reduction is independent โ each layer can be computed independently. Linear scaling with batch size until the GPU is saturated.
