---
title: "L-DPT: Scaling Dense Prediction with Latent Decoding"
description: L-DPT decodes dense predictions in a compact latent space and reads out to pixels with a lightweight operator. Trained at 768 × 768, it runs on 100-megapixel photographs. ECCV 2026.
image: https://xywu.me/l-dpt/assets/brand/og.jpg
---

[L-DPT](https://pointcept.org/l-dpt#top)

[Compare](https://pointcept.org/l-dpt#compare) [Scenes](https://pointcept.org/l-dpt#scenes) [Efficiency](https://pointcept.org/l-dpt#efficiency) [Method](https://pointcept.org/l-dpt#method) [Results](https://pointcept.org/l-dpt#results) [Code](https://pointcept.org/l-dpt#code) [Cite](https://pointcept.org/l-dpt#citation)

ECCV 2026

# Scaling Dense Prediction with Latent Decoding

Train at 768. Run at *100 megapixels*.

[Xiaoyang Wu1](https://xywu.me) Yixing Lao1 Chengyao Wang2 Senqiao Yang2 Yujia Zhang1 [Hengshuang Zhao1](https://hszhao.github.io)

1The University of Hong Kong2The Chinese University of Hong Kong

Papersoon arXivsoon Codesoon 🤗Weightssoon Demosoon [See the comparison](https://pointcept.org/l-dpt#compare)

Video

[BibTeX](https://pointcept.org/l-dpt#citation)

L-DPT keeps the dense decoder on the ViT's compact latent grid and reaches pixels with a lightweight readout. Trained only on 768 × 768 crops, it predicts depth for 100-megapixel photographs, and fine structure stays sharp as resolution grows instead of blurring.

![](https://xywu.me/l-dpt/assets/hero/poster_rgb.webp) ![](https://xywu.me/l-dpt/assets/hero/poster_depth.webp) RGBL-DPT · relative depth 102 MP · Hasselblad

this square is one training crop.  
the photo holds 173 of them.

The dashed square is 768 by 768 pixels, the size of one training crop. Left and right arrow keys move the divider between photo and depth. Enter flies into the window and back.

Fly into the 768² window

**8750 × 11664** · 102.1 MP · Hasselblad

[768² → 100 MP Trained on 0.59-megapixel crops, run on 100-megapixel photographs: 173 times the training area. Compare with DPT →](https://pointcept.org/l-dpt#compare) [15.3 GB at 64 MP Decoder peak memory, with 5.3 s latency. The DPT decoder already needs 59.4 GB and 394 s at 31 MP. Efficiency sweep →](https://pointcept.org/l-dpt#efficiency) [23M vs 230M Trainable decoder parameters. With the same frozen DINOv3 encoder, L-DPT-S beats DPT on DIODE (δ1 84.4 vs 82.2). Depth results →](https://pointcept.org/l-dpt#results) [98M vs 927M Decoder parameters. L-DPT-B matches DINOv3's segmenter on COCO-Stuff (53.8 mIoU); every L-DPT size is 3.2 to 3.9 mIoU higher on VOC 2012. Segmentation results →](https://pointcept.org/l-dpt#results)

“Put capacity in latents, and let pixels be the readout.”

The capability wall

## More pixels, blurrier depth.

Give a pyramidal DPT decoder more pixels and it repeats the same coarse layout while leaves, branches and thin poles wash out. L-DPT, trained only at 768, turns the extra pixels into finer depth detail, all the way to the full 100-megapixel photograph.

### Same 768 × 768 window · each model's own output pixels

### The whole photo at each input resolution

Qualitative: there is no ground-truth depth for this photograph. DPT columns are the DINOv3 depther (ViT-7B + DPT) and Depth Anything V2 (DPT); Depth Anything V2 predicts disparity, so its colours run the other way. Each crop is labelled with its native size: the 32 MP crops are 430 pixels across and are enlarged to match the 768-pixel crops from the 100 MP photo. The 2 and 8 MP tiles are the previews from the paper's Fig. 3; native-resolution crops at those sizes are coming.

Gallery

## Every pixel is a prediction.

Professional photographs from 24 to 100 megapixels. Boxes mark crops shown at native resolution; the grey square in the corner is the 768 × 768 training size, drawn to scale. Qualitative only: these photographs have no ground-truth depth.

The efficiency wall

## Pixels scale. Efficiency does not.

Decoder-only peak memory and latency as input resolution grows. The pyramidal DPT decoder jumps to a much slower regime near 9 and 17 MP and reaches 59.4 GB and 394 s at 31 MP, the last resolution measured. L-DPT grows smoothly to 15.3 GB and 5.3 s at 64 MP.

DPT memory L-DPT memory DPT latency L-DPT latency Provisional · from Fig. 2

Megapixels Latency axis

Log

Linear

### End to end

With its ViT-7B encoder, the DINOv3 depther (DPT decoder) runs out of memory on 32 MP inputs on a 192 GB NVIDIA B200.

Method

## Reason in latents. Render to pixels.

Pyramidal decoders put their capacity on feature maps that grow with the output. L-DPT keeps one latent token grid, aligned with the ViT patches, as the only spatial carrier, and produces pixels with a lightweight readout at the very end.

Pyramid decoder (DPT)

Latent decoder (L-DPT)

ImageH × W Frozen ViTDINOv3 feature pool 0-init Selection Selection Selection Selection One latent token grid (H/16 × W/16) throughout · attention blocks with 2D RoPEno upsampling stages, no pixel-space pyramid PixelreadoutH × W ImageH × W ViT4 tapped layers Reassemble and fuse while upsampling 1/32 1/16 1/8 1/4 HeadupsampleH × W feature maps grow with the output grid, so decoder memory and latency grow with the pixels

Select

### Every stage re-reads the whole encoder

Features from several ViT layers are concatenated into one pool. Each decoding stage applies its own linear projection to pick what it needs, instead of a fixed deep-to-shallow schedule.

Decode

### Capacity lives on the latent grid

A zero-initialised token grid at ViT patch resolution receives each selection by addition and is refined by attention blocks with continuous-coordinate 2D RoPE.

Read out

### Pixels are only the readout

A linear projection followed by pixel shuffle maps tokens to the output field; the decoder never holds pixel-resolution activations.

### Why training on 768 crops is enough

×0.5×1×2 Pixels

### Same crop, many scales

Each training image is resized by a random factor between 0.5× and 2× (bicubic), then cropped to 768 × 768. The crop never changes size; what lands in it ranges from wide context to close-up detail.

01768² crop01100 MPsame 0–1 rangefiner token grid Positions

### Positions are fractions, not indices

Attention uses 2D RoPE on coordinates normalised to \[0, 1\], so a 100 MP photo is the same range with a finer token grid. Training also rescales these coordinates (×1/8 to ×8) and jitters them (factor 1.2), so the decoder already sees token densities and aspects that a 768 crop alone would not.

“The architecture provides the mechanism for scalable decoding, but the training recipe is what turns that mechanism into robust resolution generalization.”

### Three sizes, one design

### Per-stage selection helps (Hypersim, relative depth, ARel ↓)

Results

## Up to 10× smaller decoders, competitive accuracy.

In the controlled rows every model shares the same frozen DINOv3 ViT-7B encoder and only the decoder is trained, so differences come from the decoder design. Bars show trainable parameters on a log scale.

Relative depth

Semantic segmentation

vs InfiniDepth

δ1 ↑

ARel ↓

The controlled block trains on synthetic data only (Hypersim, VKITTI2, Structured3D, TartanAir, BlendedMVG, IRS) and is evaluated zero-shot; the paper lists Depth Anything V2 under the same synthetic-only setting. Bold marks the best value per column within the controlled and mixed-domain blocks, computed from the values.

mIoU. TTA is horizontal flip. The DINOv3 segmenter is ViT-Adapter + Mask2Former on the same frozen ViT-7B. Previous best: Mask2Former on COCO-Stuff; Rethinking Pre-training and Self-training on VOC.

δ1 ↑ under matched splits and masks. The ViT-7B row repeats the mixed-domain model from the depth table. InfiniDepth uses its recommended RANSAC scale/shift estimation; L-DPT uses standard image-wise scale/shift. Memory and latency on one NVIDIA H800. InfiniDepth queries an implicit depth field; L-DPT moves feature integration into latent decoding. The two are complementary.

### Resolution comes from the design. Depth quality is bounded by open data.

What the design delivers

Training on 768 × 768 crops and running on 100-megapixel photographs, with detail that sharpens rather than blurs as pixels are added, comes from latent decoding and its training recipe. Nothing in the decoder ties it to the training resolution.

What open data still limits

How accurate the depth is, on unfamiliar scenes or hard materials such as water, glass and sky, is bounded by what we train on: public depth datasets that are limited in size, realism and resolution. We see the data, not the design, as the current limit; with better, higher-resolution training data we expect the same design to go further.

Open source

## Use L-DPT

Code, weights and full-resolution outputs are being prepared for release. Items go live here as they are published.

### Code Soon

Training and inference for relative depth and semantic segmentation.

### Weights Soon

L-DPT S / B / L decoders for depth and segmentation.

### Demo Soon

Try L-DPT on your own high-resolution photographs.

### Full-resolution outputs Soon

The depth maps on this page at native resolution.

L-DPT trains only the decoder on top of a frozen DINOv3 encoder. DINOv3 weights are distributed by Meta under their own license and access terms.

Citation

## BibTeX

```
@inproceedings{wu2026latent,
  title     = {Scaling Dense Prediction with Latent Decoding},
  author    = {Wu, Xiaoyang and Lao, Yixing and Wang, Chengyao and Yang, Senqiao and Zhang, Yujia and Zhao, Hengshuang},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}
```

Copy

Silent walkthrough of the paper: the 100-megapixel bamboo photograph and the relative depth L-DPT predicts for it, DPT and L-DPT as resolution grows, decoder memory and latency, the latent decoder, four gallery photographs, and the main results.

Photo credits will be listed here. Part of [Pointcept](https://github.com/Pointcept). © 2026 the L-DPT authors.