ECCV 2026

Scaling Dense Prediction with Latent Decoding

Train at 768. Run at 100 megapixels.

Xiaoyang Wu1 Yixing Lao1 Chengyao Wang2 Senqiao Yang2 Yujia Zhang1 Hengshuang Zhao1
1The University of Hong Kong2The Chinese University of Hong Kong

L-DPT keeps the dense decoder on the ViT's compact latent grid and reaches pixels with a lightweight readout. Trained only on 768 × 768 crops, it predicts depth for 100-megapixel photographs, and fine structure stays sharp as resolution grows instead of blurring.

RGBL-DPT · relative depth

The dashed square is 768 by 768 pixels, the size of one training crop. Left and right arrow keys move the divider between photo and depth. Enter flies into the window and back.

8750 × 11664 · 102.1 MP · Hasselblad

“Put capacity in latents, and let pixels be the readout.”

The capability wall

More pixels, blurrier depth.

Give a pyramidal DPT decoder more pixels and it repeats the same coarse layout while leaves, branches and thin poles wash out. L-DPT, trained only at 768, turns the extra pixels into finer depth detail, all the way to the full 100-megapixel photograph.

Same 768 × 768 window · each model's own output pixels

The whole photo at each input resolution

Qualitative: there is no ground-truth depth for this photograph. DPT columns are the DINOv3 depther (ViT-7B + DPT) and Depth Anything V2 (DPT); Depth Anything V2 predicts disparity, so its colours run the other way. Each crop is labelled with its native size: the 32 MP crops are 430 pixels across and are enlarged to match the 768-pixel crops from the 100 MP photo. The 2 and 8 MP tiles are the previews from the paper's Fig. 3; native-resolution crops at those sizes are coming.

Gallery

Every pixel is a prediction.

Professional photographs from 24 to 100 megapixels. Boxes mark crops shown at native resolution; the grey square in the corner is the 768 × 768 training size, drawn to scale. Qualitative only: these photographs have no ground-truth depth.

The efficiency wall

Pixels scale. Efficiency does not.

Decoder-only peak memory and latency as input resolution grows. The pyramidal DPT decoder jumps to a much slower regime near 9 and 17 MP and reaches 59.4 GB and 394 s at 31 MP, the last resolution measured. L-DPT grows smoothly to 15.3 GB and 5.3 s at 64 MP.

DPT memory L-DPT memory DPT latency L-DPT latency
Latency axis

End to end

With its ViT-7B encoder, the DINOv3 depther (DPT decoder) runs out of memory on 32 MP inputs on a 192 GB NVIDIA B200.

Method

Reason in latents. Render to pixels.

Pyramidal decoders put their capacity on feature maps that grow with the output. L-DPT keeps one latent token grid, aligned with the ViT patches, as the only spatial carrier, and produces pixels with a lightweight readout at the very end.

ImageH × W Frozen ViTDINOv3 feature pool 0-init Selection Selection Selection Selection One latent token grid (H/16 × W/16) throughout · attention blocks with 2D RoPEno upsampling stages, no pixel-space pyramid PixelreadoutH × W
Select

Every stage re-reads the whole encoder

Features from several ViT layers are concatenated into one pool. Each decoding stage applies its own linear projection to pick what it needs, instead of a fixed deep-to-shallow schedule.

Decode

Capacity lives on the latent grid

A zero-initialised token grid at ViT patch resolution receives each selection by addition and is refined by attention blocks with continuous-coordinate 2D RoPE.

Read out

Pixels are only the readout

A linear projection followed by pixel shuffle maps tokens to the output field; the decoder never holds pixel-resolution activations.

Why training on 768 crops is enough

Pixels

Same crop, many scales

Each training image is resized by a random factor between 0.5× and 2× (bicubic), then cropped to 768 × 768. The crop never changes size; what lands in it ranges from wide context to close-up detail.

Positions

Positions are fractions, not indices

Attention uses 2D RoPE on coordinates normalised to [0, 1], so a 100 MP photo is the same range with a finer token grid. Training also rescales these coordinates (×1/8 to ×8) and jitters them (factor 1.2), so the decoder already sees token densities and aspects that a 768 crop alone would not.

“The architecture provides the mechanism for scalable decoding, but the training recipe is what turns that mechanism into robust resolution generalization.”

Three sizes, one design

Per-stage selection helps (Hypersim, relative depth, ARel ↓)

Results

Up to 10× smaller decoders, competitive accuracy.

In the controlled rows every model shares the same frozen DINOv3 ViT-7B encoder and only the decoder is trained, so differences come from the decoder design. Bars show trainable parameters on a log scale.

The controlled block trains on synthetic data only (Hypersim, VKITTI2, Structured3D, TartanAir, BlendedMVG, IRS) and is evaluated zero-shot; the paper lists Depth Anything V2 under the same synthetic-only setting. Bold marks the best value per column within the controlled and mixed-domain blocks, computed from the values.

Resolution comes from the design. Depth quality is bounded by open data.

What the design delivers

Training on 768 × 768 crops and running on 100-megapixel photographs, with detail that sharpens rather than blurs as pixels are added, comes from latent decoding and its training recipe. Nothing in the decoder ties it to the training resolution.

What open data still limits

How accurate the depth is, on unfamiliar scenes or hard materials such as water, glass and sky, is bounded by what we train on: public depth datasets that are limited in size, realism and resolution. We see the data, not the design, as the current limit; with better, higher-resolution training data we expect the same design to go further.

Open source

Use L-DPT

Code, weights and full-resolution outputs are being prepared for release. Items go live here as they are published.

L-DPT trains only the decoder on top of a frozen DINOv3 encoder. DINOv3 weights are distributed by Meta under their own license and access terms.

Citation

BibTeX

@inproceedings{wu2026latent,
  title     = {Scaling Dense Prediction with Latent Decoding},
  author    = {Wu, Xiaoyang and Lao, Yixing and Wang, Chengyao and Yang, Senqiao and Zhang, Yujia and Zhao, Hengshuang},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}

Silent walkthrough of the paper: the 100-megapixel bamboo photograph and the relative depth L-DPT predicts for it, DPT and L-DPT as resolution grows, decoder memory and latency, the latent decoder, four gallery photographs, and the main results.