How DisParQ works

DisParQ turns the features of a frozen vision foundation model into a compact description of an image: which part concepts appear, where they appear, and how each one looks. It learns this description without class labels, part annotations, or language supervision. The resulting concept representation is discrete at inference time, and a decoder uses it to predict the backbone's global image representation. Read the paper or explore the concepts.

Figure 3. DisParQ describes images with concepts and attributes. Fused and upsampled VFM features z_u are matched against prototypes w_n to determine which concepts occur (top-K identities i_{1:K}) and where they occur (assignments p_u at positions u). Discrete attributes q_{i_k} encode how each concept appears in the image. The assignment map selects the learned concept embeddings e_n and attributes for the decoder, which predicts ĉ(x) by distillation from the VFM's global representation. Open the full-resolution figure.

A small selection from a shared vocabulary

A frozen vision encoder supplies local image features and a global representation to reconstruct. DisParQ combines features from several encoder layers and upsamples them to a finer grid. A learned dictionary contains 1,024 concept prototypes in the default configuration. For each image, the model selects 16 concepts from that shared dictionary, then assigns exactly one of the selected concepts to every grid location. This yields non-overlapping, spatially grounded concept regions. The dictionary is shared across images, so the same concept identity can describe similar parts in different object categories.

Appearance stays separate from identity

The same concept can appear in different forms: a wheel, for example, can vary across vehicles. DisParQ pools the features of a concept's region and measures their difference from that concept's prototype. It first learns this appearance residual continuously, then converts it into a discrete attribute using a separate product codebook for each concept. The concept answers what part is present, while its attribute describes how that occurrence looks.

Figure 4. Separating concept identity and appearance. Selected-concept maps Ā_{i_k} determine exclusive spatial assignments p_u and pool each concept's region into z̄_{i_k}. Its normalized deviation from the concept prototype, r_{i_k}, describes the occurrence relative to the concept. Product codebooks ℬ_{i_k} quantize its projection. The resulting attribute q_{i_k} is added to the concept embedding e_{i_k} and broadcast to its assigned positions. Adding positional embeddings PE_u forms the decoder tokens t_u. Open the full-resolution figure.

Learning to preserve the backbone's information

A spatial decoder receives concept identities, the location assignment map, and the quantized attributes. It predicts the frozen encoder's global representation from this description alone. Training has two stages. The first discovers concepts while appearance residuals remain continuous. The second freezes the feature fusion and concept prototypes and learns quantized attributes and the decoder. The vision encoder and the feature upsampler stay frozen throughout.

The paper evaluates the decoder's representation with classification and retrieval probes, and compares the concept maps with annotated parts. Labels are used for those downstream evaluations. They are not used to learn DisParQ's concepts. For the full objectives, training schedule, and implementation details, see the paper and appendix.

See the concepts in images

The concept explorer shows examples of learned concepts across images. Reference crops illustrate appearances found for a concept. They are examples from the data, not controlled edits of an attribute code.