LightlyTrain 0.15.0 adds Distillationv3, a new default distillation method that improves performance across both classification and dense vision tasks. Plus support for Meta EUPE backbones, custom teacher models, and new LTDETR augmentations.
LightlyTrain 0.15.0 introduces Distillationv3, a new distillation method that generalizes better across downstream tasks, and adds support for Meta's EUPE (Efficient Universal Perception Encoder) models. Together, these updates make it easier to train high-quality, versatile computer vision models, no matter whether you're targeting classification, detection, segmentation, or all of the above.
Previous distillation versions in LightlyTrain required users to choose based on their downstream task: Distillationv1 was best for global tasks like classification, while Distillationv2 excelled at dense tasks like detection and segmentation. Distillationv3 removes this tradeoff. It combines both a queue-based pseudo label loss (from v1) and a spatial feature loss (from v2), producing a single method that performs well on both global and dense tasks.
On a ViT-T/16 backbone, Distillationv3 reaches 99.5% of the classification specialist (v1) on ImageNet-1K and 96.2% of the detection specialist (v2) on COCO, making it the first distillation method in LightlyTrain that doesn't require choosing based on your downstream task.
Distillationv3 is now the default distillation method in LightlyTrain. When you call method="distillation", you automatically get v3. The previous versions remain available as method="distillationv1" and method="distillationv2" for backward compatibility.
Another key improvement: Distillationv3 works better with DINOv3 teacher models. We were therefore confident in updating the default teacher from DINOv2 ViT-B/14 to DINOv3 ViT-B/16, reflecting this improved compatibility.
Distillationv3 requires no changes to your existing distillation code, you only need to make sure that you install the latest version of LightlyTrain with pip install --upgrade lightly-train.
import lightly_train
if __name__ == "__main__":
lightly_train.pretrain(
out="out/my_experiment",
data="my_data_dir",
model="torchvision/resnet18",
method="distillation", # Uses distillationv3 by default
)Distillationv3 also introduces support for custom teacher models. Unlike v1 and v2, which were limited to a fixed set of DINOv2 and DINOv3 teachers, v3 lets you use any model supported by LightlyTrain as a teacher — including your own custom models.
This opens up new workflows, that were previously unsupported. For example, you can write a thin wrapper around a monocular depth model foundation model like DepthAnythingv3 and distill it into an efficient backbone such as a Next-ViT for edge deployment with the MiDaS line of models.
See the distillation documentation for details on the custom model interface.
For more information, check out LightlyTrain Documentation.
LightlyTrain 0.15.0 adds support for Meta's EUPE (Efficient Universal Perception Encoder) models across all distillation, pretraining, and fine-tuning tasks. EUPE models are vision encoders trained by distilling knowledge from multiple domain-expert foundation models into a single, efficient backbone. The result is a compact model with strong, versatile representations across diverse tasks — from image understanding to dense prediction.
Built on top of DINOv3, EUPE models are also available in both ViT and ConvNeXt variants:
Using an EUPE model is as simple as swapping the model string. For example, to use EUPE-backed (ViT-S/16) LT-DETR for object detection:
import lightly_train
if __name__ == "__main__":
lightly_train.train_object_detection(
out="out/my_experiment",
data={...},
model="dinov3/vits16-eupe-ltdetr",
)EUPE models are particularly interesting for edge deployment scenarios where you need a single backbone that handles multiple tasks well. They're released under Meta's FAIR Noncommercial Research License. See the DINOv3 model documentation for the full list of supported models.
Curate and label data, fine-tune foundation models — all in one platform.
Book a Demo
LightlyTrain 0.15.0 also includes several quality-of-life improvements for object detection training:
transform_args={"scale_jitter": {"step_stop": 10000}}).Check out our documentation to learn more about the new features:
Stay tuned for more updates!
Self-Supervised Pretraining
Leverage self-supervised learning to pretrain models
AI Training Data for LLMs & CV
Expert training data services for LLMs, AI Agents and vision
.png)
.png)
.png)
.svg)