‍

With the advent of foundational models and agents, the motivation for (full) transfer learning models has been decreasing. Instead, increasingly fine-tuning and few-shot learning are becoming more relevant. With the ability to tune a model on small high quality datasets one can quickly adapt state-of-the-art off-the-shelf models.

In this article we’ll explore:

1. What is Few-Shot Learning?
2. How does Few-Shot Learning work? 
3. Why Few-Shot Learning Matters for Computer Vision?
4. Key Few-Shot Learning Approaches
5. Few-Shot vs. Zero-Shot vs. One-Shot Learning: A Comparison
6. Use Cases and Applications 
7. Challenges and Limitations (Research)

And If you're working on few-shot or low-data training pipelines and looking for tools to make your data work harder, make sure to check out:

1. LightlyOne for selecting the most relevant training data
2. LightlyTrain for building strong representations using self-supervised learning- especially useful when labeled data is scarce.

What is Few-Shot Learning?

Few-shot learning (FSL) is a machine learning technique that enables a model to generalize to new tasks or classes using only a few labeled examples. In a typical few-shot setting, we might have only K examples of each new class (where K is small – e.g. 1, 2, or 5). This is in stark contrast to standard supervised learning, which usually needs a large dataset of examples per class. 

Check out this video to learn more.

Formally, few-shot learning is often described in the context of “N-way K-shot” or simply K-shot classification: the model must discriminate between N classes, given just K training examples of each class (the support set), and then classify new examples (the query set) accordingly​.

A typical fine-tuning workflow involves training a section of the model (typically the last few layers) on a small dataset. Compared to pretraining, fine-tuning datasets are typically more than 100x smaller. However, with the advent of bigger foundational models, it’s becoming harder to even fine-tune models on fairly decently sized fine-tuning datasets. 

Few-Shot learning has emerged as a viable alternative wherein we attempt to tune models with even fewer samples (10-50 per class) and achieve convincing performance on most tasks (termed few shot tasks).

Pro Tip: Read Pretraining vs. Fine-tuning: What Are the Differences?

An extreme case of few-shot learning is zero-shot learning wherein we evaluate models on new tasks and classes without tuning the model on any new classes. For example, if we take a pre-trained ImageNet model and evaluate its performance on classifying pictures of various types of faults in cement tubes. 

This is a case of zero-shot evaluation, since we’re evaluating the model’s performance on classification tasks without training the model on any images of cement tubes.

Moreover, with the shift from fully-supervised pre-training to self and semi-supervised contrastive learning, we have observed a general shift wherein models are becoming increasingly capable of zero-shot adaptation to novel tasks. 

However, these adaptations have been limited to simple tasks such as classification. For more critical use cases, one still needs to perform a few-shot fine-tuning!

The most common way, almost all of us perform few-shot learning is when we provide some examples to ChatGPT in the hopes of it being able to replicate them. For instance, you might ask ChatGPT:

“Given these input-output pairs, classify the given statement as positive or negative

Input: This is terrible. Output: Negative

Input: This is so good. Output: Positive

Input: This doesn’t work. Output:”

This involves feeding the model some examples in the instruction itself with the intent of the model identifying the pattern with the given data points.

A very important use case of Few Shot Learning is the ability to tune a model to learn about very rare classes from selective samples with little information and computation. This could be really crucial for fine grained visual classification (identifying a specific disease on plant leaves) or medical imaging where the availability of labeled data of various classes is highly limited or expensive to annotate.

Some important characteristics of few-shot learning include:

• Extremely limited training data per class: The model might see just a handful of examples (or even one, in one-shot learning) for each new class it needs to recognize. For instance, an FSL image classifier may be given 5 images of “species A” and 5 of “species B” and must learn to tell them apart, whereas a normal classifier would typically require hundreds of images of each species.
• Leverage of prior knowledge: Because data is scarce, few-shot learning algorithms transfer knowledge from prior experience. This prior experience could come from a large base training set of different classes, a pre-trained model (e.g. a network pre-trained on ImageNet), or from multiple similar tasks seen during a meta-learning phase. The goal is to encode general features or learning strategies that can be quickly applied to new, small-data problems​.
• Part of n-shot learning family: Few-shot learning is often considered alongside one-shot learning (the case where K=1) and zero-shot learning (K=0). In fact, FSL is a subset of n-shot learning methods​. One-shot learning is essentially an extreme case of few-shot learning (just one example per class), while zero-shot learning involves no direct examples of the target classes and typically relies on auxiliary information (like semantic attributes or language descriptions) to make predictions.

How Does Few-Shot Learning Work?

Let’s take the example of classification, typically we aim to approximate a function such that it is able to classify the vast majority of data points from the training dataset accurately. This is generally possible when the dataset has sufficient training samples.

 However in the case of few-shot classification, this becomes an increasingly harder problem since the number of examples is now significantly lower and therefore it’s much harder to attain higher performance on test sets within some generalization bounds.

Why Few-Shot Learning Matters for Computer Vision

Few-shot learning is especially significant in the field of computer vision, where obtaining large labeled data samples can be extremely labor-intensive and costly. 

Some reasons why FSL is valuable in computer vision:

• Expensive labeling: Annotating data at scale is time-consuming and often requires domain experts. For example, medical images may only have a few labels from specialists. Few-shot learning makes it possible to train models from those limited expert-labeled samples.
• Rare or emerging classes: In real-world scenarios, new categories can appear with little data—like a new species in wildlife monitoring or a novel defect in manufacturing. Few-shot learning allows models to adapt to these cases without needing large new datasets.
• Avoiding full retraining: Retraining large models from scratch is costly and slow. Few-shot learning enables quick updates or adaptations using small amounts of new data—ideal for fast iteration or continuous learning.
• ‍Human-like learning: Humans can generalize from just a few examples. Few-shot learning aims to bring that same adaptability to machine learning models.

Key Few-Shot Learning Approaches

So, how do we actually get a model to learn from a few examples? 

Over the years, a variety of approaches have been developed for few-shot learning. We can categorize the key approaches into a few broad strategies, each exploiting a different idea:  

• Meta-learning, 
• Transfer learning/fine-tuning
• Metric learning, 
• Data augmentation (or generative approaches)

Many practical algorithms combine elements of these strategies. Below, we outline each approach.

Meta-Learning Approaches (Learning to Learn)

Unlike traditional models that learn specific tasks, meta-learning algorithms are designed to acquire the learning process itself, enabling rapid adaptation to novel tasks with minimal data.

Instead of optimizing for performance on a single task, these training tasks optimize for the ability to learn efficiently across a distribution of tasks. This is typically achieved through a two-tiered learning process: an outer loop that learns how to learn (the meta-learning phase) and an inner loop that applies this learning strategy to specific tasks. This nested optimization allows models to extract task-agnostic learning strategies that transfer effectively to unseen problems.

Model-Agnostic Meta-Learning (MAML), introduced by Finn et al., exemplifies this approach by finding parameter initializations that can be rapidly adapted to new tasks with just a few gradient updates. MAML trains the model parameters to serve as a starting point from which minimal fine-tuning can yield optimal performance across diverse tasks.

Transfer Learning and Fine-Tuning Approaches

This approach typically begins with a pretrained model that has already learned robust feature representations from a large dataset. For example, models like ResNet or BERT, trained on ImageNet or massive text corpora respectively, develop comprehensive representations of their domains. These models encode abstractions – edges and textures in vision or syntactic patterns and semantic concepts in language.

Rather than training the entire network from scratch on limited examples, which would likely lead to overfitting, fine-tuning strategically updates select portions of the model while preserving the knowledge embedded or learnt in other layers. This technique frequently involves "freezing" early layers that capture universal features while updating later layers to specialize in the target task.

Metric Learning (Distance-Based Approaches)

Based on the simple task of learning a distance function over the various data samples, Metric Learning based approaches provide an easy to deploy solution that allows for fast inference times. Continuing with the example of few shot image classification, given two samples of images, we aim to minimize the distance between these samples if they belong to the same class or maximise if they belong to different classes. 

Even this simple training task has proven to work well in the literature. It is however known to be less adaptive to optimization in dynamic environments and grows linearly in complexity wrt the test size.

In the case of images, since the data is so high dimensional rather than optimizing the distance between raw images (largely unstructured data), we aim to minimize the distance between embeddings (structured latent space) generated from pre-trained image encoder models such as ResNet50.

These distance measures help us understand how “close” or “far apart” two points are in an embedding space. The choice of distance measure can significantly impact how our models learn and perform. Some common distance measures are:

• Hamming Distance: Given two equal-length strings or vectors, the hamming distance between the two strings is the number of positions at which a given symbol differs. 
• Manhattan Distance: The Manhattan or Taxicab Distance is a simple distance function that calculates the distance between two points as if they lie on a grid
• Euclidean Distance: calculates the length of the shortest line segment between the points
• Cosine Similarity: While not necessarily a “distance” function, cosine similarity is a widely used “measure” function which calculates the cosine of the angle between two points in the latent space.

Data Augmentation and Generative Approaches

Traditional data augmentation techniques apply domain-specific transformations to existing examples, creating variations of images of the same class that preserve class identity while introducing meaningful diversity. For instance in computer vision, these transformations include rotation, scaling, cropping, and color jittering, which simulate natural variations in object appearance. On the other hand for natural language processing, operations like random insertion, deletion, and word order swapping introduce linguistic variations while maintaining semantic content.

For few-shot learning, conditional GANs (Generative Adversarial Networks) can generate diverse examples of rare classes by learning from the limited number of examples. Similarly, Variational Autoencoders (VAEs) learn a continuous latent space representation of data that can be sampled to create novel examples, effectively interpolating between known samples.

These models not only generate novel examples but do so in ways that specifically aid learning from fewer samples. Diffusion models, which have recently gained prominence in image generation tasks, offer another promising direction for few-shot learning. 

These models, which gradually denoise random Gaussian noise into coherent data samples, can be fine-tuned on small datasets to generate class-specific examples. Their ability to capture complex data distributions makes them particularly well-suited for creating diverse, high-quality synthetic data in low-resource scenarios.

Few-Shot vs. Zero-Shot vs. One-Shot Learning: A Comparison

Few-shot, one-shot, and zero-shot learning are related concepts often mentioned together. The table below provides a quick comparison of these learning paradigms:

‍

‍

Few-Shot Use Cases and Applications 

Few-shot learning unlocks a variety of applications in computer vision and related fields. Here are some notable use cases, along with how few-shot techniques are applied in each:

• Novel Object Classification: Few-shot learning enables classification of previously unseen object categories with minimal labeled examples. This approach is particularly useful in domains like monitoring for new species or retail inventory systems that need to quickly adapt to new products.
• Few-Shot Object Detection: Traditional object detection systems require thousands of annotated bounding boxes, but few-shot detection methods can localize new object categories after seeing only a small number of examples. 
• One-Shot/Interactive Segmentation: One-shot segmentation systems can delineate object boundaries in images after seeing just a single example of that object class, often guided by user-provided reference points or examples. The SAMv2 model is a recent example of such a system. 
• Robotics and Object Learning: Few-shot learning allows robotic systems to quickly adapt their visual understanding and manipulation strategies after just a few demonstrations or examples using reinforcement learning.

Cross-Modality Applications: Few-shot techniques enable systems to bridge gaps between different data modalities, such as transferring knowledge from RGB images to infrared, depth, or medical imaging with minimal new examples

Challenges and Limitations (and Ongoing Research)

Few-shot learning is a powerful paradigm, but it also comes with several challenges and limitations that researchers and practitioners are actively working to address. Understanding these not only helps set appropriate expectations, but also points to current research trends aiming to overcome them.

The most fundamental issue is the inherent statistical uncertainty arising from limited samples, making it difficult to distinguish between meaningful patterns and random variations. This uncertainty often manifests as high variance in model performance, where success depends heavily on which specific examples are included in the support and query sets. Many current approaches also rely on strong inductive biases that work well for specific problem structures but fail when these assumptions are violated. 

Additionally, the computational intensity of meta-learning approaches presents practical barriers to deployment. Few-shot systems also typically assume clean, well-curated support sets, whereas practical applications often involve noisy or ambiguous examples. Moreover, current approaches often neglect the active learning dimension of real-world few-shot scenarios, where intelligently selecting which examples to label can significantly improve performance.