Zero-shot, online and batch learning

ZSL

Zero-shot learning (ZSL) is a machine learning technique in which a model is trained to recognize and classify new, unseen classes or concepts without having seen prior examples of those classes during the training process.

Traditional machine learning requires that all the classes in which the model is to be trained are present in the training data. The model then learns the patterns and characteristics of those classes. Zero-shot learning, however, involves the ability of a model to generalize to new classes for which no training examples are available. 

A real world example

  

Suppose we have a machine learning model trained on a dataset of images of known animals, such as dogs, cats, and elephants. However, we also have a set of unseen classes, such as giraffes and zebras, for which we don't have any training examples. 

To apply zero-shot learning, we can utilize additional information such as attributes or descriptions of the animals. Let's assume that we know giraffes have long necks and zebras have striped coats.   

During the training process, the model learns the patterns and features of the known animal classes, such as dogs, cats, and elephants. The model also learns the relationship between the visual features and the available attributes of these classes.  

When we apply the trained model to a new image of a giraffe, for which the model hasn't seen any training examples, it can recognize the attribute of a "long neck." Using this attribute, the model can reason that the image likely represents a giraffe, even without specific training data of giraffes.  

Similarly, for a new image of a zebra, for which the model also lacks training examples, the model can identify the striped coat as a key feature of a zebra. This allows the model to reasonably indicate that the image depicts a zebra. 

This is a simplified example to explain the basic principles of zero-shot learning in the context of classifying images of animals. In practice, more complex models and attributes can be employed to apply zero-shot learning across different domains and classification tasks.

So how does ZSL learn the "unknown" name?

The model does not extract the name of the unlearned class directly from the training data or attributes. Instead, it is common for the ZSL model to be provided with a separate data set containing linkage information between the attributes or features and their associated classes. This information is often referred to as a seen-unseen class mappingor ontology.

Online or batch learning

In the context of AI and machine learning, online learning and batch learning refer to two different approaches for training models.

1. Online Learning: Online learning, also known as incremental learning or streaming learning, refers to a learning paradigm where the model is trained gradually on new examples as they arrive. The model is updated and adjusted after each new data instance. In other words, it learns continuously while observing new data. Online learning is useful in situations where the data stream is constant and there is no possibility to process all training data at once. It allows for adaptive and real-time updates to the model, enabling it to respond to changes and evolutions in the data.
   
2. Batch Learning: Batch learning is a learning paradigm where the model is trained on a collection of training examples, also known as a batch. In batch learning, the training examples are processed as a whole, and the model is updated after processing the entire batch. In other words, the model is not updated directly after each individual data instance, but after processing a group of examples. Batch learning is often used when there is sufficient data available and it is feasible to process it in its entirety. It can be more efficient in terms of computations since the model is updated after each batch rather than after each individual instance.

Differences

The key difference between online learning and batch learning lies in how the model is updated during the training process. Online learning works incrementally and adjusts the model after each new instance, while batch learning works with batches of data and updates the model after each batch processing.

Both approaches have their own application areas and can be used in various AI scenarios depending on the nature of the data, the availability of data, and the requirements of the problem being solved. 

ZSL and online or batch learning

Zero-shot learning (ZSL) can be applied in conjunction with online or batch learning, depending on the specific implementation and learning paradigm used.

In online learning, the model is trained on new examples as they come in, one example at a time. In online ZSL, the model can gradually learn from new examples, including examples from unseen classes. It can use the available attributes or features to form associations between the known and unseen classes and gradually develop the ability to make predictions for the unseen classes.

In batch learning, on the other hand, the training examples are processed in batches, with the model being updated after processing each batch. Using batch learning, ZSL models can be trained on a collection of examples, including examples of known and unseen classes. The model can use the attributes and characteristics of the known classes to learn the associations and can then apply these associations to the unseen classes to make predictions.

Whether online or batch learning, the success of ZSL depends on the availability of relevant attribute or feature information and the model's ability to generalize this information and form associations between known and unseen classes.

It is important to note that the specific training process and implementation of ZSL may vary depending on the underlying algorithms and approaches used. Different ZSL methods may have different requirements and may target specific learning paradigms, such as online, batch or other forms of learning.