Module 3: Advanced Deep Learning Techniques (TD-M3-ADLT)

Course agenda:

1. Working out the data (days 1-2):
   1. Establishing schema;
   2. Dealing with missing values;
   3. Detecting anomalies w.r.t. reference data;
   4. Versioning data;
   5. Gathering data:
      1. Concept of data distribution;
      2. Deciding further collection based on baseline performance;
      3. Active learning.
   6. Succeeding with small data:
      1. Pre-trained models;
      2. Transfer learning;
      3. Representation learning;
      4. Data augmentation.
2. Advanced deep learning methods (days 2-3):
   1. Model calibration;
   2. Self-supervised learning;
   3. Knowledge-distillation;
   4. Network pruning;
   5. Adversarial training.
3. Advanced deep learning architectures (day 3):
   1. Attention-based models;
   2. Autoencoders;
   3. Diffusion models.
4. Dealing with hardware requirements via model compression (day 4).
5. Training pipelines: modularization, reproducibility, best practices (days 4-5)...
6. Best practices for successful deployments (day 5).

Target audience:

AI Engineers, AI R&D, Data Scientists (~1-3 years of experience).

Prerequisites:

1. Basics of Artificial Intelligence.
2. Computer Vision and Advanced Image Processing.

What the student will know after finishing the course:

• A data-centric approach to developing models for real use cases.
• Fundamentals of several advanced deep learning architectures.
• Practical application of several deep learning techniques.
• Robust model logging and versioning.
• How to deal with hardware constraints.
• How to modularize code for training via training pipelines.