Course Description

In this course, the following topics are covered in the area of deep learning: Neural Networks and Convolutional Neural Networks, Optimization and Regularization, Supervised and Unsupervised Methods, Discriminative Networks, Training of Networks, Deep Generative Networks, Adversarial methods, Classification applications, Recurrent Neural Networks, Attention Mechanism and Transformers, Temporal Prediction applications.

The specific aims of the course are:

  1. To introduce main techniques in Deep Learning
  2. To understand the mathematical principles of optimization and regularization of deep learning methods
  3. To be able to design deep neural networks for various problems in artificial intelligence
  4. To implement solutions to learning problems using various deep neural network techniques

Some prerequisites for the course are:

  • Machine Learning (Either an undergraduate level or MSc level Machine Learning course will be helpful)
  • Probability Theory (A good understanding of random variables, multivariate Gaussians, Bayes rule, expectations, conditional probabilities)
  • Linear Algebra (A good understanding of vectors, matrices, eigenvalues and eigenvectors)
  • Numerical Methods (Basic knowledge)
  • Python (Of course for coding assignments and the project)

The course will not follow a textbook strictly; however, it will use several of them and some other sources for references. Some of them are:

Upon successful completion of BLG561E, students will be able to:

  1. Know and discuss the main problems, application areas and the techniques of deep learning.
  2. Describe, construct and use necessary mathematical tools in deep learning such as optimization, regularization etc.
  3. Design various types of convolutional, fully connected and sequential neural networks and construct and run necessary procedures for training of deep neural networks and evaluate the results.
  4. Know and apply different mathematical formulations and solution techniques to supervised and unsupervised approaches in learning, specifically discriminative and generative network models, and recurrent networks.
  5. Implement computer realizations of deep learning applications and run necessary validation experiments.