In this course, the following topics are covered in the area of deep learning: Neural Networks and Convolutional Neural Networks, Optimization and Regularization, Supervised and Unsupervised Methods, Discriminative Networks, Training of Networks, Deep Generative Networks, Adversarial methods, Classification applications, Recurrent Neural Networks, Attention Mechanism and Transformers, Temporal Prediction applications.
The specific aims of the course are:
- To introduce main techniques in Deep Learning
- To understand the mathematical principles of optimization and regularization of deep learning methods
- To be able to design deep neural networks for various problems in artificial intelligence
- To implement solutions to learning problems using various deep neural network techniques
Some prerequisites for the course are:
- Machine Learning (Either an undergraduate level or MSc level Machine Learning course will be helpful)
- Probability Theory (A good understanding of random variables, multivariate Gaussians, Bayes rule, expectations, conditional probabilities)
- Linear Algebra (A good understanding of vectors, matrices, eigenvalues and eigenvectors)
- Numerical Methods (Basic knowledge)
- Python (Of course for coding assignments and the project)
The course will not follow a textbook strictly; however, it will use several of them and some other sources for references. Some of them are:
- Deep Learning, I. Goodfellow, Y. Bengio, A. Courville, 2016, MIT Press. (available online and for free here)
- Deep Learning with Python, F. Chollet, 2017, Manning.
- Dive Into Deep Learning here
- Fundamentals of Deep Learning Designing Next-Generation Machine Intelligence Algorithms, Nikhil Buduma, Reilly
- Machine Learning Yearning by Andrew NG (available online and for free here)
- PyTorch Course
- Coursera Deep Learning Online Course Series
- Stanford CS231N Deep Learning and its lecture videos
Upon successful completion of BLG561E, students will be able to:
- Know and discuss the main problems, application areas and the techniques of deep learning.
- Describe, construct and use necessary mathematical tools in deep learning such as optimization, regularization etc.
- Design various types of convolutional, fully connected and sequential neural networks and construct and run necessary procedures for training of deep neural networks and evaluate the results.
- Know and apply different mathematical formulations and solution techniques to supervised and unsupervised approaches in learning, specifically discriminative and generative network models, and recurrent networks.
- Implement computer realizations of deep learning applications and run necessary validation experiments.