Selected Topics in Deep Learning

Department of Mathematics, Sharif University of Technology. Fall 2023

This course offers an introductory exploration of deep learning, utilizing mathematical tools as its foundation. In the first half of the course, we revisit the core methods that underpin deep learning. Later on, we will cover a number of recent theoretical advances that aim to shed light on the mathematical foundations of deep learning.

Prerequisites

The following skills will be useful for success in this course:

• Machine Learning: Some familiarity with machine learning will be helpful but not required; we will review important concepts that are needed for this course.

• Programming: You should be comfortable programming in Python. You should be familiar with algorithms and data structures. Familiarity with numpy or similar frameworks for numeric programming will be helpful but is not strictly required.

• Probability and Linear Algebra: You should have been exposed to probability distributions, random variables, expectations, etc.

Course Work

Grading will be based on:

• Assignments (40%)
• Final Exam (30%)
• Paper Presentation (30%):

Project and Presentation

An essential component of this course involves delving deep into several papers related to specific research topics. This task is to be completed in pairs. While the choice of paper is based on your interests, it must relate to the theoretical foundations of deep learning. You are encouraged to consult with instructors and TAs when selecting your paper.

• Proposal Submission: By the 1st of Azar, 1402, you are required to submit a 1-page proposal outlining your chosen research topic.

• Presentation: During the final two weeks of the course, you will deliver a 20-minute presentation to succinctly summarize the research paper.

• Paper Review: You are expected to submit a concise review of the paper. This review should provide an overview of the paper, its motivation, and the important theoretical results of the work. Additionally, critically evaluate its strengths and weaknesses. Furthermore, you are encouraged (although it is optional) to further explore the results or implement the ideas and compare them with other findings.

Tentative Lectures Schedule