Learning from Data (Fall 2022)

Welcome to the class website of Learning from Data!

Because of the pandemic, we will temporarily have online classes until further notice!

News

2022-12-30: Submit your poster on Web Learning before noon on Jan 4th.
2022-12-15: Check out the Final Project Q&A for commonly asked questions. Feel free to leave your questions there!
2022-12-10: Programming Assignment 5 is released! Due Dec 25
2022-11-30: Please sign up in the schedule form to make a project appointment with the course staff.
2022-11-19: Final project information is released! See this page for datasets and project ideas.
2022-11-14: PA3 is released and due in two weeks.
2022-11-04: Please come to the midterm review session next Wednesday Nov 9th from 7pm-9pm at C1-402! Practice exam will be released this weekend.
2022-10-30: Written Assignment 3 is released! Due Nov 8th. Note that this assignment has no late days! The midterm exam will be held in class on November 11th. You can bring a double-sided A4-size cheat sheet to the midterm. The content will focus on all materials up to the lecture on Nov 4th.
2022-10-15: Programming Assignment 2 is released! Due Nov 8th.
2022-10-15:Written Assignment 2 has been released! Due Oct 29th.
2022-10-2: Next lecture (10/7) will be canceled due to the National Holiday. Programming Assignment 1 is released, due Oct 15th. WA1 solution is availale.
2022-9-24: Written Assignment 1 is released! Due Oct 8th.
2022-9-19: Posted lecture handout (without notes); Fixed the link to annotated slides #2.
2022-9-16: The first lecture will be rescheduled to Saturday, Sept 17, from 9:50am-12pm for this week. The classroom location is changed to C1-402 for this time only. The next lecture will resume on Friday morning.
2022-9-13: Math and Programming Review Session will be held on Sept 16 from 9:50am-12pm at C1-202. This session is optional.

Class info

Time: Friday 9:50am-12:15pm
Location: C1-402 (Currently online)

This introductory course gives an overview of many concepts, techniques, and algorithms in machine learning, from linear models such as logistic regression and SVM to more advanced topics such as deep neural networks and reinforcement learning. The course will give the student the basic ideas and intuition behind modern machine learning methods as well as a formal understanding of how, why, and when they work. The underlying theme in the course is statistical inference as it provides the foundation for most of the methods covered.

For more information about grading, homework and exam policies, see the class syllabus

Prerequisites： Undergraduate level calculus, probability and linear algebra. Basic Python programming.

Team

Yang Li
Instructor

Weida Wang
Head TA

Dexu Kong
TA

Zhiyuan Peng
TA

Wanda Li
TA

Huaze Tang
TA

Office hours

Name	Time	Location
Yang	Friday 2:00-4:00pm	Info Building 1108a
Wanda	Monday 2:00-4:00pm	Info Building, 11th floor common area)
Zhiyuan	Tuesday 3:00-5:00pm	(same as above)
Weida	Wednesday 4:00-6:00pm	(same as above)
Dexu	Thursday 7:00-9:00pm	(same as above)
Huaze	Friday 7:00-9:00pm	(same as above)

You can also make appointments outside office hours.

Recitation Sessions

Recitations will be held every Friday 9:00-9:45pm in the lecture room.

Date	Topic	Reference
Sept 16	Review Session： Basic linear algebra and probability; Scientific programming in Python. (programming demo) \| (math review) \| (math review notes) \| (recording)	The Maxtrix Cookbook by KB Petersen
Sept 23	Probability review, eigenvalue decomposition, SVD (notes) \| (recording)
Sept 30	MAE, Weighted linear regression, mean absolute error, MAP estimation (notes) \| (recording)
Oct 14	WA1 homework discussion. (notes) \| (recording)
Oct 21	PA1 homework discussion. (slides) \| (recording)
Oct 29	Kernel least square regression, SMO (slides) \| (recording)	SMO chapter from MLLN
Nov 4	WA2 homework discussion (recording)
Nov 9	Midterm review session (7pm-9pm) (notes) \| (recording)
Nov 18	Midterm review (Regular time and location)
Nov 25	VC Dimension (notes) \| (recording)
Dec 2	K-means and project Q&A (recording)
Dec 17 (3-4pm)	PA3 review and PA4 introduction. Tencent meeting ID: 791 760 726 (recording) \| PA3 sample solution (analysis questions) \| Carpole video
Dec 30	PCA and project Q&A (recording)

Class Schedule

Date	Topic	Readings & References	Homework Release
9/17	Introduction (handout) (slides with notes) Supervised Learning I (Part 1) (handout) (slides with notes)
9/23	Supervised Learning I (Part 2): Linear regression, logistic regression (slides with notes) (recording)	"Supervised learning" 1.1-1.3,2.1-2.3 (MLLN) Convex functions by Boyd & Vandenberghe (see Chapter 3)	Written Assignment 1 due Oct 8th
9/30	Supervised Learning II: Multi-class classificaiton, generalized linear models (handout) (slides with notes) (recording)	"Generalized linear models" (MLLN) Gene ralized Linear Models by Nelder & Wedderburn (1972)	Programming Assignment 1 due Oct 15th
10/14	Supervised Learning III: Gaussian discriminative analysis (GDA), naive bayes (handout) (sldies with notes) (recording) Code snippets: GDA and Logistic Regression	"Generative learning algorithms" (MLLN) Reference: Event models for NB text classification by McCallum & Nigam (1998)	Written Assignment 2 due Oct 29 WA1 solution
10/21	Supervised Learning IV: Support Vector Machines (SVM) (handout) (slides with notes) (recording)	"Support Vector Machines" (MLLN) References: Support Vector Networks by Cortes and Vapnik; SVM notes from “Selected Applications of Convex Optimization” by Li Li; Convex optimization by Stephen Boyd (See Chapter 5)	Programming Assignment 2 due Nov 8
10/28	Kernel SVM (slides with notes) (notes on soft-margin SVM support vector conditions) Supervised Learning V: Neural Networks (slides with notes), Backpropagation (slides with notes) (recording)	“Deep Feed Forword Networks” from Deep Learing by Ian Goodfellow	Written Assignment 3 due Nov 8th.
11/4	Backpropagation and practical tips on neural networks (slides with notes #1) (slides with notes #2) Model selection (slides with notes) (recording)	“Regularization and model selection” (MLLN) “Regularization for Deep Learning” from Deep Learning Notes on matrix derivatives by Learned-Miller	WA2 solution
11/11	Midterm Exam		Programming Assignment 3 WA3 solution
11/18	Basic Learning Theory (handout) (slides with notes) (recording)	"Generalization" (MLLN)	Final project information Recommended Latex template
11/25	Unsupervised Learning I: (handout) (slides with notes) (recording) K-means clustering spectral clustering	“Clustering and the k-means algorithm” (MLLN) A tutorial on spectral clustering by Ulrike von Luxburg	Written Assignment 4 due Dec 11th
12/2	Unsupervised Learning II: (recording) spectral clustering (continued) (slides with notes), PCA (handout) (slides with notes)	"Principal component analysis" (MLLN)	Project meeting sign up form: each team should make an appointment with the course staff between Dec 2 and Dec 7.
12/9	Reinforcement Learning: (slides) (handout) (slides with notes) (recording)	"Reinforcement Learning" (MLLN) Playing Atari with Deep Reinforcement Learning by Mnih et. al. Extended reference text: Reinforcment Learning: An Introduction 2nd, ed. by Sutton and Barto	Programming Assignment 5 due Dec 25 \| Q&A for PA4
12/16	Unsupervised Learning III: ICA and CCA (handout) (slides with notes) (recording)	“Independent Component Analysis” (MLLN) “Canonical Correlation Analysis” by Hardle & Simar	Final Project Q&A
12/23	Unsupervised Learning IV: Mixture of Gaussians, Expectation Maximization Factor Analysis	“EM and Factor Analysis” (MLLN) Additional reading: Neural Expectation Maximization (Greff et. al. 2017) The EM Algorithm for Mixtures of Factor Analyzers (Ghahramani & Hinton 1997)	WA4 solution
12/30	Semi-Supervised learning (recording) (handout) (slides with notes)	Additional reading: Deep semi-supervised learning (Ouali et. al., 2020) Short survey on classical semi-supervised learning methods (Xiaojin Zhu) A note on label propagation (BoDo & Csato, 2015)	Please submit the poster on web learning before noon on Jan 4th!
1/6	Final poster presentation