[Spring 2014 –16:198:536] Machine Learning

Schedule / Syllabus (Subject to Change)

 

Lecture #

Day

Date

Topic

Readings

Notes

1

Tue

Jan 21

Introduction

             Murphy Ch. 1

             Bishop Ch. 1

             http://www.cs.cmu.edu/~tom/pubs/MachineLearning.pdf

Optional

             Mitchell Ch. 1

 

2

Thu

Jan 23

Linear Regression &

Bias-variance Tradeoff

             Murphy Ch. 7.1-7.5

             Bishop Ch. 3.1-3.2

Optional

             HTF Ch. 3 & 4

 

3

Tue

Jan 28

Overfitting, Regularization,

Sparsity, & Evaluation

             Murphy Ch. 7.1-7.5

             Bishop Ch. 3.1-3.2

             http://www.autonlab.org/tutorials/overfit10.pdf

             http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.37.3325

Optional

             HTF Ch. 3 & 4

 

4

Thu

Jan 30

Logistic Regression &

Nave Bayes

             Murphy Ch. 5 & 8

             Bishop Ch. 4

 

5

Tue

Feb 4

Gaussian Nave Bayes &

Generative vs. Discriminative Classifiers

             http://www.cs.cmu.edu/~tom/mlbook/NBayesLogReg.pdf

Optional:

             http://ai.stanford.edu/~ang/papers/nips01-discriminativegenerative.pdf

             http://select.cs.cmu.edu/class/10701-F09/readings/bag-of-words.pdf

HW #1 out.

6

Thu

Feb 6

Decision Trees & IBL

             http://eliassi.org/Nilsson-ch6-ml.pdf

             Murphy Ch. 16

             HTF 13.3-13.5

Optional:

             Mitchell Ch. 3 & 8

 

7

Tue

Feb 11

Ensemble Methods

             http://eliassi.org/boosting-schapire.pdf

             http://www.springerlink.com/content/u0p06167n6173512/

             Murphy Ch. 16

             Bishop Ch. 14

Optional

             HTF Ch. 10, 15, 16

 

8

Thu

Feb 13

No class (snow day)

 

HW #1 due.

9

Tue

Feb 18

Ensemble Methods

& Perceptron

             https://www.cs.princeton.edu/~schapire/papers/explaining-adaboost.pdf

 

10

Thu

Feb 20

Artificial Neural Networks & Deep Learning

             Bishop Ch. 4.1.7 & 5

             Murphy Ch. 28

Optional

             Mitchell 4

             HTF 11

HW #2 out.

11

Tue

Feb 25

SVM & Kernels

             http://research.microsoft.com/en-us/um/people/cburges/papers/SVMTutorial.pdf

             http://select.cs.cmu.edu/class/10701-F09/readings/hearst98.pdf

             http://www.isn.ucsd.edu/pubs/nips00_inc.pdf

             Bishop Ch. 6 & 7

             Murphy Ch. 14

Optional

             HTF 6, 12.

 

12

Thu

Feb 27

Sample Complexity,

PAC, & VC Dimension

             http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=788640

             http://eliassi.org/COLTSurveyArticle.pdf

             http://jmlr.org/papers/volume6/langford05a/langford05a.pdf

             http://www.cs.princeton.edu/courses/archive/spr08/cos511/scribe_notes/0227.pdf

             Bishop Ch. 7.1.5

             Murphy Ch. 6.5 & 6.6

Optional:

             http://www.cs.cmu.edu/~avrim/Papers/survey.pdf

             Mitchell 7

             HTF 7

HW #1 graded.

13

Tue

Mar 4

Kernel Learning for Structured Prediction

(Guest Lecturer:
Chetan Tonde)

             Reading material in the Resources folder on Sakai

HW #2 due.

14

Thu

Mar 6

Bayesian Networks

             Murphy Ch. 10

             Bishop Ch. 8

 

15

Tue

Mar 11

Midterm Exam

(TA will proctor)

 

 

16

Thu

Mar 13

In-class Proposal Pitches

HW #2 graded.

17

Tue

Mar 18

Spring Break

 

 

18

Thu

Mar 20

Spring Break

 

 

19

Tue

Mar 25

Bayesian Networks & Graphical Models

             http://www.cs.cmu.edu/~aarti/Class/10701/readings/graphical_model_Jordan.pdf

             http://www.cs.cmu.edu/~aarti/Class/10701/readings/intro_gm.pdf

Proposal pitches graded.

20

Thu

Mar 27

Clustering

             Murphy Ch. 25

             Bishop Ch. 9

Optional:

             HTF 14

             http://www.cs.cmu.edu/~dpelleg/download/xmeans.pdf

             http://dl.acm.org/citation.cfm?id=1283494

Midterm graded.

21

Tue

Apr 1

Gaussian Mixture Models & Expectation Maximization

             Murphy Ch. 11

             Bishop Ch. 9

 

22

Thu

Apr 3

Hidden Markov Models

             Murphy Ch. 17

             Bishop Ch. 13

             http://www.cs.cmu.edu/~aarti/Class/10701/readings/gentle_tut_HMM.pdf

HW#3 out.

23

Tue

Apr 8

Latent Variable Models

             Murphy Ch. 27

 

24

Thu

Apr 10

Learning on Graphs

             http://eliassi.org/papers/ai-mag-tr08.pdf

             http://eliassi.org/papers/henderson-sdm10.pdf

             http://eliassi.org/papers/henderson-kdd2012.pdf

 

25

Tue

Apr 15

Dimensionality Reduction

             http://www.snl.salk.edu/~shlens/pca.pdf

             http://www.cs.cmu.edu/~tom/10701_sp11/slides/CCA_tutorial.pdf

             Bishop Ch. 12

Optional:

             http://www.cs.cmu.edu/~tom/10701_sp11/slides/pca_wall.pdf

HW #3 due.

26

Thu

Apr 17

1. What Can 20,000 Models Teach Us?
&

2. Reliable Differential Dependency Network Analysis

 (Guest Lecturer:
Dr. Alexandru
Niculescu-Mizil)

             http://www.cs.cornell.edu/~caruana/ctp/ct.papers/caruana.icml04.icdm06long.pdf

             http://www.niculescu-mizil.org/papers/calibration.icml05.crc.rev3.pdf

             http://www.cs.cornell.edu/~caruana/ctp/ct.papers/caruana.icml06.pdf

             http://www.niculescu-mizil.org/papers/owm.pdf

             http://arxiv.org/abs/1307.2611

 

27

Tue

Apr 22

Reinforcement Learning

             http://www.cs.cmu.edu/~tom/10701_sp11/slides/Kaelbling.pdf

             http://www.research.rutgers.edu/~lihong/pub/Li11Knows.pdf

Optional:

             Mitchell Ch. 13

 

28

Thu

Apr 24

In-class Final Exam

(TA will proctor)

 

 

29

Tue

Apr 29

In-class Project Presentations

 

HW #3 graded.

30

Thu

May 1

In-class Project Presentations

Last day of class.

 

--

Tue

May 6

--

 

Project presentations graded.

--

Tue

May 13

--

 

Final exam graded.

Project reports due.

--

Thu

May 15

--

 

Project reports graded.

Final grades released.