ComS 573: Machine Learning
Department of Computer Science
Iowa State University
Spring 2009

 


STUDY GUIDE

The material to be covered each week and the assigned readings (along with online lecture notes, if available) are included on this page. The assigned readings are divided into required and recommended readings. You will be responsible for the material covered in the lectures and the assigned required readings. You are strongly encouraged to explore the recommended readings.


Survival Tips


Week 1 (starting January 12, 2009)

Overview of the course

Review of probability theory.

Required Readings

Assignments

Recommended Readings for those unfamiliar with probability theory

Recommended Java Readings for those unfamiliar with Java.

Week 2 (starting January 19, 2009)

Information theory.

Bayesian Decision Theory.

Required Readings

Recommended Readings


Week 3 (starting January 26, 2009)

Probability density estimation: parameter estimation problem. Maximum-likelihood parameter estimation, Bayesian parameter estimation, parameter estimation for normal density, parameter estimation for discrete variables.

Naive Bayes Classifier

Required Readings

Assignments

Recommended Readings


Week 4 (starting February 2, 2009)

Naive Bayes Classifier: classifying text documents.

Evaluation of classifiers. Performance measures: Accuracy, Precision, Recall, ROC curves.

Evaluation of classifiers -- estimation of performance measures; confidence interval calculation for estimates.

Required Readings

Assignments

Recommended Readings


Week 5 (starting February 9, 2009)

Evaluation of classifiers --  cross-validation; comparing two hypotheses; hypothesis testing; comparing two learning algorithms.

The decision tree classifier: decision tree learning algorithm (Quinlan's ID3 algorithm).

Required Readings

Recommended Readings


Week 6 (starting February 16, 2009)

Decision tree learning algorithm: the problem of overfitting, missing data.

Linear models for classification: Linear Discriminant Functions, Perceptrons, Perceptron Learning algorithm, Multi-category classification, Winner-Take-All (WTA) networks (linear machines).

Regression: Linear regression, Least Mean Squared (LMS) Error Criterion,  Gradient descent algorithm, LMS solution for classification.

Required Readings

Assignments

Recommended Readings


Week 7 (starting February 23, 2009)

Probabilistic generative models and linear classifiers

Discriminative models for classification, logistic regression models,  training logistic regression, regularized logistic regression

Required Readings

Assignments

Recommended Readings


Week 8 (starting March 2, 2009)

Logistic regression models,  training logistic regression

Multilayer neural networks: feedforward operations, expressive power of multilayer networks, Backpropagation algorithm, Regularization, stopping criterion, Practical techniques for improving backpropagation

Required Readings

Assignments

Recommended Readings


Week 9 (starting March 9, 2009)

Neural networks: error functions for classification. Function optimization algorithms---conjugate gradient and quasi-Newton algorithms

Instance-Based Learning, k-Nearest neighbor learning

Required Readings

Recommended Readings


Spring Break


Week 10 (starting March 23, 2009)

Support Vector Machines: Dual Representation of the Perceptron Algorithm, Learning in feature spaces, Kernel Functions, Kernel-Induced Feature Spaces, Maximum Margin Classifiers, constrained optimization problem, Learning as optimization, Support Vectors, Non-Separable Case, The Soft-Margin Classifier

Required Readings

Assignments

Recommended Readings

Additional Information


Week 11 (starting March 30, 2009)

Bayesian Networks: Syntax and Semantics, D-separation. Markov Random Fields.

Required Readings

Additional Information


Week 12 (starting April 6, 2009)

Bayesian Networks: modeling, inference, learning

Required Readings

Assignments

Recommended Readings

Additional Information


Week 13 (starting April 13, 2009)

Bayesian Networks: learning

Required Readings

Recommended Readings

Additional Information


Week 14 (starting April 20, 2009)

Ensemble Classifiers: Bagging, The Adaboost Algorithm, Error correcting output coding

Required Readings

Recommended Readings


Week 15 (starting April 27, 2009)

Student oral presentation of term projects.

-------------------------------------------
Monday April 27, 10-10:50am

NAM H. PHAM and Matt Herbst
Automatic Bug Identification in Code Repositories

Chao-Chun Chang
Design an efficient orthogonal momentum-type PSO algorithm to solve a
Large Parameter Optimization Problem

Shane Griffith
Investigating a Robot's Capability to Learn about Containers: can a
robot form a container object category by probing objects?

JungGyu Yang
Building a predictive model for Customer Relationships Management with
Machine Learning

-------------------------------------------
Wednesday April 29, 10-10:50am

Lavanya Ram
Learning the k-best Bayesian network structures

Harris Lin
Bayesian Classifier

Christopher Bruno
Reinforcement Learning and Blackjack

Yetian Chen
Supervised and Semi-Supervised Learning Applied to Classification
Problems in Physics Experiment

-------------------------------------------
Friday May 1, 10-10:50am

Joshua J Clausman
Time-series turning point prediction with flexible neural trees.

Sushain Pandit
On utilizing dynamically extracted RDF knowledge-bases to enable
context-sensitive classification.

Yang Liu
KDD cup

Hailin Tang
Netflix prize

Hyuntae Na
Netflix prize

KyoungTak Cho
Netflix prize

-------------------------------------------