Department of Computer Science
    Artificial Intelligence Research Laboratory

Machine Learning

   Specification, Design, Analysis and Implementation of Learning Agents  

Spring 2006 Offering  

Course Staff

Instructor
Dr. Vasant Honavar
Professor
Artificial Intelligence Research Laboratory
Department of Computer Science and
Bioinformatics and Computational Biology
Center for Computational Intelligence, Learning, & Discovery
211 Atanasoff Hall
Iowa State University
Voice: 515 294-1098, Fax: 515 294-0258
honavar@cs.iastate.edu
Office Hours: WF 11am - 12 noon, 211 Atanasoff (or by appointment).

Teaching Assistant
Oksana Yakhnenko
Ph.D. Student, Artificial Intelligence Research Laboratory
Department of Computer Science
Office: B04 Atanasoff
Iowa State University
oksayakh@cs.iastate.edu
Office Hours: TR 10-11 AM (or by appointment).

Course Schedule
Lectures: MWF 10am to 11am
0290 Carver Hall

Course Overview


This course aims to provide an introduction to the basic principles, techniques, and applications of Machine Learning. Programming assignments are used to help clarify basic concepts. The emphasis of the course is on the fundamentals, and not on providing a mastery of specific commercially available software tools or programming environments.

In short, this is course is about the principles, design and implementation of learning agents --- programs that improve their performance on some set of tasks with experience. Upon successful completion of the course, you will have a broad understanding of machine learning algorithms and their use in data-driven knowledge discovery and program synthesis. You will have designed and implemented several machine learning algorithms in Java. You will also be able to identify, forumulate and solve machine learning problems that arise in practical applications. You will have a knowledge of the strengths and weaknesses of different machine learning algorithms (relative to the characteristics of the application domain) and be able to adapt or combine some of the key elements of existing machine learning algorithms to design new algorithms as needed. You will have an understanding of the current state of the art in machine learning and be able to begin to conduct original research in machine learning.


Course Description

Machine Learning - Com S 573 - is a 3-credit, introductory graduate course on Machine Learning offered by the Department of Computer Science at Iowa State University.

Topics covered include: Algorithmic models of learning. Learning classifiers, functions, relations, grammars, probabilistic models, value functions, behaviors and programs from experience. Bayesian, maximum a posteriori, and minimum description length frameworks. Parameter estimation, sufficient statistics, decision trees, neural networks, support vector machines, Bayesian networks, bag of words classifiers, N-gram models; Markov and Hidden Markov models, probabilistic relational models, association rules, nearest neighbor classifiers, locally weighted regression, ensemble classifiers. Computational learning theory, mistake bound analysis, sample complexity analysis, VC dimension, Occam learning, accuracy and confidence boosting. Dimensionality reduction, feature selection and visualization. Clustering, mixture models, k-means clustering, hierarchical clustering, distributional clustering. Reinforcement learning; Learning from heterogeneous, distributed, data and knowledge. Selected applications in data mining, automated knowledge acquisition, pattern recognition, program synthesis, text and language processing, and bioinformatic and computational biology.


Primary References

There is no required textbook for this course. The primary references are:

  1. Baldi, P. and Brunak, S. (2002). Bioinformatics: A Machine Learning Approach. Cambridge, MA: MIT Press.
    This book offers a good coverage of machine learning approaches - especially neural networks and hidden Markov models in bioinformatics.
  2. Baldi, P., Frasconi, P., Smyth, P. (2003). Modeling the Internet and the Web - Probabilistic Methods and Algorithms. New York: Wiley.
    A good introduction to machine learning approaches to text mining and related applications on the web.
  3. Bishop, C. M. Neural Networks for Pattern Recognition. New York: Oxford University Press (1995).
    This book offers a good coverage of neural networks
  4. Chakrabarti, S. (2003). Mining the Web, Morgan Kaufmann.
  5. Cohen, P.R. (1995) Empirical Methods in Artificial Intelligence. Cambridge, MA: MIT Press. This is an excellent reference on experiment design, and hypothesis testing, and related topics that are essential for empirical machine learning research.
  6. Cowell, R.G., Dawid, A.P., Lauritzen, S.L., and Spiegelhalter,D.J. (1999). Graphical Models and Expert Systems.Berlin: Springer.
    This is a very good introduction to probabilistic graphical models.
  7. Cristianini, N. and Shawe-Taylor, J. (2000). An Introduction to Support Vector Machines. London: Cambridge University Press.
    This is an excellent introduction to kernel methods for pattern classification.
  8. Duda, R., Hart, P., and Stork, D. (2001). Pattern Classification. New York: Wiley.
    This is a good text with primary emphasis on statistical methods for pattern classification.
  9. Hastie, T., Tibshirani, R., and Friedman, J. (2001). The elements of Statistical Learning - Data Mining, Inference, and Prediction. Berlin: Springer-Verlag.
    This is an excellent text that explains some of the key ideas in machine learning within a statistical framework.
  10. Jordan, M. (2003). Probabilistic Graphical Models. Professor Jordan has kindly shared a pre-publication draft.
    This text has an excellent coverage of generative and discriminative probabilistic models for classification.
  11. Kearns, M. and Vazirani, U. (1994). Computational Learning Theory. Cambridge, MA: MIT Press.
    This, although a bit dated, is an excellent introduction to learning theory.
  12. Mitchell, T. (1997). Machine Learning. New York: Mc Graw-Hill.
    This is, although a bit dated, an excellent introduction to Machine Learning.
  13. Russel, S. and Norvig, P. (2003). Artifiical Intelligence: A Modern Approach. 2nd Edition. New York: Prentice-Hall.
    This is an excellent text on Artificial Intelligence, with several introductory chapters on Machine Learning.
  14. Tan, P-N., Steinbach, M., and Kumar, V. (2004). Introduction to Data Mining. New York: Addison-Vesley.


In addition, we will draw on a number of additional sources for material to be covered in this course.


Other Course Materials


AI Resources


Java Resources


Computer Science Resources

This page is maintained by: Dr. Vasant Honavar . Please send suggestions, additions, or changes to: honavar@cs.iastate.edu. If you would like to be alerted by email when this page is updated, you may register with Changedetect