Teaching     Home

CSE 6363 (Fall 2009)
Machine Learning

Heng Huang


[ Administrative Basics | Course Description | Assignments | Outline of Lectures ]

Administrative Basics

Lecture

Nedderman Hall 229 | Mondays and Wednesdays 2:30-3:50 PM
Instructor

Heng Huang | Nedderman Hall 308 | Office hours: Mondays and Wednesdays 4:00-5:20 PM
Textbook

Pattern Recognition and Machine Learning, Christopher M. Bishop, 2006. (Required)
The Elements of Statistical Learning, T. Hastie, R. Tibshirani, J. H. Friedman.
Pattern Classification, 2nd edition, Richard Duda, Peter Hart, David Stork.
Work

Homework sets. (30%)
Class presentations. (20%)
Final project. (30%)
Participation. (20%)
Request

Good math and programming background, for Master student, CSE 4308 or CSE 5360 (Artificial Intelligence I) is requested.

Course Description

This course is designed to give a graduate-level student a thorough grounding in the methodologies, technologies, mathematics and algorithms currently needed by people who do research in learning and pattern recognition or who may need to apply learning or pattern recognition techniques to a target problem (e.g. computer vision, bioinformatics, data mining, speech recognition, etc.). The following broad categories will be covered:
1) Introduction to Pattern Recognition and Learning Systems
2) Regression
2) Bayesian Learning
3) Non-parametric methods
4) Linear Discriminants and Support Vector Machines
5) Neural Networks
6) Decision Trees
7) Feature selection
8) Model selection
9) Introduction to learning theory
10) Unsupervised learning methods


Assignments

Homeworks

Homework 1 (Due Oct. 5, before class)
Homework 2 (Due Oct. 28, before class)
Homework 3 (Due Nov. 26)
Paper presentations

Each student is expected to present one paper in the course of the semester and lead the discussion on the paper. The paper will be distributed to students later.
Project

Students will be asked to prepare, submit and present one project which will due at the end of the semester (I will post a project list). Students can choose his/her own topic to investigate. You will need to submit a short (one page) proposal for the purpose of approval and feedback for the final project.


Outline of Lectures

Week 1.

Mon Aug 24: Introduction to Machine Learning (slides)
Wed Aug 26: Basic Machine Learning: Probability Theory, Curve Fitting, Decision Theory, Information Theory (slides)

Week 2.

Mon Aug 31: Probability Distributions (slides)
Wed Sep 2: Bayesian Learning Theory (slides)

Week 3.

Mon Sep 7: No Class, Labor Day
Wed Sep 9: Linear Models (slides)

Week 4.

Mon Sep 14: Linear Models for Classification I: Naive Bayes Classifier (slides)
Wed Sep 16: Linear Models for Classification II: Fisher Linear Discriminant Analysis (slides)

Week 5.

Mon Sep 21: Classification Summary (slides)
Wed Sep 23: Neural Network (slides)

Week 6.

Mon Sep 28: ICCV conference
Wed Sep 30: ICCV conference

Week 7.

Mon Oct 5: Neural Network, Linear Algebra Review (slides)
A Tutorial on Support Vector Machines
Wed Oct 7: Learning with Kernels I (slides)
A simple tutorial "Support Vector Machines and Kernels for Computational Biology"

Week 8.

Mon Oct 12: Learning with Kernels II (slides)
ICML'01 Tutorial
Wed Oct 14: Learning with Kernels III and SVD (slides)

Week 9.

Mon Oct 19: PCA and SVD (slides)
Wed Oct 21: High order PCA/SVD by Dr. Chris Ding

Week 10.

Mon Oct 26: Cross Validation, KNN, Clustering Algorithms (slides)
Wed Oct 28: K-Means Clustering and EM Algorithm (slides)

Week 11.

Mon Nov 2: Graphical Models I (slides)
Wed Nov 4: Graphical Models II (slides)

Week 12.

Mon Nov 9: Hidden Markov Models (slides)
Wed Nov 11: Class Presentations
BNTagger: improved tagging SNP selection using Bayesian networks (Linbin Yu)
Supervised classification of array CGH data with HMM-based feature selection (Jin Huang)
Learning from Scarce Experience (Hamed Janzadeh)
A Probabilistic Interpretation of Canonical Correlation Analysis (Sajjad Moradi)

Week 13.

Mon Nov 16: Class Presentations
Gesture Recognition using Hidden Markov Models from Fragmented Observations (Georgios Galatas)
Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers (Alexandra Stefan)
SVM-RFE Peak Selection for Cancer Classification with Mass Spectrometry Data (Xiao Cai)
Overview and Recent Advances in Partial Least Squares (Shuo Li)
Wed Nov 18: Class Presentations
Combining Topic Models and Social Networks for Chat Data Mining (Saravanan Thirumuruganathan)
A Bayesian Framework for Reinforcement Learning (Alexandros Papangelis)
Extracting social networks and contact information from email and the Web (Miao Zhang)
Robust Principal Component Analysis for Computer Vision (Vishnukumar G. N.)

Week 14.

Mon Nov 23: Class Presentations and Conditional Random Field I
A Bayesian approach to filtering junk e-mail (Information retrieval). (Don Alexander)
Learning Social Networks from Web Documents Using Support Vector Classifiers (James McNellis)
ifile: An Application of Machine Learning to EMail Filtering (Rachit Shah)
Wed Nov 25: Conditional Random Field II

Week 15.

Mon Nov 30: Semi-supervised Learning
Wed Dec 2: Decision Tree and Boosting



Paper List for Presentation

Presentation:

Every student selects one paper from the following list and presents the paper (about 25 min) in class. Grade: slides preparation 50%, oral presentation 25%, and answer questions 25%. The presentation on paper in the second category will get plus and PhD students are suggested to select them. Please download the papers using UTA campus internet, because some digital libraries require the subscription.
Please select your presentation time from the following options: Nov. 11, 16, 18, 23.

First category:

A Bayesian approach to filtering junk e-mail (Information retrieval). (Don Alexander)
Emotion recognition using a Cauchy naive Bayes classifier (Computer vision).
A Statistical Approach to Texture Description of Medical Images: A Preliminary Study (Information retrieval).
Real Time Facial Expression Recognition in Video using Support Vector Machines (Computer vision).
Face Detection and Recognition Using Hidden Markov Models (Computer vision).
The Customized-Queries Approach to CBIR Using EM (Information retrieval).
ifile: An Application of Machine Learning to EMail Filtering (Information retrieval). (Rachit Shah)
Spam Filtering with Naive Bayes - Which Naive Bayes? (Information retrieval).

Second category:

Learning Social Networks from Web Documents Using Support Vector Classifiers (Social networks and web applications). (James McNellis)
Support vector machine active learning for image retrieval (Information retrieval).
Extracting social networks and contact information from email and the Web (Social networks and web applications). (Miao Zhang)
Combining Topic Models and Social Networks for Chat Data Mining (Social networks and web applications). (Saravanan Thirumuruganathan)
Creating Probabilistic Databases from Information Extraction Models (Databases).
Representing and Querying Correlated Tuples in Probabilistic Databases (Databases).
Robust Principal Component Analysis for Computer Vision (Computer Vision). (Vishnukumar G. N.)
Gesture Recognition using Hidden Markov Models from Fragmented Observations (Computer Vision). (Georgios Galatas)
Intra-personal kernel space for face recognition (Computer Vision).
NeuroAnimator: Fast Neural Network Emulation and Control of Physics-Based Models (Graphics).
Style Machines (Graphics).
BNTagger: improved tagging SNP selection using Bayesian networks (Bioinformatics). (Linbin Yu)
Supervised classification of array CGH data with HMM-based feature selection (Bioinformatics). (Jin Huang)
Improved prediction of protein-protein binding sites using support vector machines (Bioinformatics).
Using Bayesian Network Inference Algorithms to Recover Molecular Genetic Regulatory Networks (Bioinformatics).
A Bayesian Networks Approach for Predicting Protein-Protein Interactions from Genomic Data (Bioinformatics).
SVM-RFE Peak Selection for Cancer Classification with Mass Spectrometry Data (Bioinformatics). (Xiao Cai)
Machine Learning Applications in Grid Computing (Grid Computing).
Kernel-Based Models for Reinforcement Learning in Continuous State Spaces (Reinforcement Learning).
A Bayesian Framework for Reinforcement Learning (Reinforcement Learning). (Alexandros Papangelis)
Model based Bayesian Exploration (Reinforcement Learning).
Learning from Scarce Experience (Machine Learning). (Hamed Janzadeh)
Overview and Recent Advances in Partial Least Squares (Machine Learning). (Shuo Li)
Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers (Machine Learning). (Alexandra Stefan)
A Probabilistic Interpretation of Canonical Correlation Analysis (Machine Learning). (Sajjad Moradi)
Learning and Inferring Transportation Routines (Activity Recognition).
Location-Based Activity Recognition (Activity Recognition).

Project List (Please select one project from the following list. You must write your own code. Due on Dec. 11)

1. Bayesian learning for classifying netnews text articles:

Naive Bayes classifiers are among the most successful known algorithms for learning to classify text documents. We will provide a dataset containing 20,000 newsgroup messages drawn from the 20 newsgroups. The dataset contains 1000 documents from each of the 20 newsgroups. Please refer Table 6.3 of Dr. Mitchell's book (Machine Learning, Tom Mitchell, listed in our optional textbooks, you can borrow it from me). Please download the data from the link.

2. Support vector machines for face recognition:

Face recognition is a learning problem that has recently received a lot of attention. One standard approach involves reducing the dimensionality of the problem using PCA and then selecting the nearest class (eigenfaces). Support Vector Machines (SVM) are becoming very popular in the machine learning community as a technique for tackling high-dimensional problems. Can SVMs outperform standard face recognition algorithms? Please implement the SVM algorithm by yourself. The experimental dataset can be found at here.

3. Neural network learning to recognize faces:

A neural network learning algorithm called Backpropagation is among the most effective approaches to machine learning when the data includes complex sensory input such as images. Dr. Mitchell's web page provides the dataset discussed in Section 4.7 of Dr. Mitchell's book (Machine Learning, Tom Mitchell, listed in our optional textbooks, you can borrow it from me), containing over 600 face images. Please download the images from the link.