Teaching     Home

CSE6392
Advanced Topics in Scalable Searching and Optimization
Dept. Computer Science and Engineering
Dr. Junzhou Huang


[ Administrative Basics | Course Description | Outline of Lectures ]

Administrative Basics

Lecture

WH 221 | Friday 4:00-6:50 PM
Instructor

Junzhou Huang | ERB 650 | Office hours: Friday 1:00-4:00 PM
Request

Basic math and programming background; Basic learning and vision background preferred
Textbook

None

Course Description

This course will provide an overview of the current state-of-the-art of big data searching techniques in computer vision, machine learning and data mingearning by studying a set of cutting-edge advanced topics in these areas. Several selected research topics reflect the current state in these fields. The main objective of this course is to review cutting-edge searching& learning research in big data through lectures covering the underlying statistical & mathematical concepts and representative algorithms, paper reading, and implementation. The instructor will work with students on building ideas, performing experiments, and writing papers. Students can decide to submit his/her results to a learning/mining/vision related conference, or just play with funs.

The course is application-driven and includes advacnced topics in imaging, learning and vision, such as different imaging techniques and advanced learning tools in different applications. It will also include selected topics relating to the emerging compressed sensing and sparse learning theory and techniques. The course will provide the participants with a thorough background in current research in these areas, as well as to promote greater awareness and interaction between multiple research groups within the university. The course material is well suited for students in computer science, computer engineering, electrical engineering and biomedical engineering.

Optional Project


Outline of Lectures

Week 1.

Fri Jan 20: Introduction

Course Objectives and Administration (Slides)

Week 2.

Fri Jan 27: Math Basics, Least Square and PCA (Slides)

M. Turk and A. Pentland, "Face recognition using eigenfaces", CVPR 1991.

M. Brand, "Incremental singular value decomposition of uncertain data with missing values", ECCV 2002.

D. Ross, J. Lim, R. Lin, M. Yang, "Incremental Learning for Robust Visual Tracking", International Journal of Computer Vision, 2007.

Week 3.

Fri Feb 3: Optimization Basics and Gradient Methods (Slides)

A. Beck and M. Teboulle, "A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems", SIAM Journal on Imaging Sciences, No. 1, pp. 183-202, 2009.

A. Beck and M. Teboulle, "Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems", IEEE Trans. Image Processing, Vol. 18, No. 11, pp. 2419-2434, 2009

Yurii Nesterov, "Gradient Methods for Minimizing Composite Objective Function", 2007.

Week 4.

Fri Feb 10: Scalable Searching Via Hiearchical Kmean Tree (Slides)

D. Lowe, "Object recognition from local scale-invariant features", ICCV 1999.

D. Nist́er and H. Steẃenius, "Scalable Recognition with a Vocabulary Tree", CVPR 2006.

Week 5.

Fri Feb 17: Deep Learning (Slides)

Week 6.

Fri Feb 24:

Yann LeCun, Yoshua Bengio & Geoffrey Hinton, "Deep Learning", Nature 2015 (Akib Zaman)

Krizhevsky, A., Sutskever, I. and Hinton, G. E., "ImageNet Classification with Deep Convolutional Neural Networks", NIPS 2012 (Srinivas Varadharajan)

Week 7.

Fri Mar 3:

Silver et. al., "Mastering the game of Go with Deep Neural Networks & Tree Search", Nature 2016 (Kunwar Dev Singh)

Graves el. al., "Hybrid Computing Using a Neural Network with Dynamic External Memory", Nature 2016 (Nitin Kanwar, Jayvardhan Chinchwade)

Week 8.

Fri Mar 10:

Jon Long*, Evan Shelhamer*, Trevor Darrell, "Fully Convolutional Networks for Semantic Segmentation", CVPR 2015. (Md Abdus Aslam (Lincoln))

Ma, Chenxin, Virginia Smith, Martin Jaggi, Michael I. Jordan, Peter Richtárik, and Martin Takáč. "Adding vs. averaging in distributed primal-dual optimization." ICML 2015. (Likhit Shankar Gowda, Likhith Mayanna Gowda)

Week 9.

Fri Mar 17: Spring Break

Week 10.

Fri Mar 24:

Jasper Snoek, Hugo Larochelle and Ryan P. Adams. "Practical Bayesian Optimization of Machine Learning Algorithms", NIPS 2012. (Guodong Liu)

Mikael Henaff, Joan Bruna, Yann LeCun, "Deep Convolutional Networks on Graph-Structured Data", 2015 (Shivam Bijoria)

Week 11.

Fri March 31:

Pierre Sermanet, David Eigen, Xiang Zhang, Michael Mathieu, Rob Fergus, Yann LeCun, "OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks", ICLR 2014 (Akash Rana, Anuradha Nokku)

Ross Girshick, "Fast R-CNN", arXiv:1504.08083 (Vikas Sable, Mailtili Deshpand)

Week 12.

Fri Apr 7:

Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", NIPS 2015 (Fan Ni)

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, "SSD: Single Shot MultiBox Detector", ECCV 2016 (Girish Tejas Prakash)

Week 13.

Fri Apr 14:

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi, "You Only Look Once: Unified, Real-Time Object Detection", CVPR 2016 (Sadhana Singh, Saurabh Kumar Singh)

Joseph Redmon, Ali Farhadi, "YOLO9000: Better, Faster, Stronger", 2016 (Pranil Maharjan, Vipulkumar Mahadik)

Week 14.

Fri Apr 21:

Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation", CVPR, 2014. (Mohammad Minhazul Haq)

Zhe Zhu, Dun Liang, Songhai Zhang, Xiaolei Huang, Baoli Li, Shimin Hu, "Traffic-Sign Detection and Classification in the Wild", CVPR 2016 (SanathKumar and Bhushan yavgal)

Zhe Zhu, Dun Liang, Songhai Zhang, Xiaolei Huang, Baoli Li, Shimin Hu, "Traffic-Sign Detection and Classification in the Wild", CVPR 2016 (Muhammad Tayyab)

Week 15.

Fri Apr 28:

Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio, “Generative Adversarial Nets”, arXiv:1406.2661 (Zhifei Deng)

InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets (Arun Balchandran)

Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov, "Learning Convolutional Neural Networks for Graphs", ICML 2016 (Shashank Madhav and Ashay Rajimwale)

Vijay Badrinarayanan, Ankur Handa and Roberto Cipolla, "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling." arXiv preprint arXiv:1505.07293, 2015 (Rodrigo Linhares)

Week 16.

Fri May 5:


 

Paper List:

Scalable Optimization:

  1. Jasper Snoek, Hugo Larochelle and Ryan P. Adams. "Practical Bayesian Optimization of Machine Learning Algorithms", NIPS 2012. (http://people.seas.harvard.edu/~jsnoek/software.html)
  2. Ma, Chenxin, Virginia Smith, Martin Jaggi, Michael I. Jordan, Peter Richtárik, and Martin Takáč. "Adding vs. averaging in distributed primal-dual optimization." ICML 2015. (https://github.com/gingsmith/cocoa)
  3. Cho-Jui Hsieh, Hsiang-Fu Yu, Inderjit S. Dhillon, "PASSCoDe: Parallel ASynchronous Stochastic dual Co-ordinate Descent", ICML 2015. (http://www.cs.utexas.edu/~rofuyu/exp-codes/passcode-icml15-exp/)
  4. Y. Zhang, MI. Jordan, "Splash: User-friendly Programming Interface for Parallelizing Stochastic Algorithms", 2016 (http://zhangyuc.github.io/splash/)
  5. Y. Zhang , J. Duchi, M. Wainwright, "Communication-Efficient Algorithms for Statistical Optimization", NIPS 2012
  6. Jakub Mareček, Peter Richtárik and Martin Takáč, "Distributed block coordinate descent for minimizing partially separable functions", to appear in Recent Developments in Numerical Analysis and Optimization, Springer Proceedings in Mathematics and Statistics,
    2015
  7. Ohad Shamir, Nathan Srebro and Tong Zhang, "Communication Efficient Distributed Optimization using an Approximate Newton-type Method", ICML 2014.
  8. Virginia Smith, Simone Forte, Chenxin Ma, Martin Takac, Michael I. Jordan, Martin Jaggi, "CoCoA: A General Framework for Communication-Efficient Distributed Optimization", arXiv:1611.02189

Deep Learning

  1. Krizhevsky, A., Sutskever, I. and Hinton, G. E., "ImageNet Classification with Deep Convolutional Neural Networks", NIPS 2012:
  2. K. Simonyan, A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition", ICLR 2015
  3. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, "Deep Residual Learning for Image Recognition", CVPR 2016
  4. Andrew Gordon Wilson*, Zhiting Hu*, Ruslan Salakhutdinov, and Eric P. Xing, "Stochastic Variational Deep Learning", NIPS 2016
  5. Yann LeCun, Yoshua Bengio & Geoffrey Hinton, "Deep Learning", Nature 2015
  6. Silver et. al., "Mastering the game of Go with Deep Neural Networks & Tree Search", Nature 2016
  7. Graves el. al., "Hybrid computing using a neural network with dynamic external memory", Nature 2016

Generative Representation

  1. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio, “Generative Adversarial Nets”, arXiv:1406.2661
  2. Mehdi Mirza, Simon Osindero, “Conditional Generative Adversarial Nets”, arXiv:1411.1784
  3. Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel, “InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets”, arXiv:1606.03657
  4. Sebastian Nowozin, Botond Cseke, Ryota Tomioka, “f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization”, arXiv:1606.00709
  5. I. Sutskever, O. Vinyals, Q. V. Le, "Sequence to Sequence Learning with Neural Networks", NIPS 2014
  6. M.T. Luong, Q.V. Le, I. Sutskever, O. Vinyals, L. Kaiser, "Multitask Sequence to Sequence Learning", arXiv, 2015

Segmentation

  1. Jon Long*, Evan Shelhamer*, Trevor Darrell, "Fully Convolutional Networks for Semantic Segmentation", CVPR 2015.
  2. Seunghoon Hong, Hyeonwoo Noh, Bohyung Han, "Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation", NIPS 2015
  3. Liang-Chieh Chen*, George Papandreou*, Iasonas Kokkinos, Kevin Murphy and Alan L. Yuille, "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", ICLR 2015
  4. George Papandreou*, Liang-Chieh Chen*, Kevin Murphy, and Alan L. Yuille, "Weakly- and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation", ICCV 2015
  5. Y Wei, X Liang, Y Chen, X Shen, MM Cheng, "STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation", 2015
  6. Vijay Badrinarayanan, Ankur Handa and Roberto Cipolla, "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling." arXiv preprint arXiv:1505.07293, 2015

Detection

  1. Pierre Sermanet, David Eigen, Xiang Zhang, Michael Mathieu, Rob Fergus, Yann LeCun, "OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks", ICLR 2014
  2. Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation", CVPR, 2014.
  3. Ross Girshick, "Fast R-CNN", arXiv:1504.08083
  4. Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", NIPS 2015
  5. Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, "SSD: Single Shot MultiBox Detector", ECCV 2016
  6. Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi, "You Only Look Once: Unified, Real-Time Object Detection", CVPR 2016
  7. Joseph Redmon, Ali Farhadi, "YOLO9000: Better, Faster, Stronger", 2016
  8. Zhe Zhu, Dun Liang, Songhai Zhang, Xiaolei Huang, Baoli Li, Shimin Hu, "Traffic-Sign Detection and Classification in the Wild", CVPR 2016

Graph Learning

  1. Mikael Henaff, Joan Bruna, Yann LeCun, "Deep Convolutional Networks on Graph-Structured Data", 2015
  2. Oren Rippel, Jasper Snoek, Ryan P. Adams, "Spectral Representations for Convolutional Neural Networks", NIPS 2015
  3. Michaël Defferrard, Xavier Bresson, Pierre Vandergheynst, "Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering", NIPS 2016
  4. David Duvenaud, etc,. "Convolutional Networks on Graphs for Learning Molecular Fingerprints", NIPS 2015.
  5. Yujia Li, etc., "Gated Graph Sequence Neural Networks", arXiv:1511.05493, 2015.
  6. Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov, "Learning Convolutional Neural Networks for Graphs", ICML 2016
  7. Dai, H., Dai, B., and Song, L., "Discriminative Embeddings of Latent Variable Models for Structured Data", ICML 2016
  8. Vladimir Golkov, et. al., "Protein contact prediction from amino acid co-evolution using convolutional networks for graph-valued images", NIPS 2016


Other Information

Americans with Disabilities Act

The University of Texas at Arlington is on record as being committed to both the spirit and letter of federal equal opportunity legislation; reference Public Law 93112 -- The Rehabilitation Act of 1973 as amended. With the passage of new federal legislation entitled Americans With Disabilities Act - (ADA), pursuant to section 504 of The Rehabilitation Act, there is renewed focus on providing this population with the same opportunities enjoyed by all citizens. As a faculty member, I am required by law to provide "reasonable accommodation" to students with disabilities, so as not to discriminate on the basis of that disability. Student responsibility primarily rests with informing faculty at the beginning of the semester and in providing authorized documentation through designated administrative channels.

Academic Integrity

It is the philosophy of The University of Texas at Arlington that academic dishonesty is a completely unacceptable mode of conduct and will not be tolerated in any form. All persons involved in academic dishonesty will be disciplined in accordance with University regulations and procedures. Discipline may include suspension or expulsion from the University. "Scholastic dishonesty includes but is not limited to cheating, plagiarism, collusion, the submission for credit of any work or materials that are attributable in whole or in part to another person, taking an examination for another person, any act designed to give unfair advantage to a student or the attempt to commit such acts." (Regents' Rules and Regulations, Part One, Chapter VI, Section 3, Subsection 3.2, Subdivision 3.22)

Grade Appeal Policy

If you do not believe a grade on a particular assignment is correct, you may appeal the grade in writing (email) within 5 class days. Grade appeals must be ppealed to the appropriate GTA firstly, then to your instructor if necessary. Please refer to the UTA Catalog for the detailed guide of grade appeals.

Student Support Services Available

The University of Texas at Arlington provides a variety of resources and programs to help you develop academic skills, deal with personal situations, better understand concepts and information related to their courses, and achieve academic success. These programs include major-based learning centers, developmental education, advising and mentoring, personal couneling, admission and transition, and federally funded programs. Students requiring assistance academically, personally, or socially should contact the Office of Student Success Programs at 817-272-6107 or visit www.uta.edu/resources for more information and appropriate referrals.

Academic Integrity