home   |   research   |   teaching   |   publications   |   biosketch   |   CV

University of Texas at Arlington - Fall 2011
CSE 4392/5369

Special Topics in
Vision-based Robot Sensing, Localization and Control

1  - Course Overview
2  - Course Material, Topics and Goals
3  - Course Assignments, Exam and Policies
4  - Course/University Policies and Services

Class Schedule:
   Mon. & Wed., 4:00-5:20pm
Office Hours:
  Mariottini: Mon. & Wed., 1:30-3:30pm or by appointment
  G. Puerto (TA): Mon. & Wed., 5:20-6:30pm
Course Website:
   http://ranger.uta.edu/ ~ gianluca/teaching/CSE4392-5369

   Dr. Gian Luca Mariottini
   Web: http://ranger.uta.edu/ ~ gianluca
   Email: gianluca@uta.edu
   Computer Science and Engineering Dept.
   University of Texas at Arlington
   ERB 649, 500 UTA Blvd.
   Arlington, TX 76019-0015
Downloads: [Course Syllabus], [Course Flyer]

1  - Course Overview

Vision is arguably the most important of the five senses, and is commonly used by humans in many daily tasks (e.g., for recognizing objects or locations, as well as for safe navigation). Endowing robots with such an expert and autonomous sense of vision has been a dream of scientists and engineers for over half a century. Potential fields of interest are space exploration, home-, industrial-, and medical-robotics and image analysis.
In this course, students will be introduced to the basic techniques of autonomous vision-based robot perception, recognition, localization, and navigation. The topics covered include a description of the main robotic and vision-sensing devices (laser, stereo cams, Kinect, etc.), as well as of the basic techniques for image processing. Particular attention will be given to object recognition and robot-localization techniques. Emphasis will be given to introducing the main up-to-date strategies for vision-based robot navigation (Position- and Image-Based Visual Servoing).
Students will work individually and in groups to analyze vision-based robotics problems and to design software solutions. After successfully completing this course, students will be able to apply a variety of state-of-the-art vision techniques for the design of efficient algorithms in order to address complex problems of vision-based sensing, localization and navigation.
CSE, EE, MAE, BioE students are encouraged to register.

Course Prerequisites:

Office Hours:

The instructor is generally available before or after class and by appointment, as well as at the office hours scheduled above.
Since this is a special-topic course in robotics, and since each presented topic contains many exciting sub-fields, the interested students who want to know more about a specific problem are encouraged to schedule an appointment with Dr. Mariottini for additional material.

2  - Course Material, Topics and Goals

  1. Basics on Robotics and Computer Vision
    (TOPICS: Intro to Robotics; Pinhole camera model; Epipolar Geometry Toolbox)

        - 08/29/11:
            - Course Introduction: What is a robot? (Part1-pdf)
        - 08/31/11:
            - Robot Structures (Part2-pdf);
            - Computer Vision: introduction, applications (Part3-pdf - until pg.9).
        - 09/07/11:
            - Simplified Pinhole Camera Models (Part3-pdf - pg.9-19)
        - 09/14/11:
            - Introduction to EGT and 2-D stereo model (Part3-pdf - until the end) (MATLAB code).
  2. Image-based Feature Detection, Matching and Tracking (TOPICS: Image noise; Linear Filtering; Non-linear filtering; Edge-Detection; Corner Detection; Lucas-Kanade tracking; SIFT features)

        - 09/19/11:
            - Intro to Image Noise and Feature Extraction (edges, corners, etc.) (Part4-pdf - until pg.13) (MATLAB code);
        - 09/21/11:
            - Exercise 1 on pinhole and stereo camera models;
            - Noise Models and Linear Filtering (Part4-pdf - until pg.19)
            - Assigned Exercise 2 (no grade) on pinhole and distance calculation. Please email your results by Monday Sept. 28th.
        - 09/28/11:
            - Introduction to the OpenCV Library (Download code examples here!)
  3. Generalized Camera Models, Kinect, Stereo Cameras
    (TOPICS: Rigid-body kinematics; Generalized camera-model)

        - 10/03/11:
            - Rigid body kinematics: reference frames, translations, rotations (Part6-pdf - until pg.9);
        - 10/05/11:
            - Rigid body kinematics: 3-D transformations and exercises (Part6-pdf - until pg.16);
        - 10/10/11:
            - Generalized camera models (pinhole) and exercises (Part6-pdf) (MATLAB code);
        - 10/12/11:
            - Stereo-camera triangulation (Part6-pdf);
        - 10/17/11:
            - Homework Assignment correction; Project Abstract is due Monday 24th, 2011;
        - 10/19/11:
            - Stereo-camera triangulation (Part6-pdf);
  4. Vision-Based Estimation, Calibration and Localization
    (TOPICS: Camera Resectioning/Calibration; Multi-view Geometry; Least-Squares; Hough Transform; Camera Planar Resectioning;)

        - 10/24/11:
            - Intro to Camera calibration/localization (Resectioning) (Part7-pdf);
        - 10/26/11:
            - Camera Localization: Resectioning and SVD (Part7-pdf);
        - 10/31/11:
            - Projects Presentation;
            - Introduction to Multiple-View Geometry and applications(Part10-pdf);
        - 11/2/11:
            - Estimation of the essential matrix from point correspondences;
            - Least-Squares Solution for Ax=b (Part8-pdf -until.pg.6)
        - 11/7/11:
            - Epipole and essential matrix estimation exercises
            - Least-Squares Solution for Ax=b: MATLAB examples. (MATLAB code)
        - 11/9/11:
            - RANSAC for Robust Least Squares
            - Introduction to Robot Control (Part11-pdf)
  5. Vision-Based Robot Navigation
    (TOPICS: Position-based visual Servoing; Trajectory generation and tracking; Wall following; Image-Based Visual Servoing)

        - 11/14/10:
            - Introduction to Vision-based Robot Navigation (Part11-pdf);
        - 11/16/10:
            - Vision-based Robot Navigation: basic schemes.
            - Vision-based object-following strategy (Part11-pdf) (MATLAB code);
        - 11/21/10:
            - Vision-Based Navigation for mobile robots: intro.
            - Wall-following strategy for nonholonomic robot (Part11-pdf) (MATLAB code).
        - 11/23/10:
            - Trajectory planning (Part12-pdf) (MATLAB code).
        - 11/28/10:
            - Trajectory tracking (Part12-pdf) (MATLAB code)
            - Vision-Based Navigation for mobile robots: navigation using epipolar geometry (Part12-pdf);
(please note that these course topics are preliminary and might undergo slight changes)

Matlab code

In order to run some of the code examples available in the course material, you need to download the Epipolar Geometry Toolbox (by G.L. Mariottini and D. Prattichizzo) for MATLAB. You can access here to EGT 2.0 (beta version).

Course text

There is no required textbook for this course. Other course material [pdf of the lecture slides, technical papers, exercises,...] will be made available on the course webpage (see above). However, some suggested references which I've inspired to for the preparation of my course material are: As written above, most of the material used to design the course material has been taken from the mentioned sources which I acknowledge.

Course Goals

CSE 4392/5369 is designed to:

Course Outcomes

Upon successful completion of the course, each student will be able to:

3  - Course Assignments, Exam and Policies

Class attendance and participation

In CSE 4392/5369, the students will be presented with the state-of-the-art in vision-based robot control, localization and navigation technologies. Most of the topics are active fields of research in the robotics community. I strongly encourage the students to attend each class and to actively contribute with in-class discussion, when necessary. Students must arrive on time at class.
During the whole semester, I like to interact with the participants and ask them to actively participate to complete informal small in-class exercises. These informal activities will not be graded but will be used as a feedback or plan activities.

Homeworks and Course Project

Students will be graded based on homework assignments and a major course project.
Homeworks will mostly consist of theoretical questions, programming assignments in MATLAB, etc. related to the topics of the course.
Dates for the homeworks will be announced in class. Regarding homeworks policies, please refer also to the Course Policy section.
The course project will focus on designing and implementing a particular algorithm from a list provided in class, and walking through details related to its analysis and design (see here a list of projects from last year!) For their final project, students will have the possibility to use up to 4 iCreate robotic platforms (by iRobot) for the course project. USB cameras, laptops, and Kinects are also available to program the robot and process data from the sensors.
Additional projects related to the course topics and proposed by students can be added to the list. In evaluating the projects design, particular attention will be given to the projects that excel in creativity and effectiveness of their result. The course project can be done either individually or in a team. Students will report their findings in a 8-10 page research report, and an in-class oral presentation (with Power Point slides) during last day of class.
For students enrolled in the graduate section CSE 5369 the homework assignments, as well as the course project, will contain additional problems which are not required for students of CSE 4392.

4  - Course/University Policies and Services

Tentatively, course grades will based on the following:
Assignments % of final grade
Homeworks (total) 50 %
Course Project 40 %
       - Proj. Presentation 10 %
Percent Grade Letter
90 %-100 % A
80 %-89.9 % B
70 %-79.9 % C
60 %-69.9 % D
<60 % F

Final Project/Homework Late Submission Policy:

Late submissions for the final project will be penalized according to:

Attendance and Participation

As stated above, attendance is strongly suggested at the first day and each class session. Students are encouraged to arrive on time and attend the full class period.
Participants who need to miss class for religious observance or for a pressing personal or family matter, should contact the instructor prior to missing class or as soon as possible. Participants should plan on getting the information about the missed class from a peer.
I strongly encourage in-class collegial behaviour as well as between the project group members. NON collegial behavior includes working on other tasks during class time (text messaging, e-mailing, Web surfing, doing crosswords/Sudoku, having private conversations, etc.). Another example of non collegial behavior could be the creation of unconstructive conflicts inside a group.
Finally, I positively value the students' active participation to in-class discussions. This is extremely important because gives the instructor (and the students too!) a feedback on the audience understanding.

Academic Honesty

All students are expected to pursue their academic careers with honesty and integrity. "Scholastic dishonesty includes, but is not limited to, cheating, plagiarism, collusion, the submission for credit of any work or materials that are attributable in whole or in part to another person, taking an examination for another person, any act designed to give unfair advantage to a student or the attempt to commit such acts" (Regents' Rules and Regulations, Part One, Chapter VI, Section 3, Subsection 3.2, Subdivision 3.22.).
Students found guilty of dishonesty in their academic pursuits are subject to penalties that may include suspension from the university. Any student found guilty of academic dishonesty will receive a -100% for that work (homeworks, project, etc.) as well as having the course grade lowered one full letter grade - in addition to any other penalties assessed (suspension, expulsion, probation). These and other applying UTA rules, will be strictly enforced. Any case of academic dishonesty will be treated in accordance with the UTA Handbook of Operating Procedures or the Judicial Affairs website at http://www2.uta.edu/discipline. If you do not understand this policy, it is your responsibility to obtain clarification or any additional information you may require.
Students are allowed to discuss homework with classmates, but are not allowed to copy the solutions of others or share solutions with others. All work turned in for grading must be the student's own work.

Accommodations for Students With Disabilities

I will do my best to provide, on a flexible and individualized basis, reasonable accommodations to students who have documented disability conditions (e.g., physical, learning, psychiatric, vision, hearing, or systemic) that may affect their ability to participate in course activities or to meet course requirements. If you require any accommodation based on disability, please meet with the Instructor (with your supporting papers) in the privacy of his office the first week of the semester to be sure you are appropriately accommodated.

Grievance Procedure

Anyone feeling that a dispute exists after the grading of any assignment or exam may submit a written grievance. This grievance should identify the item in dispute and arguments supporting the student's position. Grievances must be submitted in writing within two class periods following the return of the assignment.
The instructor agrees to return a written response to the student's grievance within two class periods from receipt of the grievance. If the error is due to wrongful calculation of points, then no grievance needs to be submitted. If a written grievance is received, the instructor reserves the right to re-grade the entire exam (not just the specific point in question).

Student Support Services Available

The University of Texas at Arlington supports a variety of student success programs to help you connect with the University and achieve academic success. These programs include learning assistance, developmental education, advising and mentoring, admission and transition, and federally funded programs. Students requiring assistance academically, personally, or socially should contact the Office of Student Success Programs at 817-272-6107 for more information and appropriate referrals.

Electronic Communication Policy

The University of Texas at Arlington has adopted the University "MavMail" address as the sole official means of communication with students. MavMail is used to remind students of important deadlines, advertise events and activities, and permit the University to conduct official transactions exclusively by electronic means. For example, important information concerning registration, financial aid, payment of bills, and graduation are now sent to students through the MavMail system. All students are assigned a MavMail account. Students are responsible for checking their MavMail regularly. Information about activating and using MavMail is available at http://www.uta.edu/oit/email/ .

File translated from TEX by TTH, version 3.85.
On 28 Nov 2011, 12:53.