home   |   research   |   teaching   |   publications   |   biosketch   |   CV

University of Texas at Arlington - Fall 2010

CSE 4392/5369

Special Topics in
Vision-based Robot Sensing, Localization and Control

1  - Course Overview
2  - Course Material, Topics and Goals
3  - Course Assignments, Exam and Policies
4  - Course/University Policies and Services

Class Schedule:
   Mon. & Wed., 4:00-5:20pm
Office Hours:
   Mon. & Wed., 1:30-3:30pm or by appointment
Course Website:
   http://ranger.uta.edu/ ~ gianluca/teaching/CSE4392-5369

   Dr. Gian Luca Mariottini
   Web: http://ranger.uta.edu/ ~ gianluca
   Email: gianluca@uta.edu
   Computer Science and Engineering Dept.
   University of Texas at Arlington
   Nedderman Hall 309, 416 Yates Street
   Arlington, TX 76019-0015
Downloads: [Syllabus] [Flyer]

1  - Course Overview

Vision is arguably the most important of the five senses, and is commonly used by humans in many daily tasks (e.g., for recognizing objects or locations, as well as for safe navigation). Endowing robots with such an expert and autonomous sense of vision has been a dream of scientists and engineers for over half a century. Potential fields of interest are space exploration, home-, industrial-, and medical-robotics.
In this course, students will be introduced to the basic techniques of autonomous vision- based robot perception, recognition, localization and navigation. The topics covered include a description of the main robotic and vision-sensing devices, as well as of the basic techniques for image processing. Particular attention will be given to recognition and 3D scene and robot estimation techniques that use single and multiple images. Emphasis will be given to introducing the main up-to-date strategies for vision-based robot navigation (Position-, Image- and Hybrid-Based Visual Servoing).
Throughout the course, students will work individually and in groups to analyze vision-based robotics problems and to design software solutions. Matlab will be the primary programming language/environment used in the assignments. After successfully completing this course, students will be able to apply a variety of techniques for the design of efficient algorithms in order to address complex problems of vision-based sensing, localization and navigation.
CSE, EE, MAE, BioE students are encouraged to register.

Course Prerequisites:

Office Hours:

The instructor is generally available before or after class and by appointment, as well as at the office hours scheduled above.
Since this is a special-topic course in robotics, and since each presented topic contains many exciting sub-fields, the interested students who want to know more about a specific problem are encouraged to schedule an appointment with Dr. Mariottini for additional material.

2  - Course Material, Topics and Goals

  1. Basics on Robotics and Computer Vision
    (TOPICS: Intro to Robotics; Pinhole camera model; Epipolar Geometry Toolbox)

        - 08/30/10:
            - Course Introduction: What is a robot? (Part1-pdf)
        - 09/01/10:
            - Robot Structures (Part2-pdf);
            - Computer Vision: introduction, applications (Part3-pdf - until pg.14).
        - 09/08/10:
            - Simplified Camera Models (Part3-pdf - pg.14-26)
        - 09/13/10:
            - Introduction to EGT and 2-D stereo model (Part3-pdf - until the end) (MATLAB code).
  2. Feature Detection, Matching and Tracking
    (TOPICS: Image noise; Linear Filtering; Non-linear filtering; Edge-Detection; Corner Detection; Lucas-Kanade tracking; SIFT features)

        - 09/13/10:
            - Intro to Image Noise and Feature Extraction (edges, corners, etc.) (Part4-pdf - until pg.13) (MATLAB code);
        - 09/15/10:
            - Exercise 1 on pinhole and stereo camera models;
            - Noise Models and Linear Filtering (Part4-pdf - until pg.19)
        - 09/20/10:
            - Non-linear Filtering, Edge Detection, Corner Detection (Part4-pdf - pg.19 until end)
            - Kanade-Lucas Tracker (Part5-pdf - until pg.12);
            - Exercise 2 on pinhole and distance calculation.
        - 09/22/10:
            - SIFT features: Keypoint and Descriptors extraction; Matching. (Part5-pdf - until end) (MATLAB code);
  3. Generalized Camera Models
    (TOPICS: Rigid-body kinematics; Generalized camera-model)

        - 09/27/10:
            - Rigid body kinematics: ref. frames, rotations (Part6-pdf - until pg.10) (MATLAB code);
        - 09/29/10:
            - Rigid body kinematics: 3-D transformations and exercises (Part6-pdf - until pg.16);
        - 10/03/10:
            - Rigid body kinematics: generalized camera models (pinhole, stereo) (Part6-pdf - until end);
            - Assignment No.1 is out (start: Oct.4th, 2010 at 5:20pm; deadline: Oct.13th, 2010 at 4:00pm!) (check Sect. 3 for more details).
  4. Vision-Based Estimation and Localization
    (TOPICS: Camera Resectioning/Calibration; Least-Squares; Hough Transform; Camera Planar Resectioning; Stereo motion reconstruction; Multi-view Geometry)

        - 10/06/10:
            - Generalized stereo camera models (Part6-pdf - until end);
            - Intro on Camera calibration/localization (Resectioning) (Part7-pdf);
        - 10/11/10:
            - Camera Localization: Resectioning and SVD (Part7-pdf);
        - 10/13/10:
            - Projects Presentation;
            - Least-Squares Solution for Ax=b (Part8-pdf -until.pg.8)
        - 10/18/10:
            - Least-Squares Solution for Ax=b: Applications and RANSAC (Part8-pdf -until end) (MATLAB code);
            - Assignment No.1: Correction of exercises.
        - 10/20/10:
            - Hough Transform (MATLAB code)
            - Camera Localization from planar pattern (Part9-pdf -5) (MATLAB code);
            - Assignment (no grade): see function Ex_CameraCalibrationIncomplete.m in the MATLAB code
        - 10/25/10:
            - Camera Localization from planar pattern (Part9-pdf -until end) (MATLAB code);
        - 10/27/10:
            - Introduction to Multiple-View Geometry and applications(Part10-pdf) (MATLAB code);
        - 11/1/10:
            - Multiple-view Geometry - Essential Matrix (Part10-pdf) (MATLAB code);
        - 11/3/10:
            - Multiple-view Geometry - Estimation and Scene reconstruction (Part10-pdf) (MATLAB code);
            - Assignment No.2 is out (start:Nov.3rd, 2010 at 5:20pm; deadline: Nov.13th, 2010 at 4:00pm!) (the code of the solution is available here).
        - 11/8/10:
            - Multiple-view Geometry - Homography matrix (Part10-pdf) (MATLAB code);
  5. Vision-Based Robot Navigation
    (TOPICS: Position-based visual Servoing; Trajectory generation and tracking; Image-Based Visual Servoing)

        - 11/11/10:
            - Introduction to Vision-based Robot Navigation (Part11-pdf);
        - 11/15/10:
            - Vision-based Robot Navigation: basic schemes.
            - Vision-based object-tracking strategy (Part11-pdf);
        - 11/17/10:
            - Vision-Based Navigation for mobile robots: intro.
            - Trajectory planning (Part12-pdf) (MATLAB code).
        - 11/22/10:
            - Trajectory tracking (Part12-pdf) (MATLAB code).
            - Vision-Based Navigation for mobile robots: navigation using epipolar geometry (Part12-pdf);
        - 11/24/10:
            - Vision-Based Navigation for mobile robots: navigation using epipolar geometry.
        - 11/29/10:
            - Image-Based Visual Servoing: Introduction (Part13-pdf)
        - 12/1/10:
            - Image-Based Visual Servoing (Part13-pdf)
        - 12/6/10: Project Presentations.
        - 12/8/10: Project Presentations.
        - 12/14/10: Project Presentations.
(please note that these course topics are preliminary and might undergo slight changes)

Matlab code

In order to run some of the code examples available in the course material, you need to download the Epipolar Geometry Toolbox (by G.L. Mariottini and D. Prattichizzo) for MATLAB. You can access here to EGT 2.0 (beta version).
Once you have downloaded it, include the EGT directory in your MATLAB path. At this point you should be ready to run the examples.
In order to run other MATLAB code you need to have the Image Processing Toolbox for MATLAB installed with your version.

Course text

There is no required textbook for this course. Other course material [pdf of the lecture slides, technical papers, exercises,...] will be made available on the course webpage (see above). However, some suggested references which I've inspired to for the preparation of my course material are: As written above, most of the material used to design the course material has been taken from the mentioned sources which I acknowledge.

Course Goals

CSE 4392/5369 is designed to:

Course Outcomes

Upon successful completion of the course, each student will be able to:

3  - Course Assignments, Exam and Policies

Class attendance and participation

In CSE 4392/5369, the students will be presented with the state-of-the-art in vision-based robot control, localization and navigation technologies. Most of the topics are active fields of research in the robotics community. I strongly encourage the students to attend each class and to actively contribute with in-class discussion, when necessary. Students must arrive on time at class.
During the whole semester, I like to interact with the participants and ask them to actively participate to complete informal small in-class exercises. These informal activities will not be graded but will be used as a feedback or plan activities.

Homeworks and Course Project

Students will be graded based on homework assignments and a major course project.
Homeworks will mostly consist of theoretical questions, programming assignments in MATLAB, etc. related to the topics of the course.
To download the first assignment please login to WebCT with your UTA NetID. Once entered, you will be able to access to your specific section for this course (CSE 4392 or CSE 5369). Click on the Assignment you are interesting in accessing (e.g., Assignment No.1) and download the text of the exercise. Follow the instructions contained in there for the submission policy.
Dates for the homeworks will be announced in class. Regarding homeworks policies, please refer also to the Course Policy section.
The course project will focus on designing and implementing a particular algorithm from a list provided in class, and walking through details related to its analysis and design. For their final project, students will have the possibility to use up to 4 iCreate robotic platforms (by iRobot) for the course project. USB cameras and laptops are also available to program the robot and process data from the sensors.
Additional projects related to the course topics and proposed by students can be added to the list. In evaluating the projects design, particular attention will be given to the projects that excel in creativity and effectiveness of their result. The course project can be done either individually or in a team. Students will report their findings in a 8-10 page research report, and an in-class oral presentation (with Power Point slides) during last day of class.
For students enrolled in the graduate section CSE 5369 the homework assignments, as well as the course project, will contain additional problems which are not required for students of CSE 4392.

4  - Course/University Policies and Services

Tentatively, course grades will based on the following:
Assignments % of final grade
Homeworks (total) 50 %
Course Project 40 %
       - Proj. Presentation 10 %
Percent Grade Letter
90 %-100 % A
80 %-89.9 % B
70 %-79.9 % C
60 %-69.9 % D
<60 % F

Final Project/Homework Late Submission Policy:

Late submissions for the final project will be penalized according to:

Attendance and Participation

As stated above, attendance is strongly suggested at the first day and each class session. Students are encouraged to arrive on time and attend the full class period.
Participants who need to miss class for religious observance or for a pressing personal or family matter, should contact the instructor prior to missing class or as soon as possible. Participants should plan on getting the information about the missed class from a peer.
I strongly encourage in-class collegial behaviour as well as between the project group members. NON collegial behavior includes working on other tasks during class time (text messaging, e-mailing, Web surfing, doing crosswords/Sudoku, having private conversations, etc.). Another example of non collegial behavior could be the creation of unconstructive conflicts inside a group.
Finally, I positively value the students' active participation to in-class discussions. This is extremely important because gives the instructor (and the students too!) a feedback on the audience understanding.

Academic Honesty

All students are expected to pursue their academic careers with honesty and integrity. "Scholastic dishonesty includes, but is not limited to, cheating, plagiarism, collusion, the submission for credit of any work or materials that are attributable in whole or in part to another person, taking an examination for another person, any act designed to give unfair advantage to a student or the attempt to commit such acts" (Regents' Rules and Regulations, Part One, Chapter VI, Section 3, Subsection 3.2, Subdivision 3.22.).
Students found guilty of dishonesty in their academic pursuits are subject to penalties that may include suspension from the university. Any student found guilty of academic dishonesty will receive a -100% for that work (homeworks, project, etc.) as well as having the course grade lowered one full letter grade - in addition to any other penalties assessed (suspension, expulsion, probation). These and other applying UTA rules, will be strictly enforced. Any case of academic dishonesty will be treated in accordance with the UTA Handbook of Operating Procedures or the Judicial Affairs website at http://www2.uta.edu/discipline. If you do not understand this policy, it is your responsibility to obtain clarification or any additional information you may require.
Students are allowed to discuss homework with classmates, but are not allowed to copy the solutions of others or share solutions with others. All work turned in for grading must be the student's own work.

Accommodations for Students With Disabilities

I will do my best to provide, on a flexible and individualized basis, reasonable accommodations to students who have documented disability conditions (e.g., physical, learning, psychiatric, vision, hearing, or systemic) that may affect their ability to participate in course activities or to meet course requirements. If you require any accommodation based on disability, please meet with the Instructor (with your supporting papers) in the privacy of his office the first week of the semester to be sure you are appropriately accommodated.

Grievance Procedure

Anyone feeling that a dispute exists after the grading of any assignment or exam may submit a written grievance. This grievance should identify the item in dispute and arguments supporting the student's position. Grievances must be submitted in writing within two class periods following the return of the assignment.
The instructor agrees to return a written response to the student's grievance within two class periods from receipt of the grievance. If the error is due to wrongful calculation of points, then no grievance needs to be submitted. If a written grievance is received, the instructor reserves the right to re-grade the entire exam (not just the specific point in question).

Student Support Services Available

The University of Texas at Arlington supports a variety of student success programs to help you connect with the University and achieve academic success. These programs include learning assistance, developmental education, advising and mentoring, admission and transition, and federally funded programs. Students requiring assistance academically, personally, or socially should contact the Office of Student Success Programs at 817-272-6107 for more information and appropriate referrals.

Electronic Communication Policy

The University of Texas at Arlington has adopted the University "MavMail" address as the sole official means of communication with students. MavMail is used to remind students of important deadlines, advertise events and activities, and permit the University to conduct official transactions exclusively by electronic means. For example, important information concerning registration, financial aid, payment of bills, and graduation are now sent to students through the MavMail system. All students are assigned a MavMail account. Students are responsible for checking their MavMail regularly. Information about activating and using MavMail is available at http://www.uta.edu/oit/email/ .

File translated from TEX by TTH, version 3.85.
On 6 Dec 2010, 07:47.