Vision is arguably the most important of the five senses, and is commonly used by humans in many daily tasks (e.g., for recognizing objects or locations, as well as for safe navigation). Endowing robots with such an expert and autonomous sense of vision has been a dream of scientists and engineers for over half a century. Potential fields of interest are space exploration, home-, industrial-, and medical-robotics and image analysis.
In this course, students will be introduced to the basic techniques of autonomous vision-based robot perception, recognition, localization, and navigation. The topics covered include a description of the main robotic and vision-sensing devices (laser, stereo cams, Kinect, etc.), as well as of the basic techniques for image processing. Particular attention will be given to object recognition and robot-localization techniques. Emphasis will be given to introducing the main up-to-date strategies for vision-based robot navigation (Position- and Image-Based Visual Servoing).
Students will work individually and in groups to analyze vision-based robotics problems and to design software solutions. After successfully completing this course, students will be able to apply a variety of state-of-the-art vision techniques for the design of efficient algorithms in order to address complex problems of vision-based sensing, localization and navigation.
CSE, EE, MAE, BioE students are encouraged to register.
Course Prerequisites:
CSE 4392/5369 is self-contained and does not need special prerequisites.
The instructor is generally available before or after class and by appointment, as well as at the office hours scheduled above.
Since this is a special-topic course in robotics, and since each presented topic contains
many exciting sub-fields, the interested students who want to know more about a specific
problem are encouraged to schedule an appointment with Dr. Mariottini for additional
material.
Basics on Robotics and Computer Vision
(TOPICS: Intro to Robotics; Pinhole camera model; Epipolar Geometry Toolbox)
- 08/29/11:
- Course Introduction: What is a robot? (Part1-pdf)
- 08/31/11:
- Robot Structures (Part2-pdf);
- Computer Vision: introduction, applications (Part3-pdf - until pg.9).
- 09/07/11:
- Simplified Pinhole Camera Models (Part3-pdf - pg.9-19)
- 09/14/11:
- Introduction to EGT and 2-D stereo model (Part3-pdf - until the end) (MATLAB code).
Image-based Feature Detection, Matching and Tracking
(TOPICS: Image noise; Linear Filtering; Non-linear filtering; Edge-Detection; Corner Detection; Lucas-Kanade tracking; SIFT features)
- 09/19/11:
- Intro to Image Noise and Feature Extraction (edges, corners, etc.) (Part4-pdf - until pg.13) (MATLAB code);
- 09/21/11:
- Exercise 1 on pinhole and stereo camera models;
- Noise Models and Linear Filtering (Part4-pdf - until pg.19)
- Assigned Exercise 2 (no grade) on pinhole and distance calculation. Please email your results by Monday Sept. 28th.
- 09/28/11:
- Introduction to the OpenCV Library (Download code examples here!)
- 10/03/11:
- Rigid body kinematics: reference frames, translations, rotations (Part6-pdf - until pg.9);
- 10/05/11:
- Rigid body kinematics: 3-D transformations and exercises (Part6-pdf - until pg.16);
- 10/10/11:
- Generalized camera models (pinhole) and exercises (Part6-pdf) (MATLAB code);
- 10/12/11:
- Stereo-camera triangulation (Part6-pdf);
- 10/17/11:
- Homework Assignment correction; Project Abstract is due Monday 24th, 2011;
- 10/19/11:
- Stereo-camera triangulation (Part6-pdf);
Vision-Based Estimation, Calibration and Localization
(TOPICS: Camera Resectioning/Calibration; Multi-view Geometry; Least-Squares; Hough Transform; Camera Planar Resectioning;)
- 10/24/11:
- Intro to Camera calibration/localization (Resectioning) (Part7-pdf);
- 10/26/11:
- Camera Localization: Resectioning and SVD (Part7-pdf);
- 10/31/11:
- Projects Presentation;
- Introduction to Multiple-View Geometry and applications(Part10-pdf); - 11/2/11:
- Estimation of the essential matrix from point correspondences;
- Least-Squares Solution for Ax=b (Part8-pdf -until.pg.6)
- 11/7/11:
- Epipole and essential matrix estimation exercises
- Least-Squares Solution for Ax=b: MATLAB examples. (MATLAB code)
- 11/9/11:
- RANSAC for Robust Least Squares
- Introduction to Robot Control (Part11-pdf)
- 11/14/10:
- Introduction to Vision-based Robot Navigation (Part11-pdf);
- 11/16/10:
- Vision-based Robot Navigation: basic schemes.
- Vision-based object-following strategy (Part11-pdf) (MATLAB code);
- 11/21/10:
- Vision-Based Navigation for mobile robots: intro.
- Wall-following strategy for nonholonomic robot (Part11-pdf) (MATLAB code).
- 11/23/10:
- Trajectory planning (Part12-pdf) (MATLAB code).
- 11/28/10:
- Trajectory tracking (Part12-pdf) (MATLAB code)
- Vision-Based Navigation for mobile robots: navigation using epipolar geometry (Part12-pdf);
(please note that these course topics are preliminary and might undergo slight changes)
Matlab code
In order to run some of the code examples available in the course material, you need to download the Epipolar Geometry Toolbox (by G.L. Mariottini and D. Prattichizzo) for MATLAB. You can access here to EGT 2.0 (beta version).
Course text
There is no required textbook for this course.
Other course material [pdf of the lecture slides, technical papers, exercises,...] will be made
available on the course webpage (see above).
However, some suggested references which I've inspired to for the preparation of my course material are:
B. Siciliano, L. Sciavicco, L. Villani, G. Oriolo
"Robotics: Modelling, Planning and Control", 3rd Edition, Springer, 2009
E. Trucco, A. Verri
"Introductory Techniques for 3-D Computer Vision", Prentice Hall, 1998
R. Hartley, A. Zisserman
"Multiple View Geometry", 2nd Edition, Cambridge Univ. Press 2004
Y. Ma, S. Soatto, J. Kosecka, S. Sastry
"An Invitation to 3-D Vision. From Images to Geometric Models", Springer 2003
R. Cipolla
"Computer Vision and Robotics", Cambridge University, UK (on-line course material)
As written above, most of the material used to design the course material has been taken from the mentioned sources which I acknowledge.
Course Goals
CSE 4392/5369 is designed to:
Introduce the student to fundamentals of robotics and vision-based sensing.
Explore the mathematical/algebraic foundations of single- and multi-view geometry for pose and 3-D scene estimation.
Explore and familiarize with vision-based robot navigation techniques.
Course Outcomes
Upon successful completion of the course, each student will be able to:
Recognize the features and peculiarities of different robotic and vision devices.
Understand rigid body kinematics and relationships between 3-D reference frames.
Describe image formation and camera models. Discuss/compare the characteristics of different vision sensing devices (pinhole, omnidirectional, stereo, etc.).
Understand and discuss different strategies for feature detection, matching and tracking.
Understand and implement 3D structure and motion estimation algorithms.
Understand and design vision-based robot navigation algorithms.
In CSE 4392/5369, the students will be presented with the state-of-the-art in vision-based robot control, localization and navigation technologies. Most of the topics are active fields of research in the robotics community. I strongly encourage the students to attend each class and to actively contribute with in-class discussion, when necessary. Students must arrive on time at class.
During the whole semester, I like to interact with the participants and ask them to actively participate to complete informal small in-class exercises. These informal activities will not be graded but will be used as a feedback or plan activities.
Homeworks and Course Project
Students will be graded based on homework assignments and a major course project.
Homeworks will mostly consist of theoretical questions, programming assignments in MATLAB, etc. related to the topics of the course.
Dates for the homeworks will be announced in class. Regarding homeworks policies, please refer also to the Course Policy section.
The course project will focus on designing and implementing a particular algorithm from a list provided in class, and walking through details related to its analysis and design (see here a list of projects from last year!)
For their final project, students will have the possibility to use up to 4 iCreate robotic platforms (by iRobot) for the course project. USB cameras, laptops, and Kinects are also available to program the robot and process data from the sensors.
Additional projects related to the course topics and proposed by students can be added to the list. In evaluating the projects design, particular attention will be given to the projects that excel in creativity and effectiveness of their result. The course project can be done either individually or in a team.
Students will report their findings in a 8-10 page research report, and an in-class oral presentation (with Power Point slides) during last day of class.
For students enrolled in the graduate section CSE 5369 the homework assignments, as well as the course project, will contain additional problems which are not required for students of CSE 4392.
Tentatively, course grades will based on the following:
Assignments
% of final grade
Homeworks (total)
50 %
Course Project
40 %
- Proj. Presentation
10 %
Percent Grade
Letter
90 %-100 %
A
80 %-89.9 %
B
70 %-79.9 %
C
60 %-69.9 %
D
<60 %
F
Final Project/Homework Late Submission Policy:
Late submissions for the final project will be penalized according to:
- Late by 1-24 hrs: 5% deducted from actual score.
- Late by 24-48 hrs: 25% deducted from actual score.
- Late by 48-72 hrs: 45% deducted from actual score.
- Late by more than 72 hrs: Will receive a zero.
Attendance and Participation
As stated above, attendance is strongly suggested at the first day and each class
session. Students are encouraged to arrive on time and attend the full class period.
Participants who need to miss class for religious observance or for a pressing
personal or family matter, should contact the instructor prior to missing class or as soon as
possible. Participants should plan on getting the information about the missed class from a
peer.
I strongly encourage in-class collegial behaviour as well as between the project group
members. NON collegial behavior includes working on other tasks during class time (text
messaging, e-mailing, Web surfing, doing crosswords/Sudoku, having private conversations,
etc.). Another example of non collegial behavior could be the creation of unconstructive
conflicts inside a group.
Finally, I positively value the students' active participation to in-class discussions. This is
extremely important because gives the instructor (and the students too!) a feedback on the
audience understanding.
Academic Honesty
All students are expected to pursue their academic careers with honesty and integrity.
"Scholastic dishonesty includes, but is not limited to, cheating, plagiarism, collusion, the
submission for credit of any work or materials that are attributable in whole or in part to
another person, taking an examination for another person, any act designed to give unfair
advantage to a student or the attempt to commit such acts" (Regents' Rules and
Regulations, Part One, Chapter VI, Section 3, Subsection 3.2, Subdivision 3.22.).
Students found guilty of dishonesty in their academic pursuits are subject to penalties that
may include suspension from the university. Any student found guilty of academic dishonesty
will receive a -100% for that work (homeworks, project, etc.) as well as having the
course grade lowered one full letter grade - in addition to any other penalties assessed
(suspension, expulsion, probation). These and other applying UTA rules, will be strictly
enforced. Any case of academic dishonesty will be treated in accordance with the UTA
Handbook of Operating Procedures or the Judicial Affairs website at
http://www2.uta.edu/discipline. If you do not understand this policy, it is your
responsibility to obtain clarification or any additional information you may require.
Students are allowed to discuss homework with classmates, but are not allowed to copy
the solutions of others or share solutions with others. All work turned in for grading must
be the student's own work.
Accommodations for Students With Disabilities
I will do my best to provide, on a flexible and individualized basis, reasonable
accommodations to students who have documented disability conditions (e.g., physical,
learning, psychiatric, vision, hearing, or systemic) that may affect their ability to participate in
course activities or to meet course requirements. If you require any accommodation based on
disability, please meet with the Instructor (with your supporting papers) in the privacy of his
office the first week of the semester to be sure you are appropriately accommodated.
Grievance Procedure
Anyone feeling that a dispute exists after the grading of any assignment or exam may
submit a written grievance. This grievance should identify the item in dispute and
arguments supporting the student's position. Grievances must be submitted in writing
within two class periods following the return of the assignment.
The instructor agrees to return a written response to the student's grievance within two
class periods from receipt of the grievance. If the error is due to wrongful calculation of points,
then no grievance needs to be submitted. If a written grievance is received, the instructor
reserves the right to re-grade the entire exam (not just the specific point in question).
Student Support Services Available
The University of Texas at Arlington supports a variety of student success programs to
help you connect with the University and achieve academic success. These programs
include learning assistance, developmental education, advising and mentoring, admission
and transition, and federally funded programs. Students requiring assistance academically,
personally, or socially should contact the Office of Student Success Programs at 817-272-6107
for more information and appropriate referrals.
Electronic Communication Policy
The University of Texas at Arlington has adopted the University "MavMail" address as
the sole official means of communication with students. MavMail is used to remind
students of important deadlines, advertise events and activities, and permit the University
to conduct official transactions exclusively by electronic means. For example, important
information concerning registration, financial aid, payment of bills, and graduation are
now sent to students through the MavMail system.
All students are assigned a MavMail account. Students are responsible for checking their
MavMail regularly. Information about activating and using MavMail is available at
http://www.uta.edu/oit/email/ .
File translated from
TEX
by
TTH,
version 3.85. On 28 Nov 2011, 12:53.
