Syllabus

Syllabus

January 23: Introduction

Introduction to the problem. Brainstorming about solutions. Course organization. Reading assignments.

Data and support code in /course/cs296-4/Heads/

January 30: Models of face shape and appearance

Readings:

For each assigned paper (each person has 2) come prepared to lead a discussion of the paper and send me a paragraph containing a short critique (ie something that you think is not so good about the paper).

Everybody reads this:

A Morphable Model for the Synthesis of 3D Faces.
Blanz, V. and Vetter, T. SIGGRAPH'99 Conference Proceedingspdf,ps.gz

Read one of the following (as assigned in class):

Reanimating Faces in Images and Video, Blanz, V., Basso, C., Poggio, T. and Vetter, T.
In: Brunet, P and Fellner, D (Eds) Proceedings of EUROGRAPHICS 2003, Granada, Spain, 2003. pdf EG2003 Best Paper Award (1st Prize)

Exchanging Faces in Images, Blanz, V., Scherbaum, K., Vetter, T. and Seidel, H. P.
In: Cani, M.-P. and Slater, M. (Eds) Proc. of EUROGRAPHICS 2004,
Grenoble, France, 2004. pdf EG2004 Best Paper Award (1st Prize)

Face Recognition Based on Fitting a 3D Morphable Model
Blanz, V. and Vetter, T.
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.25 no. 9, pp 1063 - 1074, 2003. pdf

Assignment 1 due.

February 6: Camera calibration

Guest presentations by Matt Leotta and Matt Loper.

Reading for all: Criminisi, Single-view Metrology: Algorithms and Applications

Assignment 2 due (unless the majority of you tell me it can't be done in time).

February 13: Face tracking/fitting continued.

We will continue with Blanz and Vetter's optimization and the fitting of the 3D model to selected 2D image points.

For your reference for the homework:

Haralick, R. M., Lee, C., Ottenberg, K., and Nolle, M. (in /course/cs29604/Papers/haralick.pdf)

New Readings (ie papers that you should read and send me a summary/critque):

Everyone: M.R. Everingham and A. Zisserman. "Identifying individuals in video by combining generative and discriminative head models". In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV2005), pages 1103-1110, October 2005. [PDF]

Extra:
V. Lepetit and P. Fua, Monocular Model-Based 3D Tracking of Rigid Objects: A Survey, Foundations and Trends in Computer Graphics and Vision, Vol. 1, Nr. 1, pp. 1 - 89, October 2005.

P. Fua. Regularized Bundle-Adjustment to Model Heads from Image Sequences without Calibration Data , International Journal of Computer Vision, 38(2):153--171, 2000.
http://cvlab.epfl.ch/publications/publications/2000/Fua00.pdf

Assignment 3 due.

February 20: Optimization/fitting

Main readings:

Accurate Non-Iterative O(n) Solution to the PnP Problem
F.Moreno-Noguer, V.Lepetit, P.Fua
Proc. International Conference on Computer Vision (ICCV), 2007 (pdf) (code) (slides)

M. Dimitrijevic, S. Ilic, and P. Fua. Accurate Face Models from Uncalibrated and Ill-Lit Video Sequences. In Conference on Computer Vision and Pattern Recognition, Washington, DC, June 2004.
http://citeseer.ist.psu.edu/cache/papers/cs/30361/http:zSzzSzcvlab.epfl.chzSz~siliczSzpublicationzSzPCA_Face_CVPR04.pdf/accurate-face-models-from.pdf

Assignment 3 Task 3 & 4 due.

For assignment 3, I suggest using the Moreno-Noguer method for initialization, then do tasks 3 and 4.

Clicking assignments also due

(put your points in /course/cs296-4/Calib711/lmarks/frame###lmarks#.mat)

Februrary 27: Active Appearance Models and (more) 3D tracking

Main readings:

Section 2.1 on pose estimation:

Synthesizing Realistic Facial Expressions from Photographs
Frederic Pighin Jamie Hecker Dani Lischinskiy Richard Szeliskiz David H. Salesin

http://citeseer.ist.psu.edu/pighin98synthesizing.html

Resolution-Aware Fitting of Active Appearance
Models to Low Resolution Images
G¨oksel Dedeoˇglu, Simon Baker, and Takeo Kanade (PDF) (web)

Background reading on AAMs

T.F. Cootes, G.J. Edwards, and C.J. Taylor: Active Appearance Models. IEEE
Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, June,
2001, pp. 681-685. (web) (pdf)

More on AAMs

http://www.ri.cmu.edu/projects/project_448.html

Extra

L. Vacchetti, V. Lepetit and P. Fua, Stable Real-Time 3D Tracking Using Online and Offline Information, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, Nr. 10, pp. 1391 - 1391, 2004.
http://cvlab.epfl.ch/publications/publications/2004/VacchettiLF04.pdf

Assignment 4 Part 1 due: Multiple frames.

March 5: Super resolution

Main readings:

* Farsiu et al, Fast and robust multiframe super-resolution

* Farsiu et al, Advances and challenges in super-resolution

http://www.soe.ucsc.edu/~milanfar/research/resolution-enhancement.html

Some other SR papers

* Capel and Zisserman, Super-resolution form multiple views using learnt image models

* Zomet and Peleg Efficient super-resolution and applications to mosaics

Good background/overview

* Irani and Peleg, Improving image resolution by image registration

* David Capel and Andrew Zisserman, Computer Vision Applied to Super-resolution

Assignment 4 Part 2 due: Multiple frames and optimization of shape.

Form 4 teams for Assignment 5.

March 12: Fitting the whole body

Guest speaker Alexandru Balan

Reading:

Detailed human shape and pose from images,
Balan, A., Sigal, L., Black, M. J., Davis, J., Haussecker, H.,
Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Minneapolis, June 2007.
(pdf)

Shining a light on human pose: On shadows, shading and the estimation of pose and shape,
Balan, A., Black, M. J., Haussecker, H., Sigal, L.,
to appear: Int. Conf. on Computer Vision, ICCV, Rio de Janeiro, Brazil, October 2007.
(pdf)

Background paper

D. Anguelov, P.Srinivasan, D.Koller, S.Thrun, J. Rodgers, J.Davis. SCAPE: Shape Completion and Animation of People. Proceedings of the SIGGRAPH Conference, 2005.

Project proposals due.

March 19: 3D tracking

Illumination

Facial Shape-from-shading and Recognition Using Principal Geodesic Analysis and Robust Statistics
William A. P. Smith, Edwin R. Hancock

http://www.springerlink.com/content/j648420k441v767p/

Extra:

A Manifold Approach to Face Recognition from Low Quality Video Across Illumination and Pose using Implicit Super-Resolution
O. Arandjelovic' and R. Cipolla. In Proc. IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil, October 2007.
http://mi.eng.cam.ac.uk/~oa214/academic/openfile.html?publications/2007_ICCV_paper1.pdf

Lorenzo Torresani, Aaron Hertzmann, and Christoph Bregler. Non-Rigid Structure-From-Motion: Estimating Shape and Motion with Hierarchical Priors. IEEE Transactions on Pattern Analysis and Machine Intelligence. To appear. [PDF] . A Matlab implementation here.

Background: Brand, M.E., "Morphable 3D Models from Video", IEEE Computer Vision and Pattern Recognition (CVPR), December 2001

http://www.merl.com/publications/TR2001-037/

Brand, M.E.; Bhotika, R., "Flexible Flow for 3D Nonrigid Tracking and Shape Recovery", IEEE Computer Society Conference on Computer vision and Pattern Recognition (CVPR), ISSN: 1063-6919, Vol. 1, pp. 315-322, December 2001

http://www.merl.com/reports/docs/TR2001-38.pdf

Fitting a Single Active Appearance Model Simultaneously to Multiple Images
C. Hu, J. Xiao, I. Matthews, S. Baker, J. Cohn, and T. Kanade
Proceedings of the British Machine Vision Conference, September, 2004. [Abstract]
Download: pdf