Prediction of Non-Linear Aging Trajectories of Faces

1. Prediction of Non-Linear Aging Trajectories of Faces • K. Scherbaum, M. Sunkel, V. Blanz and H.-P. Seidel • [ 2007/5/9, Eurographics 2007, Prague ]

2. Motivation / Goal • automated growth-prediction system • applications • photofit-pictures of missing children • automated animation, art Kristina Scherbaum scherbaum@mpi-inf.mpg.de

3. Kristina Scherbaum scherbaum@mpi-inf.mpg.de 11 years 10 years 9 years Age Progression – Optimal Case child 1 child 2 child 3 face space

4. Kristina Scherbaum scherbaum@mpi-inf.mpg.de 11 years 10 years 9 years Real Case – Support Vector Regression • only 1 sample per person • no longitudinal study • find isosurfaces • and gradients 11 years 9 years 10 years Runge-Kutta Integration face space

5. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Main Assumption - Curved Trajectories • use machine learning • non-linear Support Vector Regression • integration of local age-gradient growing faces transform along curved trajectories

6. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Challenges • learn change over time of individual facesnon-linear dependency on time, curved trajectory • learn how the change depends on individual facenon-linear dependency in face space • sparse dataset, no longitudinal study

7. Kristina Scherbaum scherbaum@mpi-inf.mpg.de 3D Morphable Facemodel • System is based on a Morphable 3D Facemodel [Blanz,Vetter‘99] • Built from 200 3D-face-scans of adults

8. Kristina Scherbaum scherbaum@mpi-inf.mpg.de 3D Morphable Facemodel • vector space of faces • vectors with point-to-point correspondence Shape Texture

9. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Representation of Faces - Face Spaces • PCA to reduce dimensionality (yields coefficients)

10. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Extended Morphable Model • Extension by … • plus ~238 facemodels of teenagers • 3 simultaneous laser scans per face • Correspondence by … • top-down approach • fitting Morphable Model to new 3D faces • merging original data and best fit

11. Fitting the Morphable Model to 3D Scans • no optical flow because scans are often incomplete merged result best fit of the morphable model 3D laser scans Kristina Scherbaum scherbaum@mpi-inf.mpg.de

12. 1 Age Progression Algorithm • learn function • that maps any face x to a scalar age y to learn this function we use … • non-linear Support-Vector-Regression • on training sets of lpairs Kristina Scherbaum scherbaum@mpi-inf.mpg.de

13. Kristina Scherbaum scherbaum@mpi-inf.mpg.de x Fitting a Regression Curve 2 • for a given set of samples find f(x) • such that all samples are within an e-tube • preselect e • and tradeoff between smoothness and errors of outliers y e e x • Linear: f(x) = wx + b • Non-linear: fis sum of Gaussian RBF kernels K(x-xi)

14. Non-Linear SVM Regression 2 • Gaussian RBF (Radial Basis Function) as kernel • we applied grid search using cross validation • to optimize parameters such as g(Kernelwidth) • iand b are determined by SVM training • using LIBSVM for e-Support Vector Regression Kristina Scherbaum scherbaum@mpi-inf.mpg.de

15. 3 Local Aging • Isosurfaces are defined in PCA space • Gradient gives shortest path to next isosurface • Along the gradient … • many facial changes due to aging • almost no other changes (known technique, Blanz et al. 99) • Thus: Compute growth along the gradient! Kristina Scherbaum scherbaum@mpi-inf.mpg.de

16. Kristina Scherbaum scherbaum@mpi-inf.mpg.de 3 male female Gradient Example - Facial Attributes • gender manipulation original

17. Kristina Scherbaum scherbaum@mpi-inf.mpg.de 3 Growth Simulation: New Approach • growth curve with given face x0 at time t • currently we compute the local gradient • and walk along this gradient • instead we should compute the curved trajectory

18. 4 Runge Kutta Integration Solve differential equation … • to compute curved trajectories • integrate the differential equation • using Runge-Kutta algorithm • perform small steps Kristina Scherbaum scherbaum@mpi-inf.mpg.de

19. 4 Visualized Aging Trajectories Kristina Scherbaum scherbaum@mpi-inf.mpg.de

20. Kristina Scherbaum scherbaum@mpi-inf.mpg.de 4 Reducing Complexity • we did not train on all principle components • speedup of SVM training • we experimented with 20, 40 or 80 PCs • Justification … • growth leads to overall change of facial size • significant changes are represented by the first PCs [ large variance ] • facial growth should happen in the first PCs

21. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Growth Example • growth simulation for both, shape and texture

22. Kristina Scherbaum scherbaum@mpi-inf.mpg.de More Examples 10 years 3D laser scans, original age 12 12 10 14 13 15 years 20 years 30 years

23. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Rendering the Result into Images [EG’04] Background, Haircut Pose, Light Face Composed Result 3D reconstruction and aging

24. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Photofit Picture Example • Possible appearances at the age of 17 • Input at the age of 11

25. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Aging in Images - Example Picture (1999) Different prediction renderings 3D reconstruction and aging Ground truth pictures (2005)

26. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Linear vs. Non-Linear • Linear age progression • perform linear regression (yields a function)[ straight-forward least squares fit ] • transform faces also along the gradient • Disadvantages … • the gradient is constant [ linear function ] • each face moves along the same straight trajectory

27. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Linear vs. Non-Linear • comparison of age estimation error (in months) • mean squared training and generalization errors • non-linear SVM regression behaves superior! • generalization indicates: no overfitting

28. Kristina Scherbaum scherbaum@mpi-inf.mpg.de • Have different faces distinct trajectories?Mean angle of trajectories of different faces 15.7º 33.5º  the trajectories are different Remember the Challenges • Are growth trajectories curved?Mean angle between start- and target-tangent 10.3º 30.0º  the trajectories are curved, not linear

29. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Conclusions • Results … • aging involves non-linear components • trajectories are distinct for different individuals • linear systems are a reasonable approximation • technique works without longitudinaldata • But … • more data would be helpful • longitudinal data would allow for exact evaluation

30. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Thank you for your attention! MOVIE

31. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Representation of Faces - Face Spaces • arbitrary faces by linear combinations of examples • PCA to reduce dimensionality (yields coefficients)

32. 4 Aging Trajectories Main Idea … • compute aging trajectories z(t) • locally along gradient of the aging function f(x) • and going through a start vector or face x0: Kristina Scherbaum scherbaum@mpi-inf.mpg.de

33. Kristina Scherbaum scherbaum@mpi-inf.mpg.de Aging Information • extracted from the database of 200 adult face scans • and new database of 238 face scans of teenagers teenager overview

34. Kristina Scherbaum scherbaum@mpi-inf.mpg.de