EFACE - a face composite system

1. EFACE - a face composite system Colin Tredoux David Nunez, Oliver Oxtoby, Yon Rosenthal, Lisa da Costa & Bhavesh Prag Department of Psychology University of Cape Town December 2001

2. Introduction • Witnesses are often required to attempt the construction of a facial likeness • A variety of technologies are used, ranging from sketch artists to Photo-composites to software compositors (e.g. Identikit, E-Fit) • Empirical research shows that these produce poor quality composites, which are difficult to match to target faces • We have developed E-Face, a configural, eigenface compositor as an alternative

3. Existing technologies • Sketch artists • Manual montage(featural) systemse.g. Identikit, Photofit • Electronic montage (featural) systems e.g. E-Fit; Mac-a-mug • Eigenface technologiese.g. Spot-it, Evo-fit, E-Face

4. Problems with existing technologies • Poor quality composites - ‘unfacelike’ - even when the target face is in full view • Produce poor matching to target faces in laboratory studies • most empirical tests of composite systems show matching results little better than chance levels (e.g. Laughery & Fowler, 1980) • Computerised systems do not produce better reconstructions than manual systems • e.g. Kovera, Penrod, Pappas & Thill (1997)

5. Reasons for composite problems • Dependency on featural composition • Most psychological research points to the importance of configuration in face recognition. • Inherently limited nature of facial feature databases • Absence of a representational or computational theory • e.g. a theory underlying the kinds of features that are likely to appear together • Fallibility of human memory • Need for memory analysis of the task

7. E-Face - an eigenface compositor

8. Capture • Faces are difficult to photograph technically • Must balance exposure and depth of field • Flash highlights introduce artefact in analysis • Subjects find it difficult to orient their faces so that they are in a consistent geometrical relation to the film plane • All of these problems must be corrected, or the Eigenface analysis produces unusable images

9. Warp • Faces must be normalized before they can be treated as sufficiently ‘similar’ to process as data points in PCA • The best method we could find for doing this was to extract ‘texture’ and shape information separately • Each face is landmarked with a number of points, which define the shape, and are used as the vertices for a set of triangular tesselations, and the faces are then bi-linearly mapped into the average face shape

10. Eigen • PCA is conducted on the normalized faces, to derive ‘eigenfaces’ • Each face in the dataset can then be expressed as a weighted sum of the eigenfaces • E.g. Face 1 = 0.12 E1 + 0.03 E2 + … + Ek • Faces not already in the dataset can also be expressed as a weighted sum, but the representation will have a small amount of error

11. ID • Search can proceed with either of two algorithms (PBIL vs Mchoice), in shape-free or shape-variable modes • User interface designed to be simple – goal is for use to be totally transparent to witness; no expert operator needed

12. E-Face - key ideas 1 • A set of face images can be represented in a common principal component (PC) space • A new face - i.e. not in the original set - can be reconstructed by projecting the face into PC space • To put this another way, for every face, there is a representation by a unique set of PC coefficients • Therefore, the task of reconstructing a face is the same as finding such a set of coefficients • This can be conceptualised as an optimization problem, and standard optimization techniques can be applied

13. E-Face - key ideas 2 • PBIL (population based incremental learning) was identified as one useful technique by Rosenthal (1998), and applied to the problem • A random set of faces is generated, using random estimates for each of the PC coefficients • The witness chooses the best match (i.e. closest to the perpetrator) in this set of faces • The coefficients of the ‘best match’ are used to re-set the random sampling, or the witness decides that no better match is possible.

14. E-Face - key ideas 3 • An alternate ‘averaging’ algorithm was devised, since simulations showed PBIL to be unreasonably slow

15. E-Face - advantages • Face construction is entirely configural; faces are always constructed as complete entities, and not as montages of features • E-Face enables the witness to use recognition memory, and does not depend on recall memory ?? • A good approximation to the target face is guaranteed, if the database of faces is a reasonable match to the group the target is drawn from • Statistical validity for composite faces, translates to photographic quality images

16. Sample reconstruction – shape only

18. Sample reconstruction – shape + texture