Deep learning-based approaches for tasks in facial analysis yield remarkable performance, but do they have any practical limits? Moreover, with the flexibility afforded by artificial neural networks, what new potentials does such technology open up? In the first part of this talk, the limits of convolutional neural networks (CNNs) for face recognition will be assessed via a new evaluation regime grounded in psychological experimentation. Scientific fields that are interested in faces have developed their own sets of concepts and procedures for understanding how a target model system (be it a person or algorithm) perceives a face under varying conditions. In computer vision, this has largely been in the form of dataset evaluation for recognition tasks where summary statistics are used to measure progress. While aggregate performance has continued to improve, understanding individual causes of failure has been difficult, as it is not always clear why a particular face fails to be recognized, or why an impostor is recognized by an algorithm. Importantly, other fields studying vision have addressed this via the use of visual psychophysics: the controlled manipulation of stimuli and careful study of the responses they evoke in a model system. In this talk, I will suggest that visual psychophysics is a viable methodology for making face recognition algorithms more explainable. A comprehensive set of procedures is developed for assessing face recognition algorithm behavior, which is then deployed over state-of-the-art CNNs and more basic, yet still widely used, shallow and handcrafted feature-based approaches.
In the second part of this talk, the potential of CNNs for describable visual facial attribute modeling will be explored. Attributes are now commonplace in human biometrics and affective computing, with existing algorithms even reaching a sufficient point of maturity for placement into commercial products. These algorithms model objective facets of facial appearance, such as hair and eye color, expression, and aspects of the geometry of the face. A natural extension, which has not been studied to any great extent thus far, is the ability to model subjective attributes that are assigned to a face based purely on visual judgments. For instance, with just a glance, our first impression of a face may lead us to believe that a person is smart, worthy of our trust, and perhaps even our admiration — regardless of the underlying truth behind such attributes. Psychologists believe that these judgments are based on a variety of factors such as emotional states, personality traits, and other physiognomic cues. But work in this direction leads to an interesting question: how do we create models for problems where there is only measurable behavior? Here I will introduce a regression framework that allows us to train predictive models of crowd behavior for social attribute assignment. Over images from the AFLW face database, these models demonstrate strong correlations with human crowd ratings.
Seminar Speaker:
Walter Scheirer
University of Notre Dame
Walter J. Scheirer, Ph.D. is an Assistant Professor in the Department of Computer Science and Engineering at the University of Notre Dame. Previously, he was a postdoctoral fellow at Harvard University, with affiliations in the School of Engineering and Applied Sciences, Dept. of Molecular and Cellular Biology and Center for Brain Science, and the director of research & development at Securics, Inc., an early stage company producing innovative biometrics solutions. He received his Ph.D. from the University of Colorado and his M.S. and B.A. degrees from Lehigh University.
Dr. Scheirer has extensive experience in the areas of human biometrics, computer vision, machine learning and artificial intelligence. His overarching research interest is the fundamental problem of recognition, including the representations and algorithms supporting solutions to it. He has made important contributions to the field of biometrics through his work on open set recognition, extreme value theory statistics for visual recognition, and template protection. His recent work has explored the intersection between neuroscience and computer science, leading to new, biologically-informed, ways to evaluate and improve algorithms.
He is very active within the biometrics and computer vision communities, having served as the program chair of IEEE/IAPR IJCB, IEEE WACV, and the SPIE Conference on Biometric and Surveillance Technology for Human and Activity Identification. Dr. Scheirer is also a regular organizer of IEEE/CVF CVPR, and sits on the board of the Computer Vision Foundation. From 2016-2019, he was an editorial board member of the IEEE Biometrics Compendium.