Miriam Redi, Daniele Quercia, Lindsay T. Graham, Samuel D. Gosling
Like Partying? Your Face Says It All. Predicting the Ambiance of Places with Profile Pictures
In Proceedings of the 9th International AAAI Conference on Web and Social Media (ICWSM), 2015
Best Paper Award
Can you guess a restaurant's ambiance from its customers’ faces? Not only people but also algorithms can.
In 2012, two psychologists from UT Austin - Lindsay T. Graham and Samuel D. Gosling - studied whether people could tell a place’s ambiance solely based on its customers’ faces. They selected 49 Foursquare venues in Austin, and asked a group of students to physically go to each of those places and rate the places’ ambiances along 72 district ambiance types (e.g., whether a place was for party folks or for creative people). The researchers then collected the Foursquare profile pictures of 25 users who have been at each of those venues and showed them to another group of students. This group could largely assess the ambiance of the place just based on the faces of those who have been there.
That comes as no surprise as it has been shown that a face reflects one’s biography, history, personality traits, and emotions. Since computer vision has been recently used to automatically infer those very aspects from faces, the two Austin researchers teamed up with Miriam Redi (a computer vision researcher at Yahoo Labs) and Daniele Quercia (a computational social scientist, formerly at Yahoo Labs) to answer the following question: Can an algorithm automatically determine a place’s ambiance by analyzing its customers’ faces?
To answer that question, the researchers built a learning algorithm that is able to associate ambiance ratings with visual features of the profile pictures. Those features came from Redi's previous work on computational aesthetics. Her work showed that brightness, contrast, saturation, presence of circles, and symmetry are all features associated with image quality and beauty; that colors are associated with feelings (e.g., yellow tends to be associated with cheerfulness); and that face expression, age, gender, race can all be automatically extracted by existing vision algorithms.
The researchers then fed those features to a regression algorithm and found that it works surprisingly well - the algorithm was able to predict ambiance variables with an error consistently below 12%. Also, by looking at the correlations among the ambiance ratings, the researchers were able to reduce the initial 72 ambiance dimensions into 25 categories, producing the first ambiance wheel for places (png). Interestingly, when examining profile pictures, the students and the algorithm used different visual cues at times: the algorithm tended to look at structural elements in the pictures, while humans relied on color attributes and demographic traits. The students associated the presence of women with romantic and pick-up places, while the algorithm simply associated warm color pictures with romantic and pick-up places. At times, however, the students and the algorithm agreed: they both associated friendly places with smiley people, “strange” places with people who shy away from showing faces in their profiles, and nerdy places with folks with reading glasses.
This work has shown surprisingly strong associations between faces and ambiance. One practical implication of these findings is the ability to recommend places with the right ambiance to social-networking users or, conversely, to recommend fit-for-purpose employees for a business (e.g., the faces to show on a coffee shop’s web page). On a more theoretical note, the work has also shown that people rely on expected stereotypes when judging faces. For example, they associated the presence of female with romantic places, despite the algorithm not finding any statistical evidence for that.
The research paper can be found here.
For more information, please contact: Daniele Quercia, Research Scientist; email: firstname.lastname@example.org; skype: dquercia
Notes for editors: The place ambiance project involves Miriam Redi (Yahoo Labs), Daniele Quercia (formerly at Yahoo Labs, now visiting at University of Cambridge), Lindsay T. Graham (University of Texas, Austin), and Samuel D. Gosling (University of Texas, Austin). This research won the best paper award at the AAAI International Conference on Web and Social Media (ICWSM) 2015 in Oxford, the premiere conference on social media. In any news article covering this research, a link to this site (or to the original research paper) might be beneficial.