Is This Film Review Algorithm Better than Ebert?

A new study out of Northwestern University suggests that an algorithm is as good or even better at identifying quality films than a movie critic. Using the film's "significance" as the key denominator, the group of scientists led by Professor Luís Amaral studied how many times a movie was referenced in another movie. Based on these results, the group was able to pick out the films with the most cultural impact, essentially placing the vote into the hands of fellow movie directors rather than critics.

Published in this week's issue of Proceedings of the National Academy of Sciences (PNAS), the study explains how the team of scientists broke down 15,425 US films by various metrics: critical reviews, awards, public opinion, citations and box office sales before finding citations to be the best signifier of quality. By comparing the results of each approach to the particular movie's inclusion in or exclusion from the National Film Registry of the U.S. Library of Congress, the team was able to narrow down its approach.

This particular algorithm was most successful when measuring films that are 25 years old or more, with The Wizard of Oz, Star Wars, Psycho, Casablanca and Gone With the Wind coming out on top. They also found that some films like Willy Wonka and The Chocolate Factory weren't cited for a significant stretch of time before blowing up on the cultural spectrum. Amaral also says that initially underappreciated films such as these, while not currently in the National Film Registry, may very well be included in the future.

"Twenty-five years may seem like a long time to wait before we can begin quantifying film significance," concludes the study. "However, significance by definition may not be readily apparent. This is true of other forms of art, as well as any other field where influence spreads. There is a reason the Nobel Prize is no longer awarded for research done in the same year. A film’s significance should ultimately be judged on how its ideas influence filmmaking and culture in the longterm."

Amaral hopes to take his findings on significance and apply them to scientific papers, paintings and music next.