Ep 130: Using Twitter to predict the future

Ever wondered what good Twitter actually does? Personally, I love it, but really, is it anything but noise? One of the pipe dreams for online social media is the ability to track opinions and interests in real time. In their paper Predicting the Future With Social Media, Sitaram Asur and Bernardo A. Huberman have not only tracked live opinion on movies, but used it to predict their future success. Asur and Huberman, from the Social Computing Lab, HP Labs California, have shown that the rate of tweeting about a movie accurately predicts its opening weekend box office revene. After examining the rate of chatter from almost 3 million movie tweets, the researchers constructed a linear regression model for predicting box-office revenues of movies in advance of their release. These results outperformed the Hollywood Stock Exchange, a market in which people can buy and sell virtual shares in actors, directors and individual movies and produces unusually accurate predictions of film popularity. There is a strong correlation between the amount of tweets concerning a forthcoming film, and its opening weekend box office return. The rate of tweeting about a movie was determined by simply counting the number of tweets containing the movie name. The next step was to predict box office returns beyond the opening weekend, and this was achieved by including "sentiment" as a factor. Sentiment analysis is a fascinating area of linguistic study. Language classifiers were used to label the text associated with the movie tweet as Positive, Negative or Neutral. Adding these as factors into the regression significantly increased the researchers' ability to predict the box office returns beyond the opening weekend. These results are intuitive - before a movie is released, potential viewers do not know whether they will like the movie and so simply the number of tweets about a movie gives an indicator of movie "buzz" and correlates with the number of people attending the opening weekend. Once a movie is released and people start forming opinions, movie tweets start to contain sentiment. Negative tweets, whilst they have little effect on the opening box office as no one has yet seen the film, have a strong influence on further returns. Likewise for positive tweets. The question of cause and effect is very interesting. Does a high number of tweets about a movie actually cause a strong box office return, or are they correlated simply because the twitter and movie audience are arguably the same? Another way of asking this question is to ask whether an advertiser could change future box office returns by deliberately tweeting multiple times or with a particular sentiment. I had a fascinating chat with Sitaram about this work.