A team from MIT has created an algorithm that can predict what Twitter topics will trend hours in advance. Wired reports that Associate Professor Devavrat Shah and his student Stanislav Nikolov from the Laboratory for Information and Decision Systems can use the algorithm to predict with 95% accuracy which words, phrases or hashtags are going to trend. It can predict these an average of an hour and a half before Twitter’s algorithm puts them on the trending list, and sometimes as much as five hours before.
The algorithm is trained by combing through data in a sample set about previous topics that did and didn’t trend and it tries to find meaningful patterns. In Shah and Nikolov’s experiments, the set consisted of data on 200 Twitter topics that trended and 200 that didn’t. In real-time, they set their algorithm loose on live tweets, predicting trending with 95% accuracy and a 4% false-positive rate.
The algorithm could be of great interest to Twitter and also represents a new approach to statistical analysis that could be applied to other aspects like ticket sales for films or even stock prices.