With revolutions springing up across the Middle East, young activists are increasingly turning to Twitter to voice their opinions and rally support. But searching Twitter only calls up tweets from the past week. Valuable pieces of history are being lost in the digital ether.
But not for long, if Ted Stein has anything to do with it. To help an academic researching the Middle East revolutions, Stein designed Twitter Miners, which collects tweets in a database based on search terms. Twitter Miners is launching within the next couple of weeks, and provides an affordable alternative to other data mining services that cost thousands of dollars a month.
“This is Twitter data mining for the masses,” said Stein, a freelance programmer and open-source Drupal contributor. He wants to make Twitter data more accessible to organizations like small businesses, nonprofits, and universities.
But how they and others will use Twitter Miners is to be determined.
“I’m excited about seeing how people will use it because I have no idea how people will use it,” Stein admitted. After hearing that Newt Gingrich deleted most of his Twitter history, Stein decided to start using Twitter Miners himself to data mine politicians’ tweets. Scot Self, a programmer who helped Stein build Twitter Miners, used the software to data mine tweets about a fellow churchgoer who had died; he plans to create a collage of those tweets, frame it, and give it to the deceased’s family.
Stein is aware that data mining social networks can raise privacy concerns. His software collects tweets up to once a minute, so posts that are immediately deleted and edited might be captured. “I would hate it if someone started stalking an ex or if a company started stalking potential job applicants by data mining all their tweets,” Stein said. But he doesn’t expect privacy concerns to be a major hurdle since Twitter data is already public. (In fact, Twitter gifted its archive of tweets since 2006 to the Library of Congress, which will make them available for research—though the details are still a little fuzzy.)
Before Stein starts looking for funding, he is looking for clients. Sign up on his website if you’d like to be one.
Recent Jobs
