Twitter's limited API

When I first looked at the Twitter API I was surprised to see how simple it is to use. The next thing I noticed is how little it can do. In particular, as a business intelligence guy I was struck by the lack of sophisticated query methods.

The API does not provide a way to do simple things like get a sorted list of the people you follow, which is probably ok. It's easy enough to do your own sort in your client. But what if you want to sort the people you follow by say the number of people THEY follow? To do that, you need to make an API call for each person you follow to find who he follows, and then do the sort in the client. This is pretty much hopeless for analysis purposes, especially considering the calls per hour limitations the API has and the fact that there are lots of Tweeters following 20,000+ people.

Twitter doesn't deliver some API features because it can't afford to. The "firehose" is the jargon for a real time feed of all tweets from everyone worldwide. Twitter has been promising this for some time but keeps delaying it. One reason may be that they are afraid there would be too many takers. I suspect the reason Google can offer so much storage to gmail users is that no one uses it -- like a bank hoping there won't be a run. Using an add on to store lots of data there is possible, but I think Google frowns on those shenanigans and will even block your account if you up load too fast... So as long as Twitter keeps growing at its current breakneck speed, there are some things the API won't offer because offering it could break their overstrained servers.

Another reason that Twitter might not offer some functions to their API is that they want to sell analyses as an added value service. Twitter still doesn't have a business model (except "Microsoft or Google") but analytics is an obvious option. In fact I think that providing analytics is the only real prospect that Twitter has.

In particular, Twitter has a mechanism for providing that favorite business intelligence feature -- the real time alert. For example, sighting a yellow headed blackbird in Connecticut is unusual, but the news needs to be immediate for an ornithologist to profit from it.

Twitter is also about people, not just about information. It provides relatively detailed information about who knows who. In fact I see this as the keep features of the service, so I am surprised that so little effort is invested in suppressing spam.

But whatever the specific application, the ability to analyze Twitter's database is too valuable to give away, so I suspect Twitter will not do too much to make their API better for analytics in the near future.