by Devin Gaffney, Ian Pearce, Max Darham, and Max Nanis

Hello world. It’s been awhile since the Web Ecology community last made a peep on the web. Some had been speculating that we had simply up and disappeared, but reports of our demise were greatly exaggerated, as they say.

Here’s what we’ve got.

Thanks to the completely amazing work of our affiliate researchers at Bennington, we’re glad today to announce the public launch of 140Kit, Web Ecology’s very own free-to-use toolkit for exploring and data mining Twitter. It’s the final product of the various provisional tools we’ve used to produce our previous reports on the social phenomena of Twitter, and of lead researcher Devin Gaffney’s own work on high throughput humanities.

So what does it do? Notably:

• It enables complete data pulls for a set of users or terms on Twitter, with searches running continuously.
• The ability to download those data pulls in raw form to use for whatever you please.
• The ability to stand on the shoulders of giants by mixing and matching existing data pulls to generate entirely new combinations of data and analysis.
• And the ability to instantaneously generate basic visualizations around the data (term use, inequality of participation, etc).

Best of all, we are making this platform open and free to use for all interested users. This includes opening up an API for queries of all sorts, an honest, open, and editable codebase, and plans already in the works to make the program extensible to allow developers to write their own analytics for the kit, on whatever sort of metrics in whatever programming language (stay tuned for details).

So, get in there and play, people. And let us know if you have any questions! tim.hwang@webecologyproject.org or contact@webecologyproject.org.

By Jon Beilin

One of the tenets of Web Ecology is accessibility to the field through open tools and open data. At the Web Ecology Project, we’re working to get more of our code in a clean, commented, and releasable state. The first tool that we have queued up for release is a Python module allowing easy use of Google Language Tools, involving language detection and translation, with transliteration in an experimental state (Google has not yet released the API spec for the transliteration portion so that was reverse-engineered).

Now for some sample uses of the tool:

>> from googlelanguage import *

>> print lang_translate("comment dit on 'WebEcology' en francais?", dest_lang="en")
{'translatedText': 'how it says 'WebEcology' in French?', 'detectedSourceLanguage': 'fr'}


We used it ourselves to detect the language of each tweet in a sample of 1 million tweets from our database, with the following results:

We’ve also found it easy to combine the tool with SQLAlchemy to create metadata tables with linguistic information.

It is our hope that this small, MIT/X11-licensed release will prove useful to some in the Web Ecology community. Until we figure out which platform we’re going to use for open repository hosting, you can download the file here. And if you would like to contribute patches or additions, or if you have any questions, feel free to send them to Jon.Beilin@webecologyproject.org

I would also like to thank Sam Gilbert for his invaluable contributions, feedback, and support.