http://googleresearch.blogspot.com.au/2013/05/syntactic-ngrams-over-time.html

We are proud to announce the release of a very large dataset of counted dependency tree fragments from the English Books Corpus. This resource will help researchers, among other things, to model the meaning of English words over time and create better natural-language analysis tools. The resource is based on information derived from a syntactic analysis of the text of millions of English books.

Sentences in languages such as English have structure. This structure is called syntax, and knowing the syntax of a sentence is a step towards understanding its meaning. The process of taking a sentence and transforming it into a syntactic structure is called parsing. At Google, we parse a lot of text every day, in order to better understand it and be able to provide better results and services in many of our products.

Now if there was just an API for it… Those things get to big to keep local and by nature should be dynamically updated.

I was thinking the same thing, Carl, or to be more accurate, I thought,

each dataset’s compressed files takes tens of gigabytes

Holy CRAP, that’s a lot of information!!!

So yes, an API would be really helpful, I think.

What features would you like to see in an API?

By API I’m assuming that you mean some kind of “Software as a Service” which you can query online and not have to worry about keeping up to date yourself. If you take a look at projects like DBPedia and YAGO2 you’ll find lists of existing services based on those knowledge bases. Are those the kinds of things that you want to see implemented for this new batch of data from Google?

If I understand his posts correctly, Dan Hughes and his team have been working on something like that, and for what it’s worth I’m working towards a similarly ambitious goal too.

Andrew Smith - May 25, 2013:

What features would you like to see in an API?

Ability to query by “natural language” (some user input converted to plain text as a minimum, though image and/or sound will become more important as input) and have returned some subset of info:
-sentence tree
-root
-example(s)

This could make a even simple chat bot seem very sophisticated indeed when a “similarity engine” is used to generate an otherwise unscripted response in situ.

You might have to wait a while before you can have access to what sounds a lot like an IBM Watson of your very own, but in the meantime you might find this cornucopia of machine learning APIs extremely useful in your projects.

http://blog.mashape.com/post/48074869493/list-of-40-machine-learning-apis

Be sure to go on to look at the rest of the APIs that this site has to offer as there is a huge variety of them supplying real-time data on any number of topics from sports to weather to travel and more. Many of them offer free and/or limited access so you can try them out with your chatbot project without risking anything more than a little bit of time and effort.

https://www.mashape.com/