Christian Heilmann

Posts Tagged ‘keywords’

Decreasing the server load of Keywordfinder with YQL

Sunday, July 4th, 2010

Back in the days when Yahoo BOSS came out, I built Keywordfinder to show off the “keywords” functionality of BOSS. BOSS returns you keywords people entered for a certain web site in its results and I used that to list the most popular search terms for a term you enter.

Now, yesterday I got an alert by my host that Keywordfinder is exceeding my GPU limit and checking the logs I realized that it is heavily hit by users from South America. So I decided to add a JavaScript layer to the app that would take away some of the traffic and a lot of the computation from the server.

I could have just replicated the PHP functionality in JavaScript but I thought it is more beneficial for everybody to turn the functionality into a YQL table and use Yahoo’s servers to do the heavy lifting for me.

You can see the keywordfinder open table on GitHub and it will be soon replicated over to datatables and show up in the YQL console. For now you can apply it using the “use” command:

use “http://github.com/yql/yql-tables/raw/master/keywordfinder/keywordfinder.xml” as keywordfinder;
select * from keywordfinder where term=”oil spill”;

try it in the console or see the results as XML

The results right now are: oil, oil spill, spill, the Gulf of Mexico, BP, the Gulf, BP oil, Oil Spill, Deepwater, gallons, Gulf of Mexico, Gulf oil, Horizon oil, crude oil, Gulf, Gulf Coast, Gulf Oil, oil rig, birds, Mexico oil.

The table returns the search term, the region, the language and the terms. It sanitizes all of them for HTML display. By default, the table uses English as the language and USA as the region. If you provide a different language and region you get other results:

use “http://github.com/yql/yql-tables/raw/master/keywordfinder/keywordfinder.xml” as keywordfinder;
select * from keywordfinder where term=”fussball weltmeister”
and language=”de” and region=”de”;

try it out in the console or see the result as XML.

I think it is a nice example how you can speed up an app and help others by adding the functionality to YQL. Enjoy.

Keywordfinder.org is live

Monday, November 24th, 2008

It’s been a while since I released a full web app (no, shouldiusetablesforlayout doesn’t count), so I thought I spend a few hours and created keywordfinder.org:

Screenshot of Keywordfinder.org

It is a service that returns the 20 most successful keywords for a search term you enter and the first 20 sites that come up when you do a search online.

Hopefully this is useful to some, I just had fun playing with BOSS and some YUI CSS to create the whole thing from scratch in one morning :)

WordPress Plugin to add Yahoo Keywords to your posts

Friday, November 14th, 2008

Continuing my playing around with Yahoo BOSS, I’ve put together a quick WordPress plugin to add keywords to any of your blog posts.

Simply Download the plugin unpack it to your plugin directory and activate it.

Once activated you can use two commands in your blog posts:

[ yahookeywords:parisweb]

This will be replaced by a list of keywords – an HTML list of 10 terms with a CSS class called “yahoo-keywords”.

[ yahootags:parisweb]

This will be replaced by the same list, but each of the terms will be linked as a tag back to your blog.

Live examples:

[yahookeywords:parisweb]
[yahootags:parisweb]

Yahoo BOSS keyword extraction API wrappers (JS/PHP)

Thursday, November 13th, 2008

One of my favourite “old school” Yahoo APIs is the term extractor which is a service that extracts relevant keywords from a text you give it.

Yahoo BOSS is now supporting this feature for indexed web sites. While you’d normally just get a list of sites with for example:

http://boss.yahooapis.com/ysearch/web/v1/donkeys?format=xml&appid={appid}

You can get the keywords for each of the pages returned by adding the (so far undocumented) view=keyterms parameter:

http://boss.yahooapis.com/ysearch/web/v1/donkeys?format=xml&view=keyterms&appid={appid}

This can be pretty useful to get a list of keywords related to a certain term.

In order to do this, I’ve written a small API in PHP and JavaScript that gets you the related terms from the first ten search results and returns them as an array.

The PHP API wrapper

The PHP version takes three parameters: the mandatory term to search for, an optional callback method name to wrap around the JSON return value and an optional format parameter that can be set to HTML to return an HTML list instead of a JSON object.

The JavaScript API wrapper

The JavaScript wrapper uses dynamically generated script nodes to retrieve the data and can be used by simply calling a BOSSTERMS.get() method with a search term and the name of a callback method. The return object has a term property, the keywords as an array and a string that is an HTML list of the terms.

Get the lot

You can download the whole BOSS keyword API here. As always, it is BSD licensed, so go nuts using it :)