And what I meant to say is that after indexing the top 100 documents, you’d scan the content of these documents to get your relevant keywords.

I agree on what you are saying. But it seems like you have took the question and turned it around. What you have explained is taking a category and mapping it to keywords. That is not really what I wanted. Sure, it’ll be hard for the system to figure out that a colloid i in milk, pepsi, coca cola, and more. It seems like you’ve taken it a bit too far.

What I was trying to do is figure out a way to take a keyword, and then find the most relevant category. And that IS possible, but it does require data. I should have probably asked, what kind of data that we find on the internet can we use in order to get the most accurate categories for the biggest amount of keywords. Or possibly what collection of data.

My original idea worked, sort of. And it is what you sort of explain. The idea was to use data from search engines. Take a keyword such as “banking” and run it across the top 100 sites listed on Google. Since Google is a good source for relevant information, you can bank on the fact that they will provide you mainly with accurate top 100 results.

By indexing all those top 100 documents, you’ll then need a separate technology that will get all the related keywords to your main keyword.

So let’s say that you now got your top 10 related keywords which would be something like:

Music
Mp3
Portable Device
Computers
Apple

You’d be sure to get the top most repeat keywords / keyword phrases (and that would require yet another technology), that do not contain the word “IPOD”.

Now you would go back to your list of let’s say – 500 categories. Each category would need to have related keywords. So the category “Music” would need to have keywords that would be “music,mp3,rap,hiphop,pop,rock,etc”.

My idea was to now run the related keywords that you found against the related keywords of the categories. The category that has the most matches would take the spot as being the most relevant category.

The reason I liked this system was because if you have a new product that just came out, for instance let’s say IPOD was just announced yesterday – you now know that Google will definitely have it listed.

And since the categories would contain terms that could be found in the English dictionary, then you don’t need to worry about having category related terms of newly released products, therefore the categories don’t need updates, or possibly only a few updates every few years.

The only service you’d rely on is Google to get you the recent and relevant web pages for your initial search term.

But at the end of the day, this is too hard. And once again, I fall into a problem where I will be able to map the keyword “Ipod” or “Ipod Devices” to “Music” category. But, once again, how do I map something to the category “Recreation”? It’s not the same ball game again. And, how can I map something like “United States” to category “World”? I mean, yes I can map it, but that would now require me to make specific rules for countries / cities / states to map to a “LOCAL” category.

The question is, how can we use the data that is out there to build such a system?