Google’s Custom Search Engines
I’ve been playing with Google’s latest webmaster offering - Custom Search Engines - or Googlets (mini Googles), as I like to call them. What it does is allow any webmaster to put the Google search engine on his/her site, and tailor the results to some extent. The ability to tailor the results makes it different to the familiar Google searchbox that can be put on websites.
There are two ways to tailor the results:
1. Search the entire web but emphasize included sites.
2. Search only included sites.
The system allows various websites to be entered in the control panel, and they are the sites that receive ranking boosts (first option), or they are the only sites to appear in the results (second option).
1. Search the entire web but emphasize included sites
The first option returns results from Google’s full index for any searchterm. The included sites will receive ranking boosts, but, unless they normally rank well, the boosts won’t push them high enough to do any good.
2. Search only included sites
The second option is likely to be the most common way that Custom Search Engines will be used. It still pulls from Google’s entire index, but only the pages from the included sites are returned in the results, which makes it very useful for creating niche search engines. But I found a problem with it. I included about 10 sites, which I got from doing a search on Google for the niche’s generic searchterm, and, when searching for various things within the niche’s topic, I found that the results were dominated by one or two sites - and the domination went on for page after page after page of results. For instance, the top 100 results for one search were all from just two of the included sites, except for one result in the 70s. There isn’t an option to pick certain of the included sites for a rankings boost, or an option to mix ‘em up a bit, so the sites that rank the best in Google’s normal results, dominate the custom results.
Perhaps the best way of using the Custom Search Engine is to go for option 2 (included sites only), and include as many sites on the topic as possible, in the hope of getting mixed-looking results, and use the “Refinements” system to place “labels” above the first results. Labels are links that are placed before the first result on each page of results, and they can be tailored to return only pages from certain of the included sites in the results. Different included sites can be chosen for each label.
If labels aren’t suitable, then perhaps it is best to simply include as many on-topic sites as possible, and benefit from the AdSense ads that are included with the results - yes, AdWords ads come with the results, but they can be associated with your AdSense account, so they are effectively AdSense ads for you.
Of course, benefiting from the Custom Search Engine itself isn’t necessarily an essential. Creating a niche engine as part of a site, or building a site around the engine, are very valid things to do, and the site around the engine can be used to produce benefits. Also, when the results are displayed in a site, they are displayed in an iframe, which leaves the rest of the page to benefit from. For instance, it’s easy enough to get the searchterm from the URL, and display suitable content on the page with the results - maybe that’s the place to put results from your own site.
Update:
Google doesn’t show results from the Supplemental index in the Custom Search Engine results.
PhilC said,
November 1, 2006 @ 9:53 pm
Here’s an oddity that I found…
I’ve set up a custom search engine to experiment with, and to get to know the system (http://travel.in-britain.co.uk/). I’ve set it up to return the results from my selection of sites, and I’ve added 18 sites
including my own.
When I search google.com for ‘uk holidays’, my site is ranked #5. And it’s ranked #5 when I search google.co.uk - both Web search and UK only sites. But when I search using my custom engine, which only returns pages from 18 sites, including mine, my site is ranked #50!!! And none of the other 17 selected sites outrank my site in the normal Google search.
How on Google Earth does getting the results from the whole of Google’s index show my site at #5, and yet, when it’s only competing with 18 sites, it’s ranked at #50?
Mike Valentine said,
November 2, 2006 @ 12:24 am
Phil, Are you searching while logged into your Google account? I found that when I search for client sites and my own, which are both often searched by me, that I rank significantly higher in those regular searches than when I am logged out of my Google account. (Seems not to affect the Custom results when I’m logged in.) Try logging out of your Google account - you would be logged into that account at Google co-op if working on your custom search. (I’m in the US, so your results may vary)
Also, is it possible that US sites are biased to rank higher in the Google Custom Searches? I’d be interested in an answer to this as well. I set up a custom search engine for “Small Business Health Insurance” and it shows my site at position #4 for several things I rank #1 for on regular Google results.
PhilC said,
November 2, 2006 @ 8:12 am
Hi Mike.
I tested your suggestion, and searched while both logged in and logged out, and the results were the same - #51 today in my CSE, and #6 today in a normal Google search, and none of the other sites in my CSE outrank my site in google.com, or in google.co.uk.
The sites that I’ve included are all UK sites, so it can’t be a U.S. bias in this case.
I’d love to know how the results are compiled for the CSEs. In fact, there are a number of questions that I’d like to ask Google (just posted another blog here about them), but the Groups system is broken, and I can’t get a new thread posted. I’ve tried about 10 times to post a question, and I get all the right responses, including the email, but it never gets posted. I know that they are trying to fix the problem, but it’s frustrating right now.
WilliamC said,
November 2, 2006 @ 1:44 pm
I am seeing the same results, either logged in or not also.
That would be interesting tho if they are indeed tailoring results based on our fields of work or interest in the custom results search engines.
PhilC said,
November 17, 2006 @ 10:24 am
I did some testing and found that the problem only occurs when the niche CSE has fewer that 24 sites/pages included, but the results display normally with 24 or more included sites/pages. My best guess as to the reason is that Google skips the short index when compiling the results when there are < 24 sites/pages.
The short index contains words that were found in link texts and in page Titles, and the long index contains words that were found is page content. If they skip the short index, then the rankings are arranged on page content only, and that’s what I think may be happening in this case.
Hollywood Movie Wallpapers said,
July 23, 2007 @ 3:49 am
I’m trying out Google Co-op, Google’s new custom search engine product. Basically it lets you create your own search engine including Web sites or individual pages that you select.
I have been experimenting with it and one thing I can’t figure out is how to include only one category of YA or individual pages (ie, specific questions, my profile, my answers, my best answers, my questions, etc) included in the search engine.
Has anyone tried this out?