Analyzing Search Results for Fun and Profit

In a previous post, we looked at the first two components of the success of search: the density of highlighted search terms and the placement within the search engine results page. However, there are times when the science of a search engine does not match the art of marketing.

It is the role of a web manager and of marketing in general to understand the voice of the customer. They place themselves in the role of a customer and try to understand what a customer is trying to accomplish when performing a task. Additionally, the marketing and web management roles should have an understanding of the profitability of certain products versus other products and how generating sales in one versus the other will affect the bottom line of the company.

Oftentimes, a marketer will have a list of what results he or she will want to see given a search. It is possible to evaluate how the SERP compares to the marketer’s ideal list using a similar approach to the Results Utility Positioning calculation that we discussed previously. Conceptually, this is similar to the judgment list we discussed when covering how to index Chinese in Solr.

The Results Position Analysis (RPA) score is a factor of two attributes related to the SERP and the marketer’s judgment list. The first one has to do with actual matching. Does a result in the judgment list actually appear somewhere on the SERP? If so, this is positive. The second attribute has to do with the distance from the result on the SERP compared to the location in the judgment list. A result in the SERP which is tenth on the page doesn’t have much value when the judgment list has it first, and, similarly, a result that is first on the SERP doesn’t have much value if the judgment list has it showing up tenth.

We normalize results to a 0 – 100 score and add it to the RUP score to come up with a cumulative score. The RPA and the RUP should, to some extent, inform each other. Perhaps the marketer has an unrealistic ideal of what should be coming up. If someone searches for red shoes, blue elephants will never sell. On the other hand, if the marketer’s judgment list is fairly reasonable, a low RUP score should indicate failures of content or in the search algorithm to cause the quality of the SERP to match the judgment list.

Combined, RUP and RPA can be a very powerful measurement tool to improve the profitability of search within an e-commerce website. Site search is often ignored, particularly because there aren’t many known metrics to evaluate both the qualitative (RPA) and quantitative (RPA) qualities of the search engine results page.

If you need metrics on the viability of your search engine results pages, please contact me or e-mail us at talktous at opensourceconnections dot com.