Analyzing Eclipse Update Site Traffic with AWStats

knowledge is power . i don't think it is necessary to explain why having some analysis of your sites' visitors is important. as a software vendor, it is crucial to understand how many people download the software from your site. if you are developing an eclipse plugin, you probably have an eclipse update site for distributing the software to your customers (whether they are paying or not). the update site is used both for new users, installing your software for the first time, and for existing users updating their software to newer versions. that means tracking downloads is somewhat different than tracking downloads from a regular web site.

the most popular tool for web site traffic analysis is google analytics . like many other tools , google analytics works in the following manner: the user views the page along with a tracking javascript on the page (also known as page tagging ). the javascript, upon loading the page, sends the information to a main server. this works great for web sites but it is completely useless for eclipse update sites. the transactions of the update sites are not done using a full web browser, so the javascript cannot execute.

the simple solution in this case is to take a different approach to tracking visitors: analyzing the server logs . the web server can keep logs of all the requests going through the server. these logs can be analyzed using a log analysis tool. the most popular tool for that is awstats .

awstats is a free, open source tool (under gpl license) for analyzing server logs. it can be set up to use a web interface for presenting reports on site visitors. setting up awstats is a fairly simple task. there are many tutorials and articles on the subject, so i won't be covering this part. so, before you continue to the next paragraph make sure you have awstats up and running.

configuring awstats for update site analytics

in its' default configuration, awstats will not present correct update site statistics. the main reason for that is the browser identity. when eclipse connects to your site, it will identify itself as a "java" agent, not as a browser. this causes awstats to assume your client is a bot or a worm and ignore its' traffic. another reason is filtering out xml files from the analysis. this is done by default. however, the update site is mostly jars and xmls, so that's really missing its' purpose.

to change the settings, open your awstats configuration file (there's a separate file for each site you analyze) and make the following modifications:

• look for a setting called levelforrobotsdetection, the default is 2, change it to 0. this will ensure the requests from the java clients will not be detected as robots.
• look for a setting called notpagelist. you will find the list of ignored file types. remove xml from that list. this will log the access to files like site.xml and content.xml

that's it. if you already executed the analysis before the configuration changes, simply delete your data files and awstats will reanalyze your log files.

getting the most from the awstats report

the basic report of awstats is useful enough. it is in the monthly level, but you can break it down to the daily or hourly level using the databasebreak=hour or databasebreak=day option. for example, if you are running awstats directly from your browser (as opposed to generating the reports from the command line) you will need to add the following to your url: &databasebreak=day&day=23 (where 23 is the day number, no way to select it other than write it in the url). the report is the same as the monthly report, except that the data shown belongs to a specific day. this option is still "experimental", but it seems to work just fine. you can read more about it in the awstats faq .

i found two sub-reports particularly useful:

the "unknown os" report will show the java versions being used by your users. unfortunately, this is not a summary report, just a report of the last x visits.

the full list of urls will show all the jars that were downloaded.

to conclude, i should point out that i'm not an expert in web analytics. i just needed a simple solution and awstats seems to do the job. it is not perfect, but it is ok. i will be happy to learn about other tools you might be using for that purpose. i will continue to update on my awstats experience and any other solutions i might encounter for that purpose.

from http://blog.zvikico.com/