It's important to understand that you don't simply install a new version and do a little dance, there are a few things that you need to be aware of, and for that process Nick Veenhof has some advice on his blog:
If you want to migrate to Solr 3.5 coming from Solr 1.4 with low risk of changes you should keep using the LogByteMergePolicy with a mergefactor of 4 (Default in the Drupal configs).
However, the TieredMergePolicy is interesting when understood correctly. I'd love some more comments on that topic from people that know more about it. --Nick Veenhof
For Nick, the migration was a learning experience from which he also realized the massive performance benefits that Solr 3.5 brought.
Check out his blog post for the steps he took during migration and the graphical data he put together to show off his performance gains.
Also check out these docs from Apache for some background: