During the last week my team dedicated almost all its time to performance optimization. During the experience we measures metric with a series of tools from a little toolbox, and I'm going to share here which tools we chose and how we configured them to find out bottlenecks in our applications.
Xdebug can only target your development machines and not integration, staging or production ones. That said, it provides the highest level of detail in performance measurement: it goes from measuring the cumulative time spent inside each function and the number of calls, to a full trace of PHP processes.
Speaking of Xdebug profilers, we used Kcachegrind under Ubuntu for our analysis, as web-based alternatives like Webgrind just die when dealing with cachegrind files generated by Xdebug that go over hundreds of megabytes.
Xdebug is a PHP extension for your local machine and so its installation passes from the pecl command. Remember even just activating it reduces performance, so it should be relegated to a supporting role while running full test suites or load tests.
Symfony Stopwatch is one of the Symfony components, decoupled libraries that can be installed via Composer and as such pulling automatically their (few) dependencies.
The Stopwatch can measure elapsed time and memory peaks of the code that it decorates (here's an article on its usage.) Given the little overhead, unless you are using it in tight for loops you can insert it into production systems; even the overhead of writing the code is not much as you just need to instantiate an object and call its methods during the process (that may be one of the few times where to allow the global availability of an object.)
MySQL query logs
LAMP is not a given nowadays, but PHP still integrates neatly with MySQL, with its native driver and three different APIs exposed to the developer. So tuning MySQL is still important (even because it's one of the databases that needs the most tuning), and this operation always starts with finding out the problematic queries. SQL is not the most efficient or understandable language ever, but has the property of making it easy to log interactions with the database in plain text.
The my.cnf configuration file allows you to set several directives:
- general_log_file allows to log all queries made to the database. As stated in the configuration file's comments, this is a performance killer and should not be activated outside of development machines. Even there, you should only use it to find out interactions with the database of suspicious code that was supposed not to make queries at all; the noise makes it difficult.
- log_slow_queries and long_query_time repeat the previous process, but only for queries surpassing a minimum amount of time. Their overhead is smaller given the number of lines that have to br written to the log.
- log-queries-not-using-indexes provides you all the queries that go in full scan over tables. However it may provide a large set of queries as small configuration tables containing just tens of rows are not indexed; and it does not log queries that are using the wrong index and that are peforming poorly that they could.
All files are opened at the startup of mysqld, so you need to restart the daemon after changing the configuration and they can't just be cleared up while it is still running (unlike for example the PHP log).
Tools are not only open source software, but also external services. New Relic has a PHP extension that you can install directly in production to get:
- monitoring of the response times for all PHP processes.
- Division of time measurements according to single DB queries.
- Timing of external HTTP requests, divided by host.
It even offers a set of functions where you can export data for tagging processes, for example with the URL loaded by the client.
New Relic measures response times and throughput, helping you to find out when you have hit a plateau in optimization. The trial version is of course free but you may need the pro for the most useful measurements such as slow db queries.