So you're running the benchmark/stress test - how do you tell if MySQL server is really loaded ? This looks like the trivial question but in fact, especially when workload consists of simple queries I see the load generation and network really putting a lot less load on MySQL than expected. For example you may have 32 threads (or processes) running queries as fast as they can... does it really mean there is an 32 concurrent queries ran all the time ? It may be the case or it may be not...

Take a look at this server for example:

[root@db01 ~]# mysqladmin -i1  extended | grep Threads_running
| Threads_running					   | 1
| Threads_running					   | 1
| Threads_running					   | 1
| Threads_running					   | 19				   
| Threads_running					   | 1
| Threads_running					   | 1
| Threads_running					   | 20				   
| Threads_running					   | 1
| Threads_running					   | 1
| Threads_running					   | 1
| Threads_running					   | 11				   
| Threads_running					   | 1

 This corresponds to what is expected to be stress load but we can see MySQL is getting only spikes of concurrent query executions and most commonly there are no queries executing. Value 1 for Threads_running corresponds to the connection which runs "SHOW STATUS" so you need to subscribe 1 from the reported amount to see the true number. No wonder in the case above there were a lot of free CPU and IO capacity.

Take a look at another sample:

| Threads_running					   | 33				   
| Threads_running					   | 27				   
| Threads_running					   | 1
| Threads_running					   | 28				   
| Threads_running					   | 19				   
| Threads_running					   | 21				   
| Threads_running					   | 2
| Threads_running					   | 27				   
| Threads_running					   | 27				   
| Threads_running					   | 32				   
| Threads_running					   | 27				   
| Threads_running					   | 1
| Threads_running					   | 24				   
| Threads_running					   | 29

In this case the load is higher and a lot more uniform - there are cases when actually 32 queries are active (this is test with 32 connections) - but you can see most of the time it is less than that.

Looking at Threads_running is a very simple and powerful tool to see whenever you're really putting sustained load on the database you may be expecting.

It may be worth to explain what value of Threads_running represents. This is amount of queries which are being currently processing - the ball is on Server side. The server has gotten the query but has not completed sending response back yet. This is a very broad measure of activity - if query is waiting on IO, blocked on Mutex, table lock, row level lock, waiting on innodb_thread_concurrency it will be still considered running. This will be even the case when result of large query is being sent back and send operation is blocked because of slow network or the client. Because the measure is so broad it is very helpful to see if client is loading the server well - if it does the number of threads_running will be appropriately high.