Docker for Mac: Performance Tweaks
Docker for Mac: Performance Tweaks
Docker for Mac sometimes requires tweaks to be fully usable in terms of performance. Learn how to tweak certain issues concerning Docker for Mac.
Join the DZone community and get the full member experience.Join For Free
Container Monitoring and Management eBook: Read about the new realities of containerization.
Are you a Linux user who switched to Mac when you saw that Docker is now available as a native Mac app? Have you heard how great Docker is and want to give it a try? Did you think that you could just take your Docker Compose file, launch your project, and have everything work out for you? Well… you were right. Almost.
Docker for Mac is a pretty smart invention. It gives you the whole Docker API available from the terminal, even though Docker itself wasn’t created to work on Macs. To make all this possible, a light Alpine Linux image is fired up underneath with xhyve MacOS native virtualization. Because of this, you need to allocate CPU cores and RAM for the VM. Things won’t be as close to bare metal as they are in Linux. If you are – for example – a Java developer who uses Docker to run compiled JAR, you may even not notice the difference. At least, as long as you don’t try to do any heavy database work.
Docker for Mac and Full Sync on Flush Issue
First, let’s look at MacOS fsync documentation:
“For applications that require tighter guarantees about the integrity of their data, Mac OS X provides the F_FULLFSYNC fcntl. The F_FULLFSYNC fcntl asks the drive to flush all buffered data to permanent storage. Applications, such as databases, that require a strict ordering of writes should use F_FULLFSYNC to ensure that their data is written in the order they expect.”
In short, to keep our data safe, every change made in the database needs to be stored on disk in an exact order. This will guarantee that during power loss or any unexpected event your data will be safe.
Actually, this makes sense — if you decide to setup a database inside Docker for Mac on a production environment…
In most cases, though, you’ll be using your machine for dev purposes where you don’t care to recreate the database from fixtures. If you have a Macbook, even power loss isn’t a threat. In this case, you may decide to disable this.
While reading about Docker issues on GitHub, I found a solution provided by djs55. Things will get a lot faster when you type those few lines into your terminal:
$ cd ~/Library/Containers/com.docker.docker/Data/database/ $ git reset --hard HEAD is now at cafabd0 Docker started 1475137831 $ cat com.docker.driver.amd64-linux/disk/full-sync-on-flush true $ echo false > com.docker.driver.amd64-linux/disk/full-sync-on-flush $ git add com.docker.driver.amd64-linux/disk/full-sync-on-flush $ git commit -s -m "Disable flushing" [master dc32fcc] Disable flushing 1 file changed, 1 insertion(+), 1 deletion(-)
Actually, someone even placed bash script on gist to make things easier.
Does It Really Work?
I created a small test case to check this. This test uses a standard Docker MySQL image without tweaks, and an image with sysbench installed. In my test case, I decided to use one thread (I only allocated one core for Docker on my Macbook) and a table with 10,000 rows.
I ran it twice: once with flushing enabled (default), and once with flushing disabled. If you’re skeptical about performance gain after changing just one value from true to false, then let the results below change your mind.
Command to run it:
$ docker-compose build $ docker-compose up benchmark
With flush enabled:
mysql-test-bench | OLTP test statistics: mysql-test-bench | queries performed: mysql-test-bench | read: 30730 mysql-test-bench | write: 8780 mysql-test-bench | other: 4390 mysql-test-bench | total: 43900 mysql-test-bench | transactions: 2195 (36.58 per sec.) mysql-test-bench | read/write requests: 39510 (658.42 per sec.) mysql-test-bench | other operations: 4390 (73.16 per sec.) mysql-test-bench | ignored errors: 0 (0.00 per sec.) mysql-test-bench | reconnects: 0 (0.00 per sec.) mysql-test-bench | mysql-test-bench | General statistics: mysql-test-bench | total time: 60.0077s mysql-test-bench | total number of events: 2195 mysql-test-bench | total time taken by event execution: 59.9995s mysql-test-bench | response time: mysql-test-bench | min: 21.26ms mysql-test-bench | avg: 27.33ms mysql-test-bench | max: 73.00ms mysql-test-bench | approx. 95 percentile: 35.62ms mysql-test-bench | mysql-test-bench | Threads fairness: mysql-test-bench | events (avg/stddev): 2195.0000/0.00 mysql-test-bench | execution time (avg/stddev): 59.9995/0.00
With flush disabled:
mysql-test-bench | OLTP test statistics:
mysql-test-bench | queries performed:
mysql-test-bench | read: 270074
mysql-test-bench | write: 77164
mysql-test-bench | other: 38582
mysql-test-bench | total: 385820
mysql-test-bench | transactions: 19291 (321.51 per sec.)
mysql-test-bench | read/write requests: 347238 (5787.13 per sec.)
mysql-test-bench | other operations: 38582 (643.01 per sec.)
mysql-test-bench | ignored errors: 0 (0.00 per sec.)
mysql-test-bench | reconnects: 0 (0.00 per sec.)
mysql-test-bench | General statistics:
mysql-test-bench | total time: 60.0018s
mysql-test-bench | total number of events: 19291
mysql-test-bench | total time taken by event execution: 59.9613s
mysql-test-bench | response time:
mysql-test-bench | min: 2.68ms
mysql-test-bench | avg: 3.11ms
mysql-test-bench | max: 20.70ms
mysql-test-bench | approx. 95 percentile: 3.65ms
mysql-test-bench | Threads fairness:
mysql-test-bench | events (avg/stddev): 19291.0000/0.00
mysql-test-bench | execution time (avg/stddev): 59.9613/0.00
mysql-test-bench | OLTP test statistics: mysql-test-bench | queries performed: mysql-test-bench | read: 270074 mysql-test-bench | write: 77164 mysql-test-bench | other: 38582 mysql-test-bench | total: 385820 mysql-test-bench | transactions: 19291 (321.51 per sec.) mysql-test-bench | read/write requests: 347238 (5787.13 per sec.) mysql-test-bench | other operations: 38582 (643.01 per sec.) mysql-test-bench | ignored errors: 0 (0.00 per sec.) mysql-test-bench | reconnects: 0 (0.00 per sec.) mysql-test-bench | mysql-test-bench | General statistics: mysql-test-bench | total time: 60.0018s mysql-test-bench | total number of events: 19291 mysql-test-bench | total time taken by event execution: 59.9613s mysql-test-bench | response time: mysql-test-bench | min: 2.68ms mysql-test-bench | avg: 3.11ms mysql-test-bench | max: 20.70ms mysql-test-bench | approx. 95 percentile: 3.65ms mysql-test-bench | mysql-test-bench | Threads fairness: mysql-test-bench | events (avg/stddev): 19291.0000/0.00 mysql-test-bench | execution time (avg/stddev): 59.9613/0.00
Looking at those numbers, we clearly see that with flushing disabled, we gained almost 10x performance! And this is with only 10k rows.
This means that if you don’t care that much about data loss, and you’re ready to sacrifice it if something goes wrong, then there is no reason to actually not change this setting.
Tip: From my observations, this tweak seems to be preserved after Docker for Mac updates, so there is no need to fire it over and over.
With one simple step, our Docker database performs approximately 10x faster, so now everything should be great, right? Well… not yet.
Docker for Mac and Mounted Volumes
Interpreted programming languages usually come with a lot of files, cache, bootstrap, etc. PHP with Symfony framework is a good example. Symfony, with almost no cache (i.e. dev environment), writes and reads a lot of files between Request and Response. Because the language is interpreted, a PHP developer can see code change results almost immediately. Therefore, mounting code as a volume inside a Docker container is a natural way of work. Linux performance with aufs is close to native in this case, but Mac osxfs, on the other hand, isn’t.
It’s so slow that you can even get composer timeouts on “composer install” or “composer update”. What’s more, Symfony requests to a “Hello World” page can take up to 30s. There is a large thread on GitHub about this issue as well.
To prove how bad things are, a few new tests need to be run.
We’ll just create a dummy file of around ~100MB.
First, let’s run a simple command directly on a Mac terminal to have a base to compare with:
$ time dd if=/dev/zero of=test.dat bs=1024 count=100000 100000+0 records in 100000+0 records out real 0m0.291s user 0m0.021s sys 0m0.250s
Now, let’s try the same thing inside a Docker container.
$ docker-compose up Starting docker-native Attaching to docker-native docker-native | 100000+0 records in docker-native | 100000+0 records out docker-native | real 0m 0.38s docker-native | user 0m 0.00s docker-native | sys 0m 0.21s
As you can see, running something in Docker doesn’t make things much slower.
But what if we want to mount our local directory in a Docker container?
For this, let’s use the standard way.
$ docker-compose up Starting docker-mount Attaching to docker-mount docker-mount | 100000+0 records in docker-mount | 100000+0 records out docker-mount | real 0m 17.80s docker-mount | user 0m 0.12s docker-mount | sys 0m 0.66s
This time, the results are even more spectacular than in the database case. Creating the same file in a mounted volume is 45x slower. Now, think how this can actually affect composer, Symfony, database or any other app which requires hard disk writes!
Unfortunately, there is no built-in solution. You may find workarounds which will allow you to mount volumes with nfs instead of osxfs, but it still may be not enough for Symfony.
Hopefully, in this case, the internets can save the day again!
In my dev environment, I decided to use docker-sync by EugenMayer.
What does this tool do? It will allow you to create Docker volumes whose content will be synced with the host using unison. Then you can use this volume as a mounting point for your container. This speeds things up a lot. Your app is syncing data with the volume with almost native speed. Then, the volume is synced with a host in the background and doesn’t cause any slowdowns for the app.
Now it’s time for another test to show how fast this solution can really be.
Firstly, let’s create a
docker-sync.yml file, which is required by
options: verbose: true syncs: #IMPORTANT: ensure this name is unique and does not match your other container names docker-mac-sync: src: './' dest: '/test' sync_strategy: 'unison' sync_args: - "-prefer newer" - "-ignore='Path .git'" - "-ignore='BelowPath .git'"
In this file, we declare our volume
docker-mac-sync, which then can be used in the
docker-compose.yml file. I tried to keep it as simple as possible. One thing worth mentioning is the
sync_args param, which just allows you to configure standard unison flags. It’s not necessary for our test, but in most cases, the “prefer newer” strategy will be used by devs.
Now, it’s time to use
docker-sync in Docker compose:
version: "2" services: docker-sync-test: image: alpine container_name: 'docker-sync-mount' volumes: - docker-mac-sync:/test:rw command: /bin/sh -c "cd test && time dd if=/dev/zero of=test.dat bs=1024 count=100000" volumes: docker-mac-sync: external: true
volumes section, we declare
docker-mac-sync as an external one and just use as a mount point for
/test inside a Docker container.
Before we make our final test there is one more thing to mention. Because of its nature, docker-sync requires that files should be synced into the docker-sync created container before we run our main app. It may take some time if we have a large codebase, but after that, subsequent syncs will be fast (at least until you delete the sync volume).
The tool provides two commands to interact with it —
docker-sync-stack. The second is just a shortcut to
docker-sync start and
docker-compose up. I tried both, but prefer to run
docker-sync in a separate tab to see what’s actually happening in it.
To start syncing, I just need to type:
$ docker-sync start
Then, in next tab, the magic happens:
$ docker-compose up Starting docker-sync-mount Attaching to docker-sync-mount docker-sync-mount | 100000+0 records in docker-sync-mount | 100000+0 records out docker-sync-mount | real 0m 0.40s docker-sync-mount | user 0m 0.01s docker-sync-mount | sys 0m 0.24s
Benchmark shows times similar to what we saw when we ran the test inside a Docker container without any mount. That’s a huge improvement, and it will allow you to work comfortably with your new Mac… almost.
Unfortunately, nothing is perfect.
The presented solution has a few drawbacks:
- You need to remember to run
docker-syncbefore running your app.
- When you switch branches, in most cases it’ll be wise to stop the app, use
docker-sync cleamand re-sync everything
docker-syncmay freeze; it’s worth to check it from time to time and just restart it
- Because it’s a Mac workaround, it may be wise to have separate
docker-compose.linux.ymland run docker compose with an
-fflag keeping base
docker-compose.ymlas clean as possible.
Good! As a bonus, there is one more trick for those who use xdebug.
Because of its nature, docker-for-mac won’t allow you to simply put
xdebug.remote_connect_back=1 in your
xdebug.ini file; due to the nature of
connection_back, it just won’t work. There are a few workarounds, but the one I use creates
10.254.254.254 as an alias on your loopback device (
127.0.0.1). This one command in Terminal will do it:
$ ifconfig lo0 alias 10.254.254.254
You may also use a gist plist to have it fired automatically on each boot. After that, your xdebug config may look like this:
xdebug.remote_enable = 1 xdebug.remote_autostart = 0 xdebug.remote_connect_back = 0 xdebug.profiler_enable = 0 xdebug.remote_handler = dbgp xdebug.remote_port = 9000 xdebug.remote_host = 10.254.254.254 xdebug.idekey = IDEKEY xdebug.max_nesting_level = 500
Then, in PHPStorm, you may just set up DBG Proxy like this:
In my sample config, I use
remote_autostart = 0 because I prefer to start the xdebug session from a Firefox addon called The easiest xdebug. Chrome has something similar, as well.
I hope that this article gave you an idea how to tweak certain issues concerning Docker for Mac. I just want to wish you(/me/us!) that someday this whole wall of text will be obsolete, and future versions of Docker for Mac won’t require hacking to be usable. By the way, all examples presented in this article can be downloaded from GitHub.
Published at DZone with permission of Bartosz Telesinski , DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.