Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Docker for Mac: Performance Tweaks

DZone's Guide to

Docker for Mac: Performance Tweaks

Docker for Mac sometimes requires tweaks to be fully usable in terms of performance. Learn how to tweak certain issues concerning Docker for Mac.

Free Resource

Download our Introduction to API Performance Testing and learn why testing your API is just as important as testing your website, and how to start today.

Are you a Linux user who switched to Mac when you saw that Docker is now available as a native Mac app? Have you heard how great Docker is and want to give it a try? Did you think that you could just take your Docker Compose file, launch your project, and have everything work out for you? Well… you were right. Almost.

Docker for Mac is a pretty smart invention. It gives you the whole Docker API available from the terminal, even though Docker itself wasn’t created to work on Macs. To make all this possible, a light Alpine Linux image is fired up underneath with xhyve MacOS native virtualization. Because of this, you need to allocate CPU cores and RAM for the VM. Things won’t be as close to bare metal as they are in Linux. If you are – for example – a Java developer who uses Docker to run compiled JAR, you may even not notice the difference. At least, as long as you don’t try to do any heavy database work.

Docker for Mac and Full Sync on Flush Issue

First, let’s look at MacOS fsync documentation:

“For applications that require tighter guarantees about the integrity of their data, Mac OS X provides the F_FULLFSYNC fcntl. The F_FULLFSYNC fcntl asks the drive to flush all buffered data to permanent storage. Applications, such as databases, that require a strict ordering of writes should use F_FULLFSYNC to ensure that their data is written in the order they expect.”

In short, to keep our data safe, every change made in the database needs to be stored on disk in an exact order. This will guarantee that during power loss or any unexpected event your data will be safe.

Actually, this makes sense — if you decide to setup a database inside Docker for Mac on a production environment…

In most cases, though, you’ll be using your machine for dev purposes where you don’t care to recreate the database from fixtures. If you have a Macbook, even power loss isn’t a threat. In this case, you may decide to disable this.

While reading about Docker issues on GitHub, I found a solution provided by djs55. Things will get a lot faster when you type those few lines into your terminal:

$ cd ~/Library/Containers/com.docker.docker/Data/database/
 $ git reset --hard
 HEAD is now at cafabd0 Docker started 1475137831
 $ cat com.docker.driver.amd64-linux/disk/full-sync-on-flush
 true
 $ echo false > com.docker.driver.amd64-linux/disk/full-sync-on-flush
 $ git add com.docker.driver.amd64-linux/disk/full-sync-on-flush
 $ git commit -s -m "Disable flushing"
 [master dc32fcc] Disable flushing
 1 file changed, 1 insertion(+), 1 deletion(-)

Actually, someone even placed bash script on gist to make things easier.

Does It Really Work?

I created a small test case to check this. This test uses a standard Docker MySQL image without tweaks, and an image with sysbench installed. In my test case, I decided to use one thread (I only allocated one core for Docker on my Macbook) and a table with 10,000 rows.

I ran it twice: once with flushing enabled (default), and once with flushing disabled. If you’re skeptical about performance gain after changing just one value from true to false, then let the results below change your mind. 

Command to run it:

$ docker-compose build
$ docker-compose up benchmark

With flush enabled:

mysql-test-bench | OLTP test statistics:
mysql-test-bench |     queries performed:
mysql-test-bench |         read:                            30730
mysql-test-bench |         write:                           8780
mysql-test-bench |         other:                           4390
mysql-test-bench |         total:                           43900
mysql-test-bench |     transactions:                        2195   (36.58 per sec.)
mysql-test-bench |     read/write requests:                 39510  (658.42 per sec.)
mysql-test-bench |     other operations:                    4390   (73.16 per sec.)
mysql-test-bench |     ignored errors:                      0      (0.00 per sec.)
mysql-test-bench |     reconnects:                          0      (0.00 per sec.)
mysql-test-bench |
mysql-test-bench | General statistics:
mysql-test-bench |     total time:                          60.0077s
mysql-test-bench |     total number of events:              2195
mysql-test-bench |     total time taken by event execution: 59.9995s
mysql-test-bench |     response time:
mysql-test-bench |          min:                                 21.26ms
mysql-test-bench |          avg:                                 27.33ms
mysql-test-bench |          max:                                 73.00ms
mysql-test-bench |          approx.  95 percentile:              35.62ms
mysql-test-bench |
mysql-test-bench | Threads fairness:
mysql-test-bench |     events (avg/stddev):           2195.0000/0.00
mysql-test-bench |     execution time (avg/stddev):   59.9995/0.00

With flush disabled:

mysql-test-bench | OLTP test statistics:
mysql-test-bench |     queries performed:
mysql-test-bench |         read:                            270074
mysql-test-bench |         write:                           77164
mysql-test-bench |         other:                           38582
mysql-test-bench |         total:                           385820
mysql-test-bench |     transactions:                        19291  (321.51 per sec.)
mysql-test-bench |     read/write requests:                 347238 (5787.13 per sec.)
mysql-test-bench |     other operations:                    38582  (643.01 per sec.)
mysql-test-bench |     ignored errors:                      0      (0.00 per sec.)
mysql-test-bench |     reconnects:                          0      (0.00 per sec.)
mysql-test-bench |
mysql-test-bench | General statistics:
mysql-test-bench |     total time:                          60.0018s
mysql-test-bench |     total number of events:              19291
mysql-test-bench |     total time taken by event execution: 59.9613s
mysql-test-bench |     response time:
mysql-test-bench |          min:                                  2.68ms
mysql-test-bench |          avg:                                  3.11ms
mysql-test-bench |          max:                                 20.70ms
mysql-test-bench |          approx.  95 percentile:               3.65ms
mysql-test-bench |
mysql-test-bench | Threads fairness:
mysql-test-bench |     events (avg/stddev):           19291.0000/0.00
mysql-test-bench |     execution time (avg/stddev):   59.9613/0.00

Looking at those numbers, we clearly see that with flushing disabled, we gained almost 10x performance! And this is with only 10k rows.

This means that if you don’t care that much about data loss, and you’re ready to sacrifice it if something goes wrong, then there is no reason to actually not change this setting.

Tip: From my observations, this tweak seems to be preserved after Docker for Mac updates, so there is no need to fire it over and over.

With one simple step, our Docker database performs approximately 10x faster, so now everything should be great, right? Well… not yet.

Docker for Mac and Mounted Volumes

Interpreted programming languages usually come with a lot of files, cache, bootstrap, etc. PHP with Symfony framework is a good example. Symfony, with almost no cache (i.e. dev environment), writes and reads a lot of files between Request and Response. Because the language is interpreted, a PHP developer can see code change results almost immediately. Therefore, mounting code as a volume inside a Docker container is a natural way of work. Linux performance with aufs is close to native in this case, but Mac osxfs, on the other hand, isn’t.

It’s so slow that you can even get composer timeouts on “composer install” or “composer update”. What’s more, Symfony requests to a “Hello World” page can take up to 30s. There is a large thread on GitHub about this issue as well.

To prove how bad things are, a few new tests need to be run.

We’ll just create a dummy file of around ~100MB.

First, let’s run a simple command directly on a Mac terminal to have a base to compare with:

$ time dd if=/dev/zero of=test.dat bs=1024 count=100000
100000+0 records in
100000+0 records out
real    0m0.291s
user    0m0.021s
sys    0m0.250s

Now, let’s try the same thing inside a Docker container.

Test:

$ docker-compose up
Starting docker-native
Attaching to docker-native
docker-native | 100000+0 records in
docker-native | 100000+0 records out
docker-native | real    0m 0.38s
docker-native | user    0m 0.00s
docker-native | sys    0m 0.21s

As you can see, running something in Docker doesn’t make things much slower.

But what if we want to mount our local directory in a Docker container?

For this, let’s use the standard way.

Test:

$ docker-compose up
Starting docker-mount
Attaching to docker-mount
docker-mount | 100000+0 records in
docker-mount | 100000+0 records out
docker-mount | real    0m 17.80s
docker-mount | user    0m 0.12s
docker-mount | sys    0m 0.66s

This time, the results are even more spectacular than in the database case. Creating the same file in a mounted volume is 45x slower. Now, think how this can actually affect composer, Symfony, database or any other app which requires hard disk writes!

Unfortunately, there is no built-in solution. You may find workarounds which will allow you to mount volumes with nfs instead of osxfs, but it still may be not enough for Symfony.

Hopefully, in this case, the internets can save the day again!

In my dev environment, I decided to use docker-sync by EugenMayer.

What does this tool do? It will allow you to create Docker volumes whose content will be synced with the host using unison. Then you can use this volume as a mounting point for your container. This speeds things up a lot. Your app is syncing data with the volume with almost native speed. Then, the volume is synced with a host in the background and doesn’t cause any slowdowns for the app.

Now it’s time for another test to show how fast this solution can really be.

Firstly, let’s create a docker-sync.yml file, which is required by docker-sync:

options:
  verbose: true
syncs:
  #IMPORTANT: ensure this name is unique and does not match your other container names
  docker-mac-sync:
    src: './'
    dest: '/test'
    sync_strategy: 'unison'
    sync_args:
      - "-prefer newer"
      - "-ignore='Path .git'"
      - "-ignore='BelowPath .git'"

In this file, we declare our volume docker-mac-sync, which then can be used in the docker-compose.yml file. I tried to keep it as simple as possible. One thing worth mentioning is the sync_args param, which just allows you to configure standard unison flags. It’s not necessary for our test, but in most cases, the “prefer newer” strategy will be used by devs.

Now, it’s time to use docker-sync in Docker compose:

version: "2"
services:
  docker-sync-test:
    image: alpine
    container_name: 'docker-sync-mount'
    volumes:
        - docker-mac-sync:/test:rw
    command: /bin/sh -c "cd test && time dd if=/dev/zero of=test.dat bs=1024 count=100000"
volumes:
    docker-mac-sync:
        external: true

In the volumes section, we declare docker-mac-sync as an external one and just use as a mount point for /test inside a Docker container.

Before we make our final test there is one more thing to mention. Because of its nature, docker-sync requires that files should be synced into the docker-sync created container before we run our main app. It may take some time if we have a large codebase, but after that, subsequent syncs will be fast (at least until you delete the sync volume).

The tool provides two commands to interact with it — docker-sync and docker-sync-stack. The second is just a shortcut to docker-sync start and docker-compose up. I tried both, but prefer to run docker-sync in a separate tab to see what’s actually happening in it.

To start syncing, I just need to type:

$ docker-sync start

Then, in next tab, the magic happens:

$ docker-compose up
Starting docker-sync-mount
Attaching to docker-sync-mount
docker-sync-mount | 100000+0 records in
docker-sync-mount | 100000+0 records out
docker-sync-mount | real    0m 0.40s
docker-sync-mount | user    0m 0.01s
docker-sync-mount | sys    0m 0.24s

Benchmark shows times similar to what we saw when we ran the test inside a Docker container without any mount. That’s a huge improvement, and it will allow you to work comfortably with your new Mac… almost.

Unfortunately, nothing is perfect.

The presented solution has a few drawbacks:

  • You need to remember to run docker-sync before running your app.
  • When you switch branches, in most cases it’ll be wise to stop the app, use docker-sync cleam and re-sync everything
  • Sometimes, docker-sync may freeze; it’s worth to check it from time to time and just restart it
  • Because it’s a Mac workaround, it may be wise to have separate docker-compose.mac.yml and docker-compose.linux.yml and run docker compose with an -f flag keeping base docker-compose.yml as clean as possible.

Still Reading?

Good! As a bonus, there is one more trick for those who use xdebug.

Because of its nature, docker-for-mac won’t allow you to simply put xdebug.remote_connect_back=1 in your xdebug.ini file; due to the nature of connection_back, it just won’t work. There are a few workarounds, but the one I use creates 10.254.254.254 as an alias on your loopback device (127.0.0.1). This one command in Terminal will do it:

$ ifconfig lo0 alias 10.254.254.254

You may also use a gist plist to have it fired automatically on each boot. After that, your xdebug config may look like this:

xdebug.remote_enable = 1
xdebug.remote_autostart = 0
xdebug.remote_connect_back = 0
xdebug.profiler_enable = 0
xdebug.remote_handler = dbgp
xdebug.remote_port = 9000
xdebug.remote_host = 10.254.254.254
xdebug.idekey = IDEKEY
xdebug.max_nesting_level = 500

Then, in PHPStorm, you may just set up DBG Proxy like this:

Image title

In my sample config, I use idekey and remote_autostart = 0 because I prefer to start the xdebug session from a Firefox addon called The easiest xdebug. Chrome has something similar, as well.

That’s All!

I hope that this article gave you an idea how to tweak certain issues concerning Docker for Mac. I just want to wish you(/me/us!) that someday this whole wall of text will be obsolete, and future versions of Docker for Mac won’t require hacking to be usable. By the way, all examples presented in this article can be downloaded from GitHub.

Find scaling and performance issues before your customers do with our Introduction to High-Capacity Load Testing guide.

Topics:
docker ,mac ,docker api ,performance ,tutorial

Published at DZone with permission of Bartosz Telesinski. See the original article here.

Opinions expressed by DZone contributors are their own.

THE DZONE NEWSLETTER

Dev Resources & Solutions Straight to Your Inbox

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

X

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}