Pifpaf, a Tool to Run Any Daemon Briefly

DZone 's Guide to

Pifpaf, a Tool to Run Any Daemon Briefly

Is it working? Pifpaf will help you spin up a daemon and end it for your software testing.

· Integration Zone ·
Free Resource

There's a lot of situations where you end up needing a software deployed temporarily. This can happen when testing something manually, when running a script, or when launching a test suite.

Indeed, many applications need to use and interconnect with external software: an RDBMS (PostgreSQL, MySQL), a cache (memcached, Redis) or any other external component. This tends to make it more difficult to run a piece of software (or its test suite). If you want to rely on this component being installed and deployed, you end up needing a full environment set-up, properly configured to run your tests, which is discouraging.

The different OpenStack projects I work on ended up pretty quickly spawning some of their back-ends temporarily to run their tests. Some of those unit tests somehow became entirely what you would call functional or integration tests. But, that's just a name. In the end, what we ended up doing was testing to see that the software was really working. And, there's no better way of doing that than talking to a real PostgreSQL instance rather than mocking every call.

To solve that issue, I created a new tool, named Pifpaf. Pifpaf eases the run of any daemon in a test mode for a brief moment, before making it disappear completely. It's pretty easy to install as it is available on PyPI:

$ pip install pifpaf
Collecting pifpaf
Installing collected packages: pifpaf
Successfully installed pifpaf-0.0.7

You can then use it to run any of the listed daemons:

$ pifpaf list
| Daemons |
| redis |
| postgresql |
| mongodb |
| zookeeper |
| aodh |
| influxdb |
| ceph |
| elasticsearch |
| etcd |
| mysql |
| memcached |
| rabbitmq |
| gnocchi |

Pifpaf accepts any shell command line to execute after its arguments:

$ pifpaf run postgresql -- psql
Expanded display is used automatically.
Line style is unicode.
psql (9.5.2)
Type "help" for help.

template1=# \l
 List of databases
 Name │ Owner │ Encoding │ Collate │ Ctype │ Access privileges
 postgres │ jd │ UTF8 │ en_US.UTF-8 │ en_US.UTF-8 │
 template0 │ jd │ UTF8 │ en_US.UTF-8 │ en_US.UTF-8 │ =c/jd ↵
 │ │ │ │ │ jd=CTc/jd
 template1 │ jd │ UTF8 │ en_US.UTF-8 │ en_US.UTF-8 │ =c/jd ↵
 │ │ │ │ │ jd=CTc/jd
(3 rows)

template1=# create database foobar;
template1=# \l
 List of databases
 Name │ Owner │ Encoding │ Collate │ Ctype │ Access privileges
 foobar │ jd │ UTF8 │ en_US.UTF-8 │ en_US.UTF-8 │
 postgres │ jd │ UTF8 │ en_US.UTF-8 │ en_US.UTF-8 │
 template0 │ jd │ UTF8 │ en_US.UTF-8 │ en_US.UTF-8 │ =c/jd ↵
 │ │ │ │ │ jd=CTc/jd
 template1 │ jd │ UTF8 │ en_US.UTF-8 │ en_US.UTF-8 │ =c/jd ↵
 │ │ │ │ │ jd=CTc/jd
(4 rows)

template1=# \q

What Pifpaf does is that it runs the different commands needed to create a new PostgreSQL cluster and then runs PostgreSQL on a temporary port for you. So, your PSQL session actually connects to a temporary PostgreSQL server, which is then trashed as soon as you quit PSQL. And, all of that in less than 10 seconds, without the use of any virtualization or container technology!

You can see what it does in detail using the debug mode:

$ pifpaf --debug run mysql $SHELL
DEBUG: pifpaf.drivers: executing: ['mysqld', '--initialize-insecure', '--datadir=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmpkut9bg']
DEBUG: pifpaf.drivers: executing: ['mysqld', '--datadir=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmpkut9bg', '--pid-file=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmpkut9bg/mysql.pid', '--socket=/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmpkut9bg/mysql.socket', '--skip-networking', '--skip-grant-tables']
DEBUG: pifpaf.drivers: executing: ['mysql', '--no-defaults', '-S', '/var/folders/7k/pwdhb_mj2cv4zyr0kyrlzjx40000gq/T/tmpkut9bg/mysql.socket', '-e', 'CREATE DATABASE test;']
$ exit
DEBUG: pifpaf.drivers: mysqld output: 2016-04-08T08:52:04.202143Z 0 [Note] InnoDB: Starting shutdown...

Pifpaf also supports my pet project Gnocchi, so you can run and try that time series database in a snap:

$ pifpaf run gnocchi $SHELL
$ gnocchi metric create
| Field | Value |
| archive_policy/aggregation_methods | std, count, 95pct, min, max, sum, median, mean |
| archive_policy/back_window | 0 |
| archive_policy/definition | - points: 12, granularity: 0:05:00, timespan: 1:00:00 |
| | - points: 24, granularity: 1:00:00, timespan: 1 day, 0:00:00 |
| | - points: 30, granularity: 1 day, 0:00:00, timespan: 30 days, 0:00:00 |
| archive_policy/name | low |
| created_by_project_id | admin |
| created_by_user_id | admin |
| id | ff825d33-c8c8-46d4-b696-4b1e8f84a871 |
| name | None |
| resource/id | None |
$ exit

And, it takes less than 10 seconds to launch Gnocchi on my laptop using Pifpaf. I'm then able to play with the gnocchi command line tool. It's by far faster than using OpenStack devstack to deploy everything.

We leverage Pifpaf in several of our OpenStack telemetry related projects now, and even in tooz. For example, to run unit/functional tests with a memcached server available, a tox.ini file should like this:

commands = pifpaf run memcached -- python setup.py testr

The tests can then use the environment variable PIFPAF_MEMCACHED_PORT to connect to memcached and run tests using it. As soon as the tests are finished, memcached is killed by Pifpaf and the temporary data is trashed.

We have moved a few OpenStack projects to using Pifpaf already, and I'm planning to make use of it in a few more. My fellow developer Mehdi Abaakouk added support for RabbitMQ in Pifpaf and added support for more advanced tests in oslo.messaging (such as failure scenarios) using Pifpaf.

Pifpaf is a very small and handy tool. Give it a try and let me know how it works for you!


Published at DZone with permission of Julien Danjou . See the original article here.

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}