The Pimpl Pattern: What You Should Know

In this article, I'd like to gather all the essential information regarding this dependency breaking technique. We'll discuss the implementation (const issue, back pointer, fast impl), pros and cons, alternatives and also show examples of where is it used. You'll also see how modern C++ can change this pattern. Moreover, I hope you'll help me and provide your examples.

The article reposted from www.bfilipek.com

Intro

A lot has been written about the pimpl pattern. Starting with some old posts by Herb Sutter: GotW #24: Compilation Firewalls and GotW #7b Solution: Minimizing Compile-Time Dependencies.
To some recent ones: GotW #100: Compilation Firewalls and GotW #101: Compilation Firewalls, Part 2, and even a few months ago from Fluent C++ How to implement the pimpl idiom by using unique_ptr. Plus ,of course, tons of other great articles...

So, why would I like to write about pimpl?

First of all, I'd like to make a summary of the essential facts. The pattern is used to break dependencies - both physical and logical - of the code. The basics sound simple, but as usual, there's more to the story.

There's also an important question: should we all use pimpl today? Are there better alternatives?

Let's start with a simple example to set the background:

The Basics

Pimpl might appear with different names: d-pointer, compiler firewall, or even the Cheshire Cat pattern or Opaque pointer.

In its basic form the pattern looks as follows:

• In a class, we move all private members to a newly declared type - like a PrivateImpl class.
• It's only forward declared in the header file of the main class
• In the corresponding CPP file, we declare the PrivateImpl class and define it.
• Now, if you change the private implementation, the client code won't have to be recompiled (as the interface hasn't changed).

So it might look like this (crude, old-style code!):

// class.h
class MyClassImpl;
class MyClass {
    // ...
    void Foo();
private:    
    MyClassImpl* m_pImpl; // warning!!! 
                          // a raw pointer! :)
};

// class.cpp
class MyClassImpl
{
public:
    void DoStuff() { /*...*/ }
};

MyClass::MyClass () 
: m_pImpl(new MyClassImpl()) 
{ }

MyClass::~MyClass () { delete m_pImpl; }

void MyClass ::DoSth() {
    m_pImpl->DoSth();
}

Ech... ugly raw pointers!

So briefly: we pack everything that is private into that forward declared class. We use just one member of our main class - the compiler can work with only the pointer without having a full type declaration - as only the size of the pointer is needed. Then the whole private declaration and implementation happen in the .cpp file.

Of course, in modern C++, it's also advised to use unique_ptr rather than raw pointers.

The two obvious downsides of this approach: we need a separate memory allocation to store the private section, and the main class just forwards the method calls to the private implementation.

Ok... but that's all... right? Not so fast!

The above code might work, but we have to add a few bits to make it work in real life.

More Code

We have to ask a few questions before we can write the full code:

• Is your class copyable or only movable?
• How to enforce const for methods in that private implementation?
• Do you need a "backward" pointer - so that the impl class can call/reference members of the main class?
• What should be put in that private implementation? Everything that's private?

The first part - copyable/movable relates to the fact that with the simple - raw - pointer we can only shallow copy an object. Of course, this happens in every case where you have a pointer in your class.

So, for sure, we have to implement a copy constructor (or delete it if we want only movable type).

What about that const problem? Can you catch it in the basic example?

If you declare a const method, then you cannot change members of the object. In other words, they become const. But it's a problem for our m_pImpl, which is a pointer. In a const method, this pointer will also become const, which means we cannot assign a different value to it, but, we can happily call all methods of this underlying private class (not only constant!).

So what we need is a conversion/wrapper mechanism. Something like this:

const MyClassImpl* Pimpl() const { return m_pImpl; }
MyClassImpl* Pimpl() { return m_pImpl; }

And now, in all of our methods of the main class, we should be using that function wrapper, not the pointer itself.

So far, I didn't mention that "backward" pointer ("q-pointer" in QT terminology). The answer is connected to the last point - what should we put in the private implementation - only private fields? Or maybe even private functions?

The basic code won't show those practical problems. But in a real application, a class might contain a lot of methods and fields. I've seen examples where all of the private sections (with methods) go to the pimpl class. Still, sometimes, the pimpl class needs to call a 'real' method of the main class, so we need to provide that "back" pointer. This can be done at construction - just pass the pointer to this.

The Improved Version

So here's an improved version of our example code:

// class.h
class MyClassImpl;
class MyClass
{
public:
    explicit MyClass();
    ~MyClass(); 

    // movable:
    MyClass(MyClass && rhs) noexcept;   
    MyClass& operator=(MyClass && rhs) noexcept;

    // and copyable
    MyClass(const MyClass& rhs);
    MyClass& operator=(const MyClass& rhs);

    void DoSth();
    void DoConst() const;

private:
    const MyClassImpl* Pimpl() const { return m_pImpl.get(); }
    MyClassImpl* Pimpl() { return m_pImpl.get(); }

    std::unique_ptr<MyClassImpl> m_pImpl;
};

// class.cpp
class MyClassImpl
{
public:
    ~MyClassImpl() = default;

    void DoSth() { }
    void DoConst() const { }
};

MyClass::MyClass() : m_pImpl(new MyClassImpl()) 
{

}

MyClass::~MyClass() = default;
MyClass::MyClass(MyClass &&) noexcept = default;
MyClass& MyClass::operator=(MyClass &&) noexcept = default;

MyClass::MyClass(const MyClass& rhs)
    : m_pImpl(new MyClassImpl(*rhs.m_pImpl))
{}

MyClass& MyClass::operator=(const MyClass& rhs) {
    if (this != &rhs) 
        m_pImpl.reset(new MyClassImpl(*rhs.m_pImpl));

    return *this;
}

void MyClass::DoSth()
{
    Pimpl()->DoSth();
}

void MyClass::DoConst() const
{
    Pimpl()->DoConst();
}

A bit better now.

The above code uses:

• unique_ptr - but see that the destructor for the main class must be defined in the CPP file. Otherwise, the compiler will complain about missing a deleter type.
• The class is movable and copyable, so four methods were defined.
• To be safe with const methods, all of the proxy methods of the main class use the Pimpl() method to fetch the proper type of pointer.

Have a look at this blog post, Pimp My Pimpl - Reloaded, by Marc Mutz for a lot of information about pimpl.

As a Separate Class

For an example of how to use a separate class, Herb Sutter, in GotW #101: Compilation Firewalls, Part 2, suggests the following wrapper:

// taken from Herb Sutter
template<typename T>
class pimpl {
private:
    std::unique_ptr<T> m;
public:
    pimpl();
    template<typename ...Args> pimpl( Args&& ... );
    ~pimpl();
    T* operator->();
    T& operator*();
};

Still, you're left with the implementation of copy construction, if required.

If you want a full blown wrapper, take a look at this post, PIMPL, Rule of Zero and Scott Meyers, by Andrey Upadyshev.

In that article you can see a very advanced implementation of such a helper type:

SPIMPL (Smart Pointer to IMPLementation) - a small header-only C++11 library with the aim to simplify the implementation of the PIMPL idiom
https://github.com/oliora/samples/blob/master/spimpl.h

Inside the library, you can find two types: spimpl::unique_impl_ptr - for movable only pimpl, and spimpl::impl_ptr for movable and copyable pimpl wrapper.

Fast pimpl

One obvious point about impl is that a memory allocation is needed to store private parts of the class. If you'd like to avoid this, and you really care about that memory allocation, you can try to:

• Provide a custom allocator and use some fixed memory chunk for the private implementation.
• Reserve a large block of memory in the main class and use placement new to allocate the space for pimpl.

Herb Sutter wrote about this idea here: GotW #28: The Fast Pimpl Idiom.

A modern version - which uses a C++11 feature - aligned_storage is described here:
My Favourite C++ Idiom: Static PIMPL / Fast PIMPL by Kai Dietrich or Type-safe Pimpl implementation without overhead | Probably Dance blog.

But be aware that it's only a trick, so it might not work. Or it might work on one platform/compiler, but not on other configurations.

In my personal opinion, I don't see this approach as a good one. Pimp is usually used for larger classes (maybe managers, types in the interfaces of a module), so that additional cost won't make much.

We've seen a few core parts of the pimpl pattern, so we can now discuss its strengths and weaknesses.

Pros and Cons

Pros

• Provides Compilation Firewall: if the private implementation changes the client code, it doesn't have to be recompiled.
  • Headers can become smaller, as types mentioned only in a class implementation need no longer be defined for the client code.
  • So, all in all, it might lead to better compilation times.
• Provides Binary Compatibility: very important for library developers. As long as the binary interface stays the same, you can link your app to a different version of a library.
  • To simplify, if you add a new virtual method, then the ABI changes, but adding non-virtual methods (of course without removing existing ones) doesn't change the ABI.
  • See Fragile Binary Interface Problem.
• Possible advantage: No v-table (if the main class contains only non-virtual methods).
• Small point: Can be used as an object in a stack.

Cons

• Performance - one level of indirection is added.
• A memory chunk has to be allocated (or preallocated) for the private implementation.
• Complex code and it requires some discipline to maintain such classes.
• Debugging - you don't see the details immediately, and the class is split.

Other Issues

• Testability - there's an opinion that when you try to test such a pimpl class, it might cause problems. But as usual, if you test only the public interface it shouldn't matter.
• Not for every class. This pattern is often best for large classes at the "interface level." I don't think vector3d with that pattern would be a good idea, for example.

Alternatives

• Redesign the code.
• To improve build times:
  • Use precompiled headers:
  • Use build caches.
  • Use an incremental build mode.
• Abstract interfaces.
• .COM

How About Modern C++?

As of C++17, we don't have any new features that target pimpl. With C++11, we got smart pointers, so try to implement pimpl with them - not with raw pointers. Plus, of course, we get a whole lot of template metaprogramming stuff that helps when declaring wrapper types for the pimpl pattern.

But in the future, we might want to consider two options: modules and operator dots.

Modules will play an important part in reducing the compilation times. I haven't played with modules a lot, but as I see how using pimpl just for the compilation speed might become less and less critical. Of course, keeping dependencies low is always essential.

Another feature that might become handy is the operator dot - designed by Bjarne Stroustrup and Gabriel Dos Reis. PDF - N4477 - didn't make it into C++17, but maybe will see it in C++20?

Basically, it allows you to overwrite the dot operator and provide much nicer code for all of the proxy types.

Who Uses Pimpl? 

I've gathered the following examples:

• QT:
  • This is probably the most prominent example (that you can find publicly) where private implementation is heavily used.
  • There's even a nice intro article discussing d-pointers (as they call pimpl): D-Pointer - Qt Wiki
  • QT also shows how to use pimpl with inheritance. In theory, you need a separate pimpl for every derived class, but QT uses just one pointer.
• OpenSceneGraph
  • Framebuffer object
• Assimp library
  • Exporter
  • Have a look at this comment from assimp.hpp.

  // Holy stuff, only for members of the high council of the Jedi.
  class ImporterPimpl;
  
  // ...
  
  // Just because we don't want you to know how we're hacking around.
  ImporterPimpl* pimpl;

• Open Office 
  • For example laycache.hxx - link
  • Other pimpl uses
• PhysX from Nvidia

It looks like the pattern is used somewhere!

Let me know if you have other examples.

Summary

Pimpl looks simple, but, as usual in C++, things are not simple in practice. 

The main points:

• Pimpl provides ABI compatibility and reduced compilation dependencies.
• Starting with C++11, you should use unique_ptr (or even shared_ptr) to implement the pattern.
• To make it work, decide if your main class has to be copyable, or just movable.
• Take care of the const methods so that the private implementation honors them.
• If the private implementation needs to access the main class members, then a "back pointer" is needed.
• Some optimizations are possible (to avoid separate memory allocation), but might be tricky.
• There are many uses of the pattern in open source projects, QT uses it heavily (with inheritance and back pointers).