Generic Programming With Scala and Shapeless

Last year, I spent some time playing with and writing about Scala and Shapeless, walking through the simple example of generating random test data for a case class.

Recently, I have played some more with Shapeless. This time with the goal of generating React (javascript) components for case classes. It was a very similar exercise, but this time I made use of the LabelledGeneric object, so I could access the field names. I thought I'd re-visit that here and talk a bit about some of the internals of what is going on.

Getting Started

As I mentioned before, I had to define implicits for the simple types that I wanted to be able to handle, and the starting point is, of course, accepting a case class as the input.

implicit def caseClassToGenerator[A, Repr <: HList](
    implicit generic: LabelledGeneric.Aux[A, Repr],
    gen: ComponentGenerator[Repr],
    reactComponent: ReactComponentGenerator[A]
  ): ComponentGenerator[A] =
    new ComponentGenerator[A] {
      override def generate = reactComponent.generate(gen.generate)
    }

So, there are a few interesting things going on here:

First of all, the method is parameterized with two types: caseClassToGenerator[A, Repr <: HList] A is simply going to be our case class type, and Repr is going to be a Shapeless HList.

Next up, we are expecting several implict method arguments. We will ignore the third implicit for now — that is an implicit that I am using purely for the react side of things. This can be skipped if the method handles everything itself:

implicit generic: LabelledGeneric.Aux[A, Repr], gen: ComponentGenerator[Repr],

Now, as this method's purpose is to handle the input of a case class and we are using shapeless, we want to make sure that from the starting input we can transform it into an HList. Then, we can start dealing with the fields one by one. In other words, this is the first step to converting a case class to a generic list that we can then handle element by element. In this setting, the second implicit argument is asking the compiler to check that we have also defined an appropriate ComponentGenerator.  This is my custom type for generating React components that can handle the generic HList representation. It's no good being able to convert the case class to its generic representation if we then have no means to actually process a generic HList.

Straight forward so far?

The first implicit argument is a bit more interesting. Functionally, all the LabelledGeneric.Aux[A, Repr] is doing is asking the compiler to make sure we have an implicit LabelledGeneric instance that can handle converting between our parameter A (the case class input type) and Repr (the HList representation). This implicit means that if we try to pass some type A to this method, the compiler will check that we have a shapeless LabelledGeneric that can handle it. If not, we will get a compile error.

But, things get more interesting if we look more at what the .Aux is doing!

Path Dependent Types and the Aux Pattern

The best way to work out what is going on is to just jump into the Shapeless code and have a dig. I will proceed to use Generic as an example, as its a simpler case, but its the same for  LabelledGeneric:

trait Generic[T] extends Serializable {
  /** The generic representation type for {T}, which will be composed of {Coproduct} and {HList} types  */
  type Repr

  /** Convert an instance of the concrete type to the generic value representation */
  def to(t : T) : Repr

  /** Convert an instance of the generic representation to an instance of the concrete type */
  def from(r : Repr) : T
}

That's a lot simpler than I expected to find, to be honest, but as you can see from the above example, there are two types involved: there is the trait parameter T and the inner type Repr. The Generic trait is just concerned with converting between these two types.

The inner type, Repr, is what is called a path dependent type in Scala. That is, the type is dependent on the actual instance of the enclosing trait or class. This is a powerful mechanism in Scala. But, it is also one that can also catch you out, if you are in the habit of defining classes within other classes or traits. This is an important detail for our Generic here, as it could be given any parameter T,   so the corresponding  HList  could be anything, but this makes sure it must match the given case class T. That is, the Repr is dependent on what T is.

To try and get our head around it, let's take a look at an example:

case class Simple(number: Int, working: Boolean, someText: String)

val s = Simple(2, true, "hello")
// s: Simple = Simple(2,true,hello)

val sGen = Generic[Simple]
// sGen: shapeless.Generic[Simple]{type Repr = Int :: Boolean :: String :: shapeless.HNil} = anon$macro$4$1@215c6699

Cool. So, as we expected, we can see that, in our Generic example, we can also see that the type Repr has been defined by matching the HList representation of our case class. It makes sense that we want the transform the output HList to have its own, specific type based on whatever input it was transforming. But, it would be a real pain to have to actually define that as a type parameter in the class along with our case class type, so it uses this path-dependent type approach.

So, we still haven't got any closer to what this Aux method is doing, so let's dig into that.

The Aux Pattern

We can see from our code that the Aux method is taking two parameters. Firstly, we will look at A , which is the parameter that our Generic will take, but the Aux method also takes the parameter Repr, which we know — or at least pretend we can guess — corresponds to the path dependent type that is defined nested inside the Generic trait.

The best way to work out what is going on from it is to take a look at the Shapeless code!

type Aux[T, Repr0] = Generic[T] { type Repr = Repr0 }

As we can see, the Aux type, which is defined within the Generic object, is just an alias for a  Generic[T], where the inner path-dependent type is defined as Repr. They have a pretty decent explanation of what is going on in the comments, so I will reproduce that here:

/** Provides a representation of Generic[T], which has a nested Repr type, as a 
   * type with two type parameters instead.
   *
   * Here, we specify T, and we find a Generic.Aux[T,R] by implicit search. 
   * We then use R in the second argument.
   * Generic.Aux[T, R] is exactly equivalent to Generic[T] { type Repr = R }, 
   * but Scala doesn't allow us to write it this way:
   *
   * {{{
   *   def myMethod[T]()(eqGen: Generic[T] { Repr = R }, reqEq: Eq[egGen.Repr) = ???
   * }}}
   *
   * The reason is that we are not allowed to have dependencies between arguments 
   * in the same parameter group. So Aux neatly sidesteps this problem.
   *
   */

The above code has been abbreviated for the more relevant pieces. 

This nicely sums up the Aux pattern. This pattern allows us to essentially promote the result of a type-level computation to the higher level parameter. It can be used for a variety of things where we want to reason about the path dependent types. Besies this, this is a more common use for the pattern.

So, thats all I wanted to get into for now. You can see the code here!