How to Streamline the Customer Experience with Monads in Kotlin

 At my company, we see a lot of SDKs and Swagger-generated clients that could throw exceptions at any time. This could be a fault in our logic, or it could be a fault with some 3rd party SDKs that have no rhyme or reason to how their exception handling works. But either way, when our customers want to fetch a Git commit history for a service, they do not want to be greeted with an error message.

We've seen GitHub go down during a customer demo and 3rd party integrations throwing other unexpected exceptions. Overall, it was a very discouraging experience for our customers that we had no control over.

Thus, `IntegrationValue<T>` was born. It is a type that can return any value but encapsulates all possible errors from 3rd party integrations.

Now expected and unexpected exceptions and errors are (mostly) handled gracefully and we can surface clean error messages to our customers when we cannot retry or handle them.

Okay... So What Is This IntegrationValue<T>?

It's a monad. (Just kidding, but will get back to this concept at the end.)

We do all of our backend development in Kotlin, and `IntegrationValue<T>` is first and foremost a way to work around the fact that Kotlin doesn't support checked exceptions. But even more than exceptions, it can capture unexpected errors that don't necessarily raise exceptions.

For example, if I want to fetch a service's Github repo details, but the workspace doesn't have a GitHub integration enabled, what should it return? We could return null, but what does this mean for the caller of `GithubService.gitDetails(serviceId: Long): GitDetails?`?

It could be that:

• There is no GitHub integration
• There is a GitHub integration, but the service is not tied to any GitHub repository
• There is a GitHub integration, the service  has a repo attached, but the repo doesn't actually exist on GitHub
• There is a GitHub integration, the service has a repo attached, the repo exists, but our GitHub app doesn't have the right permissions to see it... or we hit a 429, or GitHub is down, or a random exception was thrown somewhere along the way
• ???

Basically, it could be one of a plethora of errors, and we don't want to callously bundle them all together in our type system as a null. Rather we should be explicit on what exactly those error types could be.

And so we created `IntegrationError`, which is a sealed class (like a compiler-friendly enum in Kotlin) of possible error types for all of our integrations.

This is exactly what we need from our 3rd party integrations, where we need to surface clean error messages back to the customer as well as have tightly scoped error handling strategies, depending on the error.

One failing of a monolithic error type is that it might encompass errors that are not possible for a specific operation, but for this reason, we only limit this type to code involving 3rd party integrations. But we’ll get back to this at the end.

Okay Fine, but You Still Haven’t Said What Integrationvalue<T> Is

Almost there! So now if we need all of our integration services to return possible error types, and we don't want to throw exceptions because callers are unaware, we need a way to bake errors into our type system.

Basically, the resulting type of integration service function is either a failure (`IntegrationError`) or a success (any type `T`). And there's a handy concept from functional programming land that captures this concept: `Either<L, R>`.

Going back to our GitHub example, now we can transform that useless null version into:

sealed class IntegrationError(open val errorMessage: String) {
    data class MissingIntegration(integration: IntegrationType) : IntegrationError("Missing integration ${integration.name}")
    data class MissingData(override val errorMessage: String) : IntegrationType(errorMessage)
    data class HttpException(statusCode: Int, override val errorMessage: String): IntegrationError("Unexpected HTTP error ($statusCode): $errorMessage")
    data class UnexpectedException(t: Throwable): IntegrationError(t.message ?? "Unexpected error")
}

fun gitDetails(serviceId: Long): Either<IntegrationError, GitDetails?> {
    if (missingIntegration()) {
        return Left(IntegrationError.MissingIntegration(IntegrationType.GITHUB))
    }
    val repoName = fetchRepoName(serviceId) ?: return Right(IntegrationError.MissingData(“Invalid GitHub repository”))
    val client = generateClient()
    return try {
        Right(client.fetchRepoDetails(repoName))
    } catch (e: HttpException) {
        Left(IntegrationError.HttpException(e.statusCode, e.message))
    } catch (e: Throwable) {
        Left(IntegrationError.UnexpectedException(e))
    }
}

So now callers of `GithubService.gitDetails` can explicitly handle specific error cases:

• Missing integration? Should alert the user to go to the settings page and add integration.
• Unexpected 403? Should alert the user if they need to update permissions on their API key.
• Unexpected exception? Should log and return opaque error message back to the client.

And to finally answer your question, `IntegrationValue<T>` is just a synonym for `Either<IntegrationError, T>`.

This is done because all integrations should have unified error types, and allows us to chain these requests with the power of monads.

MO-What?

Monads!

Monads are a pretty abstract concept — just google "What Are Monads" to see literally hundreds of articles trying to explain it simply. And now n+1 articles.

But the best explanation to conceptualize them is that they "box" values. We could have some value `T`, and the "boxed" version of `T` is `IntegrationValue<T>`.

Explicitly monads can do exactly 3 things:

• Box a value:

  • fun `box(t: T): Boxed<T>`

• Transform the inner boxed value to another value with some function:

  • `fun<U> Boxed<T>.map(f: (t: T) -> U): Boxed<U>`

• Transform the inner boxed value to another boxed value:

  • `fun<U> Boxed<T>.flatMap(f: (t: T) -> Boxed<U>): Boxed<U>`

• flatMap is needed to avoid cases where we continually box values... we don't want a situation where we get `Boxed<Boxed<Boxed<U>>>>`

These methods `map` and `flatMap` look familiar because our friend `List<T>` is secretly a monad! And many other Collections. And Kotlin's nullable type `T?`. And many more.

To drive the point home, the 3 methods for `List<T>` are:

• `listOf` to box a value
• `map` to transform within the box
• `flatMap` to transform within the box to ANOTHER box, yet end up with a flattened `List<T>`, instead of `List<List<T>>`

Similarly, the Either monad (IntegrationValue) has:

• `T.success()`, and `IntegrationError.failure()` to create boxed success/failure types respectively
• `map` to map a successful T to another successful U.
• `flatMap` to map a successful T to a possibly errored `IntegrationValue<U>`.

  • If `IntegrationValue<U>` is `Either.Left`, the final result will be the new failure

  • If `IntegrationValue<U>` is `Either.Right`, the final result will be the new success case.

How IntegrationValue as a monad works are that it eagerly fails the first time it encounters a failure (IntegrationError).

Suppose we try to map a failed `IntegrationValue<T>` to another `IntegrationValue<U>`. In that case, the computation to map won't even happen, and the final result will be the first error we found.

Bringing It All Together

And this brings us to the final point, that Arrow (our functional programming library of choice) comes with some handy utilities so that we don't need to chain a million maps and flatMaps to get any work done. We can use Arrow’s Fx library to easily chain together IntegrationValues with some syntactical sugar:

fun first(): IntegrationValue<T>
fun second(): IntegrationValue<U>
fun third(t: T, u: U): IntegrationValue<V>
val finalResult: IntegrationValue<V> = Either.fx {
    val unwrappedFirst: T = !first()
    val unwrappedSecond: U = !second()
    val unwrappedThird: V = !third(t = unwrappedFirst, u = unwrappedSecond)
    unwrappedThird
}

Using that magical `!`.

The Fx library is built upon Kotlin coroutines and guarantees the purity of the monads with referential transparency. Which is a fancy way of saying we can’t cheat and “escape” the monad (or boxed value).

How it works is that it (in the context of the Either.fx block) tries to unbox an IntegrationValue, and if the result was an error (a Left), then it eagerly exits with that error without doing any following computation. Otherwise, it continues sequentially through the block.

It's handy just because the alternative would be a nested mess:

fun first(): IntegrationValue<T>
fun second(): IntegrationValue<U>
fun third(t: T, u: U): IntegrationValue<V>
val finalResult: IntegrationValue<V> = first()
    .flatMap { unwrappedFirst ->
        second().flatMap { unwrappedSecond -> {
            third(t = unwrappedFirst, u = unwrappedSecond)
        }
    }

And you can just imagine how arbitrarily complex that could get.

Monads are especially helpful for us here at Cortex because as our codebase grows, it becomes easier and easier for errors to propagate without our knowledge. Pure monads enable us to pin down possible sources of errors and force callers to deal with all possible types of errors if they want to unbox the value they’re looking for.

Overall, monads have been an amazing abstraction for our codebase, and have allowed us to streamline the customer experience through unexpected errors.