Optional Is the New Mandatory

Since the beginning of the "computer programming era," developers searched for a solution to one of the biggest mistakes ever made in computer science, the invention of the null reference. Since functional programming became mainstream, the solution to this problem surfaced, the use of the optional type. In this post, we analyze how this type is used in different programming languages, trying to understand the best practices to apply in each situation.

You may also like: 26 Reasons Why Using Optional Correctly Is Not Optional

Returning the Nothing

Let's start with a concrete example. Think about a repository that manages users. One classic method of such type is the method def findById(id: String): User, whatever an id could be. This method returns the user that has the given id, or nothing if no such user exists in your persistence layer.

How can we represent the concept of nothing? It is common to use the null reference to accomplish this task.

val maybeUser: Option[User] = repository.findById("some id")
val longName: String = maybeUser match {
  case Some(user) => user.name + user.surname
  case None => "Not defined"
}

So, what's a possible solution? A first attempt can be to return a List[User]. The signature of the method becomes the following, def findById(id: String): List[User]. An empty list means that there is no user associated with the given id. A not empty list means that there is a user associated with the given id. The problem is that the user is exactly one, whereas the semantic of lists if to store zero or more elements.

So using a list seems to be convenient, but not semantically correct. The use of an empty list to represent the nothing produces programs that are hard to read (violating the Principle of Least Astonishment) and to maintain.

However, the idea behind the use of lists is perfect. How can we refine it?

The Option Type

The answer to our question is in the type that Scala calls Option. Other programming languages call this type in different ways, such as Optional in Java, or Maybe in Haskell.

The Option[T] type is just like a list that can be empty or can contain exactly one element. It has two subtypes that are None and Some[T]. So, an Option object that contains something is an instance of the Some[T] subtype, whereas an empty option is an instance of None. There is only one instance of the None type that in Scala is defined as an object.

Option is indeed an Algebraic Data Type, but the definition of such a concept is behind the scope of this post.

It is possible to use the factory method Option(value: T), declared in the Optionobject to build a new instance of an Option class. The factory method will create a new Option instance: If value is equal to null, then the None object is returned, Some(value) otherwise.

Using the Option type, our previous method changes its signature in def findById(id: String): Option[User]. OK, interesting. However, how can I use the user value contained inside the option type? There are many ways to do that. Let's look at them.

Getting the Value Out of the Option Type

One of the possible choices is to try to get the value out of the option. The API of the Option type lists the methods isDefined and get. The first check is if an option contains a value, and the second allows you to extract that value. The usage pattern raising from the use of the above methods is the following:

val maybeUser: Option[User] = repository.findById("some id")
val longName: String = maybeUser match {
  case Some(user) => user.name + user.surname
  case None => "Not defined"
}

Despite the use of a var reference, the above pattern is to avoid. If you call the get method on a None object, you will obtain an error at runtime. So, you don't improve the code that much from the version that checks for the null directly.

For this reason, the Scala Option type provides a variant of the get method, which is getOrElse. This method allows you to provide a default value to return in case of an empty option. Imagine that our User type provides a gender: Options[String] method, which eventually returns the gender of a user. It is possible to use the following safer pattern using getOrElse.

val maybeUser: Option[User] = repository.findById("some id")
val longName: String = maybeUser match {
  case Some(user) => user.name + user.surname
  case None => "Not defined"
}

Using Pattern Matching

In Scala, it is possible to use pattern matching on Algebraic Data Types. The Option type is not an exception in this sense. Although it is not such idiomatic, this approach does what it promises us — raw and straightforward.

val maybeUser: Option[User] = repository.findById("some id")
val longName: String = maybeUser match {
  case Some(user) => user.name + user.surname
  case None => "Not defined"
}

Using pattern matching is a little more verbose than using the next approach I am going to show you, but it is still more elegant than checking the existence of a value using the isDefined method.

Using the Option Type as a List

The suggested approach by the Scala community is to use the Option type, just as if it was a list. This approach takes advantage of the fact that the Option type is indeed a monad (also, in this case, explaining what a monad is in functional programming is behind the scope of this post).

The Option type, like lists, exposes a lot of useful methods to transform e manage the Option inner value, without even the need to extract it. Among the others, such methods are the map, flatMap, and filter functions.

The map method allows you to transform the value owned by an Option object if any.

val maybeUser: Option[User] = repository.findById("some id")
val longName: Option[String] = maybeUser.map(user => user.name + user.surname)

The flatMap method allows you to compose functions that in turn, return an object of type Option.

val maybeUser: Option[User] = repository.findById("some id")
// Using a simple map function, we obtain an Option[Option[String]]...uhh, ugly!
val gender: Option[String] = maybeUser.flatMap(user => user.gender)

Finally, the filter method allows you to discard any value that does not fulfill a given predicate.

val maybeUser: Option[User] = repository.findById("some id")
val maybeNotARiccardo: Option[User] = maybeUser.filter(user => user.name != "Riccardo")

All the above methods, if invoked on a None object, return None automatically, preserving any method call concatenation.

Note that we can combine the use of the map, flatMap, and filter method using for-comprehension, giving to our code a look more like Haskell.

val maybeUser: Option[User] = repository.findById("some id")
val maybeNotARiccardo: Option[User] = maybeUser.filter(user => user.name != "Riccardo")
val maybeTheGenderOfANoRiccardo: Option[String] = maybeNotARiccardo.flatMap(user => user.gender)

The above code can be rewritten using a simple for...yield expression.

val maybeTheGenderOfANoRiccardo: Option[String] =
  for {
    user <- repository.findById("some id")
    gender <- user.gender if user.name != "Riccardo")
  } yield (gender)

Last but not least, if you need to apply some side-effects if a value is present in an Option object, you can use the foreach method. This function cycles on the single value of the option, if present, and executes a procedure.

val maybeUser: Option[User] = repository.findById("some id")
maybeUser.foreach(user => println(s"$user.name $user.surname"))

Option Type in Other Languages

The Option type is not present only in Scala. Many programming languages widely use it.

Java

Since Java 8, the language introduced the Optional<T> type in the mainstream JVM language. It is very similar to its Scala cousin.

Also, the Java API defines different factory methods to build a new instance of Optional: Optional.of, which throws an exception if the given value is null, and Optional.ofNullable, that returns an Optional.empty in case of null value.

A small difference if the behavior of the map method. If the function used by map returns a null value, the method interprets this as the Optional.empty value. The map method behaves like the flatMap method, in this particular case:

Optional<User> maybeUser = repository.findById("some id");
Optional<String> maybeUserName = maybeUser.map(user -> {
    if ("Riccardo".equals(user.getName()) {
        return user.getName();
    } else {
        return null;
    }
});

The above code returns an Optional.empty value if the name of the user is equal to the String"Riccardo". However, as some Reddit user pointed out, the above approach is no recommended in Java. Use the following code instead.

Optional<String> maybeUserNameOtherThanRiccardo = 
    repository.findById("some id")
              .map(User::getName)
              .filter(not("Riccardo"::equals));

Another interesting method of the Java Optional type is orElseThrow. This method allows us to choose which exception to rising in case of an empty Optional object.

Is it worth reporting the answer that Brian Goetz gave to the StackOverflow question, Should Java 8 getters return optional type?.

"NEVER call Optional.get unless you can prove it will never be null; instead use one of the safe methods like orElse or ifPresent. In retrospect, we should have called get something like getOrElseThrowNoSuchElementException or something that made it far clearer that this was a highly dangerous method that undermined the whole purpose of Optional in the first place. Lesson learned."

Kotlin

The Kotlin programming language does not have built-in support for something similar to the Scala Option type or the Java Optional type. The reason why is mainly that the language support compile-time null checking.

In Kotlin, every type has two variants. Continuing our previous example, Kotlin defines both the User, and the User? type. A reference of the first type cannot be null, whereas an instance of the User? type can also hold the null value. Any non-nullable type is a child type of the corresponding nullable type.

In other words, Kotlin attempts to solve the problem by forcing you to handle null references. Kotlin forces you to handle the exception or to take full responsibility. For example, you can't use the dereferencing operator, the . (dot), if you are managing nullable type. The following code won't even compile.

val maybeUser: User? = repository.findById("some id")
val longName = maybeUser.name + maybeUser.name

Kotlin tries to prevent NullPointerException for you. The safe call operator, ?. must be used in this case. It checks if the reference is null, and if it is not, it calls the method on it. And it propagates the null value, otherwise:

val maybeUser: User? = repository.findById("some id")
val longName: String? = maybeUser?.name + maybeUser?.name

The power of the safe call operator shines with chained methods calls. In this case, it also reminds very strictly the use of the methods map and flatMap defined in the Option type.

val maybeGender: String? = 
  repository.findById("some id")?.gender

The above method is equal to the following Java counterpart:

String maybeGender = 
    repository.findById("some id")
              .flatMap(user -> user.getGender())
              .getOrElse(null);

If you don't mind about NPEs, you can always use the !!. operator that does not perform any check on reference nullability.

Finally, Kotlin defines the same behavior of the orElse method in Scala or Java for nullable types, using the Elvis operator, ?:. If we want to return a particular value of the gender if something is null during the gender resolution process, we can proceed as the following code shows.

val maybeTheGender: String = 
  repository.findById("some id").?gender ?: "Not defined"

As you can see, the type of the last maybeTheGenderOfANoRiccardo variable is not nullable anymore, because the elvis operator eliminates the possibility for the reference to be null.

Conclusions

In this post, we analyzed a modern solution to deal with the null value and in general, the absence of information. We started to see how the Scala programming language defines Option[T] type, listing the best practice of its use. Then, we saw how the good old Java approaches to the problem, the Optional<T> type, concluding that it is very similar to the solution proposed in Scala.

Finally, we looked at a different solution, the one proposed by Kotlin. Kotlin does not have any optional type, because it tries to handle the nullability of references at compile-time. Although this approach is exquisite and as powerful as the approaches proposed in Scala and Java, it poorly composes with the other constructs used in functional programming. For this reason, Kotlin has its library for functional structures, such as the optional type, which is ARROW.

Bonus: Are you planning to use an option value as a parameter for a method? Please, don't do that ( Why should Java 8's Optional not be used in arguments).

References

• Null References: The Billion Dollar Mistake
• The Neophyte’s Guide to Scala Part 5: The Option type
• Should Java 8 getters return optional type?
• Chapter 2: Functional programming in Kotlin: An overview. The Joy of Kotlin, Pierre Yves Saumont, 2019, Manning Publications

Further Reading

26 Reasons Why Using Optional Correctly Is Not Optional

Java 8 Optional: Handling Nulls Properly

Java 8 Optional Usage and Best Practices