Scala Is Dead, Long Live Scala!

Recently I came across yet another post about the adoption of Scala in the IT world nowadays. The article is “The Rise and Fall of Scala” on DZone. In that post, the author describes the reasons for the failure of the Scala language in becoming the next big general purpose programming language. But, are we sure that authors of Scala ever pretended the language would be used by every developer on the planet? I don’t think so.

Introduction

First of all, who am I to write such an article? Basically, nobody. All I want to do is to report my experience with the Scala language in the last five years. I started learning Scala in 2013, attending at Martin Odersky Coursera courses. I read some books about Scala programming, among them “Scala for the Impatient” by Cay Horstmann (in my opinion, one of the best books about the Scala programming language). Last but not least, since 2015, I worked as a senior developer in a Big Data project that uses the Scala programming language extensively.

Then, during the last year, I had the opportunity to get an idea of the past, present, and future of Scala.

As-Is

Have you ever installed the JDK from scratch? I know you have. Do you remember the image that reminds you there are at least 3 billion devices running Java out there?

That's a lot of applications, don’t you think? Twenty years of Java applications built on top of a huge number of Java libraries. There are millions of developers who live and sustain their families creating and maintaining applications written in Java. Do you really think that some new language can really substitute for Java nowadays?

I don't think so. But, let’s give Scala a chance.

The Language

For those of you that don't know it, Scala is an awesome programming language. Is it simple to master? No, it is not. Definitively. Its syntax is elegant and powerful, it but hides a lot of lessons learned from the design of older programming languages, Java top among them.

For example, class declaration is mixed with constructor and attribute declaration.

class Person(val name: String, val surname: String)

There are a lot of concepts going on in the above example. The corresponding Java class would be the following.

public class Person {
    private final String name;
    private final String surname;
    public Person(String name, String surname) {
        this.name = name;
        this.surname = surname;
    }
    public String getName() {
        return this.name;
    }
    public String getSurname() {
        return this.surname;
    }
}

As you can see, a single row is written as more than 10 lines in Java. A bunch of best practices and conventions are used to translate a simple line of code into a not-so-naive class declaration.

Taking another example, let’s analyze the method apply of a class companion object. This is a native way to implement the factory method pattern, overly abused in all the modern interfaces.

class Person(val name: String, val surname: String) 
object Person {
    def apply(json: String): Person = {
        val name = // Retrieve the name from json string
        val surname = // Retrieve the surname from json string
        new Person(name, surname)
    }
}

The above example is equal to implementing a static factory method, usually named of in Java. And if you choose to use a Scala case class, the companion object comes along for free.

There are a lot of other examples that prove that the Scala programming language incorporates a lot of software engineering best practices (have a look at the post A (too) short introduction to Scala for more examples). You can deeply understand the process that brought the creators of the Scala language to make their decisions during the design only after a few years of development.

How can we pretend that the vast majority of developers all over the world can immediately understand the subtle features of this language called Scala?

Functional Programming

For years, Scala was one of the few programming languages running on the JVM that mixed the object oriented paradigm with the functional paradigm. In 2015, Java 8 introduced lambdas. Many started to say that Scala would die because Java, our favorite mainstream programming language, introduced the functional approach, too.

Unfortunately for Java, lambdas alone aren't sufficient to become a functional language. In fact, for many reasons Java lambdas are very close to being merely syntactic sugar for the use of anonymous classes.

Did you know that you can have functional code in Java 5, 6, and 7, too?

// Define a type for a Function with two parameters
public interface Function<T, U> {
    U apply(T arg);
}

// Implement a specific function
Function<Integer, Integer> square = new Function<Integer, Integer>() {
    @Override
    public Integer apply(Integer arg) {
        return arg * arg;
    }
};

Quite unpleasant, isn’t it? To overcome the problem of defining a custom type for each type of function and having such a verbose syntax, in Java 8, we added a bunch of *Function types to the API and a special syntax to declare lambdas. Then, the above code in Java 8 becomes the following.

Function<Integer, Integer> square = x -> x * x;

Is this sufficient to say that Java 8 is a functional programming language? Unfortunately, it is not.

There are many features of “real” functional programming languages that Java 8 lacks. For one, it lacks the capability for lambdas to close over variables. In fact, Java 8 lambda expressions can access variables of the outer scope if they are final, i.e. immutable. Java lambdas close over values only.

final Integer three = Integer.valueOf(3);
Function<Integer, Integer> triple = x -> x * three;

Is this a real problem? Not at all. Many programming languages discourage the use of these types of closures (also known as full-blown closures).

The main functional feature absent from Java 8 is that lambdas are not first-class citizens. Java 8 does not define any function type. Lambdas are implementations of interfaces with only one method (functional interfaces, or SAMs, classes implementing a single abstract method). What does this mean?

First of all, there is no possibility to declare a signature of an anonymous function in Java 8. Indeed, the following does not compile.

public static void runCalc((BigInteger -> BigInteger) calc) {/* ... */} 

Once you have defined a functional interface to be passed as a parameter of the method runCalc, we still have some problems. Even the following code does not compile.

interface MyCalcLambda {
    BigInteger run(BigInteger input);
} 
public static void runCalc(MyCalcLambda calc) {
    // You cannot use the functional interface as it was a function!
    System.out.println(calc(BigInteger.TEN));
}

The only way you have to overcome to this problem is to directly call the method defined in the interface.

public static void runCalc(MyCalcLambda calc) {
    System.out.println(calc.run(BigInteger.TEN));
}

In addition, Java 8 completely lacks support for Pattern matching, for-comprehension syntax, ranges, and so on. Though these features are not intimately related to functional programming, they help to work in such a domain.

In conclusion, please stop saying that Java 8 has outclassed Scala as a functional programming language.

Scala at Its Best

In conclusion, Java 8 and Scala are not comparable from many points of view. If Java can be defined a general purpose programming language, where can we use Scala effectively?

In my opinion, the best application of Scala is in those domains that request a heavy load of mathematical concepts. A clear example is the Big Data domain. Scala constructs fit perfectly with the data structures we find in Big Data libraries, i.e. Apache Spark.

Spark RDD[T], Resilient Distributed Dataset, looks like a collection at the top level of its abstraction. Then, Scala features related to collections manipulation fit perfectly to work with such a data structure.

Conclusion

As I tried to prove in this post, the aim of Scala was not to substitute for Java — not today, not in the future. The language has its own features, as does any other existing language. These features combine a lot of software engineering concepts. Also, mathematics play an important role to have a deep understanding of Scala. These facts clearly explain why Scala cannot be a widely adopted language.

However, these facts also suggest in which field of IT Scala can be successfully applied. Fields like Big Data or the creation of DSL are the best environments for Scala.

In conclusion, please, stop writing articles and posts in which the death of Scala is seen as certain. And if you think so, please try to write an Apache Spark job using solely Java 8 first. :)

PS: Indeed, speaking of functional programming, Scala is not a pure functional programming language. A pure functional programming language does not allow mutable values: Scala allows you to declare the var variable. However, if you try to define a ranking of programming languages to understand which is the nearest in purity, certainly Scala is purer than Java 8.

PPS: The Internet community is still debating about whether Java 8 lambda expressions are or are not first-class citizens.