Challenges Scala Must Overcome to Cross the Chasm to the Mainstream
Scala is a wonderful language that I've come to love. Unfortunately, in the last 6 months I've been guilty of being overtly critical at times. My criticism probably stemmed from actual long term use of the language on bigger projects in combination with an infatuation with Haskell (an infatuation which remains strong). There's nothing to take the shine off a new toy like pro-longed use and discovery of its inevitable warts. Despite this, I still love Scala, and to this day it is my goto language when it comes to getting stuff done, while at the same time having a high level of confidence in that the language and type system will mostly help me rather than undermine me.
I believe Scala is the most likely candidate among the current crop of functional languages to cross the chasm into mainstream use, and I certainly hope it does so. The combination of a growing community and eco-system, backwards compatibility with existing investments, the familiarity of the JVM in operations and a genuinely solid language ensures such hopes are not far fetched. However, to cross the chasm (term borrowed from Geoffrey A Moore's seminal book), Scala faces a number of challenges, many which have become apparent to me after seeing the language used on multiple projects, across multiple teams with varying skillsets. Rather than avoiding facing these challenges, I think the leadership of the language should face them head on and try to overcome them.
Just Being "Multi-Paradigm" Doesn't Cut it - Scala Must Find It's Style
Scala allows for a wide range of programming styles, from "Java without
semi-colons" to "Haskell light on the JVM". It seems to me elements of
the Scala community have prided themselves on the fact that there is "no
idiomatic way" in Scala. Unfortunately this wide range of stylistic
choices is absolutely toxic on any project with more than a couple of
developers where skills and preferences differ widely: not only do the
styles at different ends of the spectrum not mix particularly well (an
understatement), but the impact on an inter-personal/communication level
can also be damaging.
If Scala is to move beyond being a temporary curiosity, Scala must develop its own style, idiomatic ways and patterns of building applications. I'm not suggesting going all "Java EE" on the language, but I am suggesting the language must become more opinionated, in the way that Clojure and Haskell are opinionated: in both Clojure and Haskell, it is immediately apparent when you are not doing things in a "Clojure way" or a "Haskell way", as the languages nudges you by making things very hard for you. This nudge forces you to think about more idiomatic ways of achieving your goal. Rather than pussy-footing around with "no right or wrong way of doing things", Scala must become more opinionated and nudge developers away from bad ways of doing things.
"Java without semi-colons" Is Selling Scala Short And Not Good Enough
Selling Scala as "Java without semi-colons" is a massive disservice to Scala. It doesn't even scratch the surface of what Scala could potentially offer to an organisations software development efforts. The only end result of selling Scala as a syntactically sugared improvement of Java is that organisations will use it as if it was Java, with Java libraries and patterns, and eventually back off and go back to Java as the investment in re-skilling a lot of people simply isn't worth the minor improvement. This is in fact a pattern I've both heard about and seen: projects try out Scala, fail to see what all the fuss is about, then proceed to go back to Java. Status Quo rules.
Acknowledge That There Is a Learning Curve And Educate People About It
Most people pushing for Scala adoption are surprisingly shy about
acknowledging that there is a significant learning curve in order to
master the language. In fact, I was once chastised on Twitter for
suggesting that without a solid background in functional programming it
is difficult to master Scala in less than a couple of years.
This missed the point entirely: you can quite easily be very productive in a new language quickly, especially in Scala, but productivity does not imply mastery. Consider the analogy to martial arts grading: In many schools of Wing Chun Kung Fu, a "Yellow sash" is the first level of grading, a holder of a Yellow sash may very well be competent at self-defense compared to someone untrained in martial arts, however they are no Black sash, or even anywhere close to it. Competence and productivity do not imply anything resembling mastery.
Furthermore, I believe the lack of acknowledgement of a learning curve does two other disservices to both Scala and learners: firstly, the common perception from Java developers that Scala is "complex" or "hard to read" is often likely due to having unrealistic expectations of little/no learning curve, then getting the shock of not immediately understanding everything.
The second disservice is the opposite, the Dunning-Kruger effect of relatively new Scala developers believing they have mastered the language because they are productive, despite barely having scratched the surface of what they could learn about the language. This not only slows down the learning progress of those afflicted, but also holds back teams as a whole as they may not realise just how much more leverage they can get from the language. It is also the most common cause of "Java without semi-colons"-like Scala codebases, littered with all the same issues that a Java codebase would have.
I have high hopes for Scala as a language, and I am an enthusiastic proponent of its adoption. However practical experience from "real-life projects" has somewhat dampened my initial belief. Like any new technology or innovation, Scala faces significant challenges to cross the chasm from early adopters into the mainstream. A failure to address these challenges will mean a failure of getting adoption beyond the enthusiastic group of early adopters. In itself, it wouldn't be a disaster, but it would certainly be a tragedy: mainstream software development needs a solid functional language with a strong, static type-system, and it seems to me that Scala is by far the most likely contender given its JVM credentials.