Quorum 2.0 - Designing a Human Friendly Programming Language

While working on SodBeans, a NetBeans-based programming environment for visually impaired developers, Andreas Stefik and his project teammates had an insight: If designing an IDE for the blind required careful use of language to make the tool intuitive for its users, could the same process and criteria be adapted for designing programming languages for wider use. The answer to that question became Quorum--an object-oriented programming language designed, tested and backed by research to be easy to understand and use. Quorum 2.0 was released recently, and in this q&a, Andreas Stefik, a research professor at the University of Nevada, Las Vegas, and a member of the NetBeans Dream Team, talks about the on-going effort behind creating a whole new language.

What’s new in Quorum 2.0?

Quorum 2.0 finalizes a host of long-standing changes to the language based on empirical data we have been gathering about programming language usability. Some of these changes are very minor, like changing or removing keywords based on results of our studies. For example, we changed the word “print”--for outputting to the console--to the word “output,” a word choice that received higher scores in a controlled scientific survey we conducted.

Other changes were more substantive. For example, the academic literature has recently been revisiting the old argument regarding static and dynamic type systems. The evidence requires deep knowledge of statistics to fully understand, but the results so far suggest that static typing tends to help developers code slightly faster under a wide variety of conditions (although not all). We even have a pretty good hint as to why, which is that when using dynamic typing systems, we observe that programmers flip more often between files as they use a library and have to figure out what to pass it. While this is the primary result, another study found that novices have difficulty correctly using type annotations. Quorum 2.0 is the first version, and maybe the first language, that explicitly takes into account the results of all of the known scientific experiments related to type systems. A brief overview of some of the changes are listed in our release notes.

How and where is Quorum being used?

Quorum began as a National Science Foundation project geared toward teaching blind children to program. As such, our primary focus was initially in making sure it works for that target group at residential schools for the blind. We actually just held the 4^th annual Quorum workshop and brought in about 25-30 teachers for the blind across the U.S and Canada. While Quorum was originally designed for this purpose, it has changed quite a bit over the years and we now have people downloading it from all over the world.

Is Quorum an easier language to learn or to use than other programming languages?

As you can probably imagine, evaluating whether a programming language is easy to use is not a simple question and many have written extensively about it in peer reviewed papers, including myself. While it should be obvious to most people that programmers have different needs, work in different environments, and are trying to solve different problems, academics studying usability issues carefully are finding that some designs really do lead to more productivity over others under common circumstances.

In Quorum, we have a process for improving the usability of the language over time. It basically goes like this: we 1) conduct regular cycles of experiments on the use of the language, we 2) conduct regular analysis of new evidence in the scientific literature, and we 3) have regular cycles of implementation, to change the language according to the latest evidence. As ideas come out that confirm, or refute, particular design features, Quorum is adjusted.

What's been the feedback about Quorum?

Overall, extremely positive. Birthing a programming language isn't an easy process and this project will take years of data collection to really hone the language to be as easy to use as possible. With that said, when people actually use it, we overwhelmingly get positive feedback. I think most reasonable people recognize that there are tradeoffs in language design and that no design is perfect, including that of essentially all commercial programming languages.

With that said, most of the evidence on Quorum's design is either publicly available or is made available over time as we get it peer reviewed in academic journals. I think most people that have used Quorum, we are told, appreciate that we aren't making decisions arbitrarily, that they can lookup the evidence for or against particular decisions, and that if they feel strongly enough that they can run their own controlled scientific study to refute us.

For example, in Quorum 1.7, we changed the if statement design to be much closer to the syntax used in the language Ruby, as one of our studies showed their syntax was easier for novices to use. This study isn't out yet, but has been accepted by the journal ACM Transactions on Computing Education (TOCE).

Is Quorum only for use in the SodBeans IDE?

No, Quorum is a general purpose programming language that can connect to other languages (e.g., access the JDK or C++) or do whatever else you would expect. Developers can use it freely as part of Sodbeans, with or without the blind accessibility features turned on, but the NBM files can be added as an update center from NetBeans just like any other language. We also have a command line client for developers that just prefer a text editor. You can also run a limited version of Quorum from the web.

The updates go beyond the language itself. There's also a new website?

Yes, that's true. We have quite a few plans for the website in the future, which we started integrating in this release. For example, the Quorum compiler now has an opt-in feature allowing users to submit data to us on usage, in addition to having a Netflix-style ratings system for the growing standard library. Besides this, the website now has a web interface, allowing users to type Quorum code and run it online. Our hope is that, over the next few years, this will make it easy for users to give Quorum a try, in addition to, hopefully, giving us yet more data on usage of the language.

What's your long-term goal/vision for Quorum?

I'm sure some will say it's a pipe dream, but my long-term vision with Quorum is to, in a very real and literal sense, have formal scientific data on every aspect of Quorum's design and to use this data to improve the language. For example, I want to know the tradeoffs with every word choice in the standard library. I want to document the impact of as many syntactic alternatives as possible and to formally compare them in scientific studies. I want to know how every feature of alternative programming languages stacks up against each other (e.g., should closures be in a language, what kind of type system should be used?) and to include only the best in Quorum.

Realistically, this will take decades. Fortunately for me, as a professor, I have the luxury to be able to think about this project in the short term (e.g., what features should we put out this release?) and also the long term (e.g., if we spend 30 years running usability studies on Quorum, how easy to use can we make it compared to typical languages from 2013?).

A final thought on this question: I think another way I hope Quorum can contribute is to help domain specific language designers. Many folks that create these languages are prolific computer scientists, but do not have sufficient training in psychology or statistics to be able to fairly evaluate usability claims. I hope that Quorum is helpful to these individuals in the sense that they can use the language as a baseline for their syntax and semantics. If they expand the language further, then all the better, as it gives our lab something new to test to see if it's helpful, while providing them a solid foundation of evidence for changes they probably don't want to think about anyway.

Read more about Andreas Stefik's work on accessibility on NetBeans Zone here, here and here.