Laziness in Clojure - Some Thoughts
Join the DZone community and get the full member experience.
Join For FreeSome readers commented on my earlier post on Thrush
implementation in Clojure that the functional way of doing stuff may
seem to create additional overhead through unnecessary iterations and
intermediate structures that could have been avoided using traditional
imperative loops. Consider the following thrush for the collection of
accounts from the earlier post ..
(defn savings-balance
[& accounts]
(->> accounts
(filter #(= (:type %) ::savings))
(map :balance)
(apply +)))
Is it one iteration of the collection for the filter and another for the map ?
It's not. Clojure offers lazy sequences and the functions
that operate on them all return lazy sequences only. So in the above
snippet Clojure actually produces a composition of the filter and the map that act on the collection accounts in a lazy fashion. Of course with apply,
everything is computed since we need to realize the full list to
compute the sum. Let's look at the following example without the sum to
see how Clojure sequences differ from a language with eager evaluation
semantics ..
user> (def lazy-balance
(->> accounts
(filter #(= (:type %) ::savings))
(map #(do (println "getting balance") (:balance %)))))
#'user/lazy-balance
lazy-balance has not been evaluated - we don't yet have the printlns. Only when we force the evaluation we have it computed ..
user> lazy-balance
(getting balance
getting balance
200 300)
Had Clojure been a strict language it would have been stupid to follow
the above strategy for a large list of accounts. We would have been
doing multiple iterations over the list generating lots of intermediate
structures to arrive at the final result. An imperative loop would have
rested the case much more cheaply.
Laziness improves compositionality. With laziness, Clojure
sequences and the higher order functions on them essentially reify
loops so that you can transform them all at once. As Cale Gibbard
defends laziness in Haskell with his comments on this LtU thread .. "It's laziness that allows you to think of data structures like control structures."
Clojure is not as lazy as Haskell. And hence the benefits are also not
as pervasive. Haskell being lazy by default, the compiler can afford to
make aggressive optimizations like reordering of operations and
transformations that Clojure can't. With Haskell's purity that
guarantees absence of side-effects, deforestation optimizations like
stream fusion generates tight loops and minimizes heap allocations. But
I hear that Clojure 1.2 will have some more compiler level
optimizations centered around laziness of its sequences.
Laziness makes you think differently. I had written an earlier post
on this context with Haskell as the reference language. I have been
doing some Clojure programming of late. Many of my observations with
Haskell apply to Clojure as well. You need to keep in mind the idioms
and best practices that laziness demands. And at many times they may
not seem obvious to you. In fact with Clojure you need to know the
implementation of the abstraction in order to ensure that you get the
benefits of lazy sequences.
You need to know that destructuring's & uses nthnext function which uses next that needs to know the future to determine the present. In short, next doesn't fit in the lazy paradigm.
The other day I was working on a generic walker that traverses some recursive data structures for some crap processing. I used walk from clojure.walk, but later realized that for seqs it does a doall
that realizes the sequence - another lazy gotcha that caught me
unawares. But I actually needed to peek into the implementation to get
to know it.
Being interoperable with Java is one of the benefits of Clojure.
However you need to be aware of the pitfalls of using Java's data
structures with the lazy paradigm of Clojure. Consider the following
example where I put all accounts in a java.util.concurrent.LinkedBlockingQueue.
(import '(java.util.concurrent LinkedBlockingQueue))
(def myq (new LinkedBlockingQueue))
(doto myq (.put acc-1) (.put acc-2) (.put acc-3))
Now consider the following snippet that does some stuff on the queue ..
(let [savings-accounts (filter #(= (:type %) ::savings) myq)]
(.clear myq)
(.addAll myq savings-accounts))
Should work .. right ? Doesn't ! filter is lazy and hence savings-accounts is empty within the let-block. Then we clear myq and when we do an addAll, it fails since savings-accounts is still empty. The solution is to use a doall, that blows away the laziness and realizes the filtered sequence ..
(let [savings-accounts (doall (filter #(= (:type %) ::savings) myq))]
(.clear myq)
(.addAll myq savings-accounts))
Of course laziness in Clojure sequences is something that adds power to
your abstractions. However you need to be careful on two grounds :
- Clojure as a language is not lazy by default in totality (unlike Haskell) and hence laziness may get mixed up with strict evaluation leading to surprising and unoptimized consequences.
- Clojure interoperates with Java, which has mutable data structures and strict evaluation. Like the situation I described above with LinkedBlockingQueue, sometimes it's always safe to bite the bullet and do things the Java way.
From http://debasishg.blogspot.com/2010/05/laziness-in-clojure-some-thoughts.html
Opinions expressed by DZone contributors are their own.
Comments