Every well-behaved clojure source file starts with a namespace declaration. The
ns macro, as we all know, is responsible for declaring the namespace to which the definitions in the rest of the file belong, and generally also includes some requirements and imports and whatnot. But today (at Clojure/West, shoutout!) Stuart Sierra made a passing reference to the internals of
nsduring his talk that got me interested.
But what is a namespace really?
Some things in Clojure are implemented as Java classes of terrifying scope and, for reasons known and important only to the Man himself, using a perplexing and obscure formatting style. But, namespace is not one of those things, which is nice because otherwise the rest of this post would be Java source code and nobody wants to see that. (I’m perfectly comfortable with normal Java code, by the way, it’s just that clojure’s, for probably reasonable reasons, is not that).
No, you came to see some down and dirty Clojure internals, and that’s what you’re going to get. It turns out that
ns is just a regular old macro that expends to a bunch of regular old functions that just happen to be how Clojure happens to load code. In other words, we can use our good friend
macroexpand to peer (partway) down this particular rabbit hole, which as you might have guessed is what is about to happen. Luckily,
clojure.core’s code is about as transparent as the underlying Java is opaque, so everything went better than expected for the purposes of this post.
The result of
(macroexpand '(ns namespaces.test)) is the following:
(do (clojure.core/in-ns (quote namespaces.test)) (clojure.core/with-loading-context (clojure.core/refer (quote clojure.core))) (if (.equals (quote namespaces.test) (quote clojure.core)) nil (do (clojure.core/dosync (clojure.core/commute (clojure.core/deref (var clojure.core/*loaded-libs*)) clojure.core/conj (quote namespaces.test))) nil)))
Hey, that’s not so bad! Inside that do, there are 3 things happening:
in-nsis called, setting the current value of the ns var in the
clojure.corenamespace. Stateful! Shame! But, there’s a rich tradition of lisps in general working this way behind it, and it saves adding an extra indentation level to every source file by including it in some
with-nsmacro, so I think we can let this slide. (Also, judging by the Java code, someone has a vendetta against excess indentation).
- Inside the
referthe clojure.core namespace. This is good, because we always want the functions in
clojure.coreto be handy. Also,
referturns out to be the root of all code-loading in clojure; more on this later.
- Finally, if the namespace in question is not
clojure.core, we append its name (as a symbol) to the
dosynctransaction as one does when working with refs.
“I hope he’s not about to go into excruciating detail about each of those steps,” I hear you psychically mutter. Too bad for you!
From the Top:
in-ns is one of those things that Clojure defines in Java. If I wasn’t clear before, I have no intention of delving beyond that barrier, but from context it’s clear that
in-ns does what
ns would do if
ns wasn’t dedicated to encapsulating the half-dozen different things involved in initializing said namespace.
In other words,
in-ns is what you would be using if you were also using the
use functions in your code, instead of the
:use (p.s. don’t use
ns. In code form, this…
(in-ns 'namespaces.test) (require 'clojure.string)
… is about equivalent to:
(ns namespaces.test (:require clojure.string))
Well, except for all the other stuff that
ns does. Incidentally, while sternly recommending against it, the
clojure.core internals use
in-ns extensively, sometimes spreading a namespace across files. But who are we to judge?
The next thing that happens is that the
clojure.core namespace is
refer’d into the context of the namespace that was just declared the current namespace in
in-ns above. This is important because we probably want access to the functions in
refer ends up calling the
referencemethod on the
*ns* (current namespace) var, which is an instance of
clojure.lang.Namespace, which does a voodoo dance that results in the relevant symbols being made available to us.
I didn’t explore too deeply into what the undocumented
with-loading-contextmacro does, accomplishes, because Java, but perhaps it has something to do with how we’re able to use functions defined in
clojure.core to load
clojure.core. Or perhaps not?
Finally, the rigamarole surrounding the
*loaded-libs* refs is just there so that
load-lib doesn’t reload a loaded lib (er, namespace) unless specifically asked to. More about
load-lib right now:
:import) into your
ns declaration just adds a matching call to
import) to the
with-loading-context part of the namespace macro expansion.
Require and use turn out to be different flavors of the same thing:
(defn require "...docs..." [& args] (apply load-libs :require args)) (defn use "...docs..." [& args] (apply load-libs :require :use args))
load-libs is a collection of checks and whatnot wrapping a bunch of calls to
load-lib, which we met above. In turn,
load-lib translates the namespace into a path and does some other stuff and then calls
load on the path, while
loadinvokes the Java methd
clojure.lang.RT/load on it. The eldritch magic contained therein in turn grabs the file, compiles it, and puts all the top-level symbols it found into the current namespace.
import macro is a bit different, deferring to
import*, another magic clojure symbol that’s only defined in Java. The details aren’t clear to me, but I think it’s safe to assume that ultimately this just uses Java reflection to load the class.
So what can we take away from this mutually enlightening experience?
nscall is actually not so special, until you get to the special parts.
- We could manually implement just about everything with
requiredirectly, but we shouldn’t.
- I should talk to someone about my underlying Java phobia.
That’s all for tonight!