Hyperlambda, Living Tree Structures
By imagining tree structures as executable objects, weird things surfaces as a natural result of that imagination, such as having computer systems that generates code
Join the DZone community and get the full member experience.Join For Free
This is the continuation of my previous article where I walked you through how to create database HTTP CRUD REST endpoints. This article though goes into the depths of Hyperlambda as a programming language, and in such a regard explains how the computer can generate the code it is generating.
Fundamentally Hyperlambda from Magic is just a text representation of a tree structure, the same way XML, YAML and JSON is. Hyperlambda's syntax though is both simpler, more humanly readable, and more easily parsed than any of the alternatives - Including YAML. To illustrate this let us look at some example Hyperlambda.
In the above Hyperlambda the [.data] parts is a Node. This node has a name of ".data" and a null value. In addition it has a list of children nodes being [item1], [item2] and [item3]. These children nodes have values though, which can be found to the right hand side of the ":" parts. To understand this, realise that the Node class again looks roughly like the following.
The above example is "pseudo code" intended to simplify understanding, but this is basically the main parts of the Node class. If you compare the Node class to for instance JSON, you will see it has one additional axiom, being an explicit children collection. JSON in such a regard is simpler, because it only has a name and a value, in addition to that in JSON the same name cannot be repeated because JSON maps to fields on classes - So substituting Hyperlambda with JSON, would make things much more verbose, even though technically it is possible. The same is true for XML and YAML. Fundamentally Hyperlambda has only 3 tokens.
- A colon ":" separates a node's name from its value
- A carriage return ends the declaration of a node
- 3 additional spaces below a node declares its children collection
In addition Hyperlambda allows for declaring node's values using the same semantics as a C# string, which allows you to declare values with carriage returns, and other special characters. As a final touch, Hyperlambda also allows you to declare types, allowing you to declare what type its value parts is. Below is a Hyperlambda snippet that declares an integer, a decimal, and a DateTime value.
Type declarations are basically injected in between the node's name and its value, and are simply references to CLR types such as System.DateTime, System.Decimal, etc. The default type if no type is explicitly supplied is string.
The typing system of Hyperlambda is crucial for what follows, and also makes it almost impossible to substitute Hyperlamdba with JSON or YAML - In addition, substituting Hyperlambda with XML, would literally explode its verbosity, resulting in that the number of bytes required to represent the same object in XML as you represent in Hyperlambda, would probably increase the size of your files by one order of magnitude, possibly more. So Hyperlambda cannot easily be replaced, and it serves a very specific purpose, which you can read about in what follows.
If you break down the compilation process of for instance a C# method, you will see that the first thing the compiler does, is to create a tree structure, or a graph object from your method. Below is an example.
During compilation the above is broken down into a tree structure, where you have one root object, being the method as a whole, and a list of children being the if and else parts in the above example. This process is recursively applied for all of your C# code. As the compilation process is finished, this temporary tree structure is discarded, and some sort of final executable is created as its result. Hyperlambda is similar, except it skips the last parts, and ends up using this "temporary tree structure" as its actual "executable". Let's illustrate how the above code could be "transpiled" into Hyperlambda to clarify.
The above is more or less the exact replica of the first C# code, except of course it is Hyperlambda. As you execute the above code, what you're basically doing, is to provide the root Node as an argument to Hyperlambda's [eval] slot, which again sequentially reads each child node of your execution node, and unless the node starts with a period ".", it is assumed to be a reference to a "slot". This slot is then "signaled", implying some sort of CLR method on some class will be invoked. This makes Hyperlambda super extendible, allowing you to extend it with your own C# methods as you see fit. Below is an example.
The above C# code creates a slot who's name is [foo] that you can invoke from your own Hyperlambda code. Below is an example of consuming the above slot from Hyperlambda.
The above code invokes your [foo] slot, and then the semantics of the [if] will check its first argument after evaluating it, and if it's true, it will execute the [.lambda] node. If [foo] is not true, it will execute the [else] node. This process allows you to easily extend Hyperlambda with your own custom slots, and in fact everything in Hyperlambda is a slot - Including the [if] and [else] parts from the above code. You can check out the code for both of these two slots below to confirm that even the "keywords" in Hyperlambda are just simple slots.
This makes Hyperlambda a DSL engine, or "An engine to create your own Domain Specific Language". Hence, the slots you create yourself is in no ways discriminated between that which you probably perceive as "keywords" in Hyperlambda - And you could replace the [if] slot quite easily with your own custom implementation if you wanted. Or for that matter simply remove the [if] slot, if you for some reasons don't want to have if statements in your own programming language.
Hence, Hyperlambda doesn't have keywords, only slots, and all slots are recursive in nature, and lambda objects by themselves. Something you can verify by seeing how even the [if] slot recursively invokes [eval] to evaluate its children.
This is true to such an extent that even what you'd imagine as "operators" in traditional programming languages are also slots. This has huge advantages, in that it among other things allows you to declare your own "operators", in addition to that Hyperlambda can easily be traversed semantically, to see which operators it is using, and Hyperlambda files can also be easily "generated", by combining programming language fundamentals together, to create a result of some sort - Which of course is the foundation for the "Generator" in Magic.
Hyperlambda is weird for a purpose
Hyperlambda literally does not have variables - Or to be specific, everything is a variable in Hyperlambda. This is because of that Hyperlambda has the ability to change any node's value, name, and/or children collection. Run the following code through for instance the "Evaluator" in Magic's dashboard to see this process.
The above Hyperlambda changes the value of one node, and the name of another node. This makes Hyperlambda extremely dynamic in nature - Much more dynamic than other dynamic programming languages, since the code might change itself, during the execution of itself. The code is said to be "self evolving during its execution process". In fact, in Hyperlambda there is no differences between creating code, modifying code, and executing code. These constructs are basically just aspects of the same process in Hyperlambda, and of course the reason why it is so easy to create code that generates code with Hyperlambda.
However, you can still provide "data segments" in your code, which of course is crucial to hold temporary variables as your code is executing. This is achieved by making sure you start your node's name with a period "." - Which only works because the [eval] keyword will simply ignore all nodes starting with a period in their names. Below is an example of a [while] snippet that increments some integer value, and iterates 500 times before ending its iteration.
The [mt] slot above is an acronym for "more than", and simply returns true as long as its value is more than its second argument - And the [math.decrement] slot simply decrements the value of its expression by one. This results in that the above [.lambda] object, which is an argument to the [while] slot executes 500 times - Once for every value between 500 and 0. As the [mt] returns false, the while loop is stopped. The [.no] node above is said to be a data segment.
Every node starting with a period (.) is ignored by the Hyperlambda execution engine, and assumed to be a data node
Since the Hyperlambda [eval] slot is recursive in nature, this allows you to declare entire segments of data using the same technique, without having to prefix children of data segments with a period. Below is an example.
Since the above [.data] segment is ignored by [eval] as a whole, none of its children will be raised as signals, and inside of our data segment, we no longer need to prepend nodes' names with a ".".
This is probably the most complex part of Hyperlambda, but if you have ever used XPath expressions, it shouldn't be too problematic, since they're more or less similar in nature. An expression is basically a "pointer" into a sub-set hierarchy of your node structure. For instance, the following expression ...
... will return all children nodes of all nodes named [.data] that can be found as children of your root node. This is because of that its first part ".." implies "give me the root node". The second part "/*" implies give me all its children. The third part "/.data" implies filter out all nodes not named ".data". The last part "/*" implies give me all its children nodes again. This gives you query capabilities into your node structure, allowing one node to reference other nodes. Combined with for instance the [get-value] or the [get-name] slots, this allows you to retrieve any parts of ay other nodes in your tree. Below is an example.
An expression is composed of iterators, which are basically IEnumerable<Node> function objects, where the result of the first iterator is applied as the input of the next iterator in a chain, resulting in that the expression as a whole yields a list of nodes. The above expression being the value of the [get-value] slot for instance has 3 iterators. Magic contains many different iterators, allowing you to query your node structure any ways you see fit. Below you can find the links to the reference documentation for these parts of Magic.
- Read more about expressions, nodes and Hyperlambda
- Read more about eval and its programming language constructs
This was probably the hardest tutorial we've gone through so far - But once you understand expressions, nodes, Hyperlambda, and its relationship to slots and signals - Everything becomes easy. In this tutorial we covered the following subjects.
- Hyperlambda syntax
- The Node class
- Executing lambda objects or nodes
- Creating slots from C#
Golly gosh this is weird!
Hyperlambda is "weird", but it needs to be weird, in order to facilitate for that which it does. Every time you think about how weird it is, and this bothers you, please realise that without this "weirdness", things such as the automatic generating of your HTTP backends, etc, would simply not be possible. I didn't create it weird because I wanted it to be weird, I created it weird because it needed to be weird. However, you can also just simply completely ignore its weirdness, and let the computer do all the work. Simply put because ...
Hyperlambda's purpose is to have your computer understand Hyperlambda
... whether or not you actually understand it is second priority. Simply since this simple trait allows for one computer to create programs that runs on another computer, or itself for that matter. And that's its purpose; To replace the need for manually creating code!
Published at DZone with permission of Thomas Hansen. See the original article here.
Opinions expressed by DZone contributors are their own.