Tsonnet #16 - Late binding and Jsonnet inconsistency
Teaching Tsonnet the art of late binding
Welcome to the Tsonnet series!
If you're just joining, you can check out how it all started in the first post of the series.
In the previous post, we added support for local variables:
There's one aspect that I intentionally left out of scope that I want to talk about now: **late binding**.
What is late binding? Paraphrasing Wikipedia:
Late binding is a computer programming mechanism in which the method being called upon an object, or the function being called with arguments, is looked up by name at runtime.
In Tsonnet, even though we lack objects and functions yet, we aim to make the whole language lazily evaluated — a requirement to be compatible with Jsonnet.
The Rationale for Lazy Semantics
In the page Rationale for Lazy Semantics, the design rationale of Jsonnet explains why it adopts the lazy evaluation semantics.
It says:
Therefore, for consistency, the whole language is lazy.
However, Jsonnet is rather inconsistent with late binding.
Let me show you.
Late binding examples
Here's a late binding assignment in Jsonnet:
This works according to the design rationale:
Now, let's go one step back and try a simpler scenario:
It does not work:
Our minds are conditioned to think in a linear fashion, but given lazy semantics, the `local c` binding should be left unevaluated, not causing any error. When `c` is eventually required, `b`, which was not in scope earlier, now is, making `c` valid.
I know it seems confusing if you're coming from a language where the compiler eagerly evaluates definitions from top to bottom. How are we supposed to use something not in scope yet, right!? But in lazy semantics, the compiler defers the evaluation, keeping track of the symbols that refer to the blocks of code (still unevaluated).
Turns out, Tsonnet supports late binding out of the box, given the implementation covered in the previous post:
Array access, not yet:
But only because we haven't implemented index-based access for arrays.
Let's solve this now.
Index-based access for arrays
We need a new `expr` variant type to cover index-based access:
The `string_of_type` will come in handy to output friendlier error messages.
The parser can then make use of `IndexedExpr`:
Let's add a new function to the `Env` module to facilitate the variable lookups:
The memoizing is a bonus to make our implementation faster in future lookups. It's nice when we can get a performance boost almost for free.
Now comes the interpreter.
We were returning arrays as-is before, but to make indexed access work properly, we need to evaluate each entry before returning it:
The identity match can now use `Env.find_var`:
Didn't change much here, but we now have memorized lookups.
And, the `IndexedExpr` to wrap it up:
Since we need to look up the array index value here, `Env.find_var` will look it up, and then we can do all sorts of checks, like index out of bounds, invalid index type, etc.
It looks ugly, I know, sorry! Refactoring will follow, but this should be sufficient to get done with our important feature.
Attentive readers might have noticed the `open Syntax_sugar`. This is a new module to combine function aliases reused across the project:
Let's add a few new sample files to cover some use cases — I'm ignoring the others already presented earlier.
This is to cover arrays' index out of bounds:
This is to cover arrays' invalid index-based access:
Cram tests for the happy paths:
And here are the cram tests for the unhappy paths:
And we are done:
Conclusion
We can write code declaratively in Tsonnet, independent of declaration order. It should keep working consistently, and tests will guarantee that this property still holds in the future.
Eventually, when Tsonnet is ready for practical use, editor tooling such as jump to definition will help with the non-linear declarations. Tools surrounding the language are as important as the language itself.
As mentioned, refactoring will follow to address the ugly `IndexedExpr`. The `interpret` function is becoming bloated the more we introduce new code, making it difficult to read. I will explore some improvements soon.
The diff with all the changes can be seen here.