Tsonnet #24 - Getting to the root of things: top-level object access
Tsonnet now supports top-level object access with the $ operator, allowing you to reference root-level fields from anywhere in nested structures.
Welcome to the Tsonnet series!
If you’re not following the series so far, you can check out how it all started in the first post of the series.
In the previous post, I implemented object self-referencing using the self keyword:
Now it’s time to tackle another reference mechanism -- reaching the outermost object from anywhere in your nested structure.
When self isn’t enough
The self keyword is great when you want to reference fields in the current object:
{
answer: 42,
answer_to_the_ultimate_question: self.answer
}
But what happens when you’re nested several levels deep and need to reference something at the root?
{
answer: {
value: 42
},
answer_to_the_ultimate_question: {
value: self.answer
}
}
Wait, that won’t work! The self.answer reference is not valid inside answer_to_the_ultimate_question because this object has no answer field.
This is where Jsonnet’s $ operator comes in -- it always refers to the outermost object, no matter how deep you are in the nesting:
{
answer: {
value: 42
},
answer_to_the_ultimate_question: $.answer.value
}
Let’s teach Tsonnet about the dollar sign.
A quick detour: skipping type checks
Before diving into the main feature, I added a small improvement -- a flag to skip type checking during development:
The implementation is straightforward -- when skip_typecheck is true, we bypass the type checking step entirely:
This is handy when you’re iterating quickly and just want to see what the interpreter does, though you’ll get a warning about potential runtime errors. I intentionally didn’t add a shorter version of this parameter to avoid misuse, as this is intended for development only.
Now back to the main event!
Teaching $ to the compiler
First, we need to recognize the $ token:
Then we extend the parser to handle $ in field access expressions. Notice how we support both dot notation and bracket notation:
The new obj_field_access rule captures all four combinations: self.field, self[’field’], $.field, and $[’field’].
Distinguishing object scopes
Now we need to distinguish between self-references and top-level references in the AST. Enter the object_scope type:
The ObjectFieldAccess now carries an object_scope to tell us whether we’re looking at self.field or $.field.
We also need a helper to convert the scope to a string for error messages:
Keeping $ in its place
Just like with self, we need to ensure $ isn’t used outside of objects. We extend the scope validator to handle both scopes:
The key change is in validate_object_field_access -- we now check the scope and provide the appropriate error message:
Let’s see it in action:
$ dune exec -- tsonnet samples/errors/object_outer_most_ref_out_of_scope.jsonnet
samples/errors/object_outer_most_ref_out_of_scope.jsonnet:2:13 No top-level object found
2: local _two = $.one + 1;
^^^^^^^^^^^^^^^^^^^^^^^
So far, so good!
Environment: the subtle magic of add_local_when_not_present
Here’s where things get interesting. We need to add $ to the environment, but only once -- at the outermost object. Nested objects should keep the same $ reference pointing to the root.
First, we add a helper function to the environment module:
This function only adds the binding if it doesn’t already exist -- crucial for preserving the top-level $ reference in nested objects.
The key to making $ work correctly is the add_local_when_not_present function. Why do we need it?
Consider this nested structure:
{
root_value: 1,
nested: {
self_value: 2,
uses_root: $.root_value,
uses_self: self.self_value
}
}When we enter the outer object, we add both “self” and “$” to the environment, both pointing to the outer object’s ID (let’s say EnvId 1).
When we enter the nested object, we want:
“self”to point to the nested object’s ID (EnvId 2)“$”to still point to the outer object’s ID (EnvId 1)
By using add_local for “self”, we shadow the outer self. But by using add_local_when_not_present for “$”, we preserve the original top-level reference. It’s only added to the environment if it doesn’t exist yet -- which means only the outermost object sets it.
Neat, right?
Type checking: translating $ references
The type checker needs to handle both self and $. When translating objects, we add $ using add_local_when_not_present:
See what happened there? We add self normally (which shadows any outer self), but we only add $ if it’s not already present. This means the outermost object’s ID gets locked in as the $ reference for all nested objects.
For field access translation, we use the scope to look up the right reference:
We also need to update cycle detection to handle the scope:
Interpretation: making $ work at runtime
The interpreter follows the same pattern as the type checker. When we enter an object, we add both self and $:
And when accessing fields, we look up the appropriate scope reference:
Does it actually work?
Let’s verify our implementation with some tests. First, the identifier notation:
// samples/objects/toplevel_reference.jsonnet
{
one: 1,
two: $.one + 1
}
$ dune exec -- tsonnet samples/objects/toplevel_reference.jsonnet
{ “one”: 1, “two”: 2 }
Then, the bracket notation:
// samples/objects/toplevel_bracket_lookup.jsonnet
{
answer: 42,
answer_to_the_ultimate_question: $[’answer’]
}
$ dune exec -- tsonnet samples/objects/toplevel_bracket_lookup.jsonnet
{ “answer”: 42, “answer_to_the_ultimate_question”: 42 }
Beautiful! And what about using $ outside of objects?
// samples/errors/object_outer_most_ref_out_of_scope.jsonnet
local _one = 1;
local _two = $.one + 1;
{
one: _one,
two: _two
}$ dune exec -- tsonnet samples/errors/object_outer_most_ref_out_of_scope.jsonnet
samples/errors/object_outer_most_ref_out_of_scope.jsonnet:2:13 No top-level object found
2: local _two = $.one + 1;
^^^^^^^^^^^^^^^^^^^^^^^
Exactly what we want!
Catching infinite loops and testing
Just like with self, we need to catch cycles involving $:
// samples/semantics/invalid_binding_cycle_outer_object_fields.jsonnet
{
a: $.b,
b: $.a,
}$ dune exec -- tsonnet samples/semantics/invalid_binding_cycle_outer_object_fields.jsonnet
samples/semantics/invalid_binding_cycle_outer_object_fields.jsonnet:3:7 Cyclic reference found for 1->a
3: b: $.a,
^^^^^^^^^^^The cycle detection works perfectly -- it catches the circular dependency at type-check time, before the interpreter ever runs.
The cram tests capture all of these scenarios:
Conclusion
We can now reference root values from anywhere in our object tree -- essential for real-world configuration files where you want to define common values once at the top level and reuse them throughout your structure.
Here is the entire diff.
The journey of implementing $ was a great exercise. It lets us maintain different scoping rules for different references without complicating the core environment logic.
Objects are yet too static, and we don’t have field chain access. We’ll tackle it next.