BKL

Example

bkl knows that service.test.toml inherits from service.yaml by the filename pattern (override with $parent) and uses filename extensions to determine formats.

Install

You can also download binaries directly here.

Formats

Output defaults to the same format as the specified input file. Use -f (or --format) to override.

jsonl is an alias for json to fully support JSON Lines.

You can also set the output format implicitly by using a different file extension.

Write

Output goes to stdout by default. Errors always go to stderr.

Output format is autodetected from output filename (unless specified with -f).

Inputs

Specifying multiple input files merges all layers in order.

Specifying an input file with the base name - and a valid format extension causes bkl to read that format from standard input. Use -- before the filename to avoid it being treated as a flag.

Inheritance

Inheritance is determined using filenames by default. After stripping the extension, the remaining filename is split on . and treated as an inheritance hierarchy (e.g. a.b.c.yaml inherits from a.b.<ext> inherits from a.<ext>). Parent layers may have any supported file extension.

bkl will check for all supported endings of a manually-specified parent file and will still evaluate layers under the parent in the normal order.

Setting $parent to a list or wildcard allows inheriting from multiple files. All parent files are loaded before the child.

* does not match .. a.* matches a.b.yaml but not a.b.c.yaml.

Layer order can be specified with commandline argument order. The -P flag (or --skip-parent) tells bkl to not load parent layers using filenames or $parent.

Setting $parent to false or null stops any further inheritance regardless of filename structure.

Streams

Streams package multiple documents into a single file. YAML/TOML streams are delimited with --- or sometimes +++. JSON streams are concatenated or delimited with newlines (see JSON Lines and ndjson).

To layer streams, bkl has to match documents between layers. By default, child documents are applied to all parent documents. $match allows applying to specific parent documents.

The supplied pattern can match multiple documents and the updates will be applied to all of them. $match: {} matches any documents with a map as their root element, regardless of parent.

$invert: true inside a $match block inverts the match, causing it to apply to any documents that do not contain the match criteria.

$match: null forces the updates to apply to a new document.

Maps

Maps are merged by default. To change that, use $replace: true or remove individual entries with $delete.

bkl returns an error if you use $delete, $replace: true, or a key: value pair when they don't override a value from a lower layer. This helps keep upper layers minimal.

Lists

Lists are merged by default. To change that, use $replace: true or remove individual entries with $delete.

bkl returns an error if you use $replace: true or $delete when they don't override a value from a lower layer. This helps keep upper layers minimal.

To update a specific list item from a parent layer, use $match. Combine it with $value for scalars.

$""

Use $"" to interpolate format strings with $merge-style references.

$decode

$decode transforms a string into a tree by parsing standard formats.

$env

Use $env: to substitute environment variables. This is supported in keys and values.

Note that all $env: substitutions result in string values even if the substituted value is true, false, null, or all digits.

$encode

$encode transforms the subtree into the specified format.

$required

Use $required in lower layers to force upper layers to replace the value.

$merge

Use $merge to merge the contents one subtree or scalar value into another.

You can also merge across documents using $match and optionally $path:

$merge also supports a shorthand syntax equivalent to $match and $path:

$replace

Use $replace to merge the contents of one subtree or scalar value with another.

Note that $merge and $replace are equivalent for scalar values.

You can also replace across documents using $match and optionally $path:

$replace also supports a shorthand syntax equivalent to $match and $path:

$output

Use $output: true to select a subtree for output instead of the document root.

Multiple instances of $output: true in a document will generate multiple output documents (delimited with ---).

Use $output: false to omit a subtree or entire document from the output.

Combine $output with $replace or $merge to have hidden "template" subtrees that don't appear in the output but can be copied in as needed.

$repeat

$repeat: at the top level of a document causes the document to be duplicated the given number of times. Within the document, $repeat can be used to reference the zero-based repeat index.

$repeat: can also contain a map of named repeat iterators to repeat count. The result is the cartesian product (all combinations) of repeat values.

$repeat: can contain a list of values to iterate over:

$repeat: supports range parameters for numeric sequences:

Range parameters require exactly 2 of $first, $last, and $count. Optional $step defaults to 1.

$repeat: also works at the object level within maps and lists:

Object-level repeat in lists:

$$

$$ translates to a literal $ in cases where it would otherwise trigger special handling.

TOML

TOML doesn't allow unquoted $ in keys, which impacts bkl directives. Work around this by quoting keys:

bklb

bklb is a wrapper for CLI programs that take configuration files as commandline arguments but do not support bkl format. It transparently merges layers, translates formats, writes to temporary files, alters the commandline arguments, then execs the wrapped program.

Note that the files mentioned don't have to exist; bklb will search for files with the same root name but different extensions, then merge layers and translate into the specified format.

bklb takes the name of the program it wraps from its own filename, hence the ln -s symlink creation in the example above. It trims at most one b from the end of its name before searching for the wrapped program so they can coexist in your PATH.

bkld

bkld (d for "diff") generates the minimal intermediate layer needed to create the target output from the base layer. Along with bkli, it automates splitting existing configurations into layers.

bkli

bkli (i for "intersect") generates the maximal base layer that the specified targets have in common. Along with bkld, it automates splitting existing configurations into layers.

Any fields that exist in all upper layers but have different values will be marked $required.

bklr

bklr (r for "required") generates a document containing just the required fields and their ancestors from the lower layer. The output can be edited into a minimal upper layer.

kubectl bkl

kubectl bkl is a kubectl plugin that wraps all the normal kubectl commands by evaluating any input files as bkl layers and passing the output to kubectl.

Docker

bkl docker images are available at ghcr.io/gopatchy/bkl:latest for amd64 and arm64. Compose these images for your development and CI/CD workflows.

diff

It's easy to see differences between processed bkl files using shell syntax.

Migrate

Below is an example process for migrating an existing set of configurations to bkl. It contains some Kubernetes-specific items but the use of the bkl* tools applies to any configuration source.

We assume you start with two K8s deployments called deploy-dev and deploy-prod which are similar but not identical.

First, install tools:

• bkl
• kubectl
• krew
• neat

It's possible to export configuration directly from existing templating systems (e.g. kustomize build, helm install --dry-run --debug). Instead, we do this in a generic way and make sure we get the latest configuration by fetching it directly from the K8s API server.

You now have 3 bkl files:

• deploy.yaml is the base layer containing values common to both deployments
• deploy.dev.yaml is the upper layer containing values specific to deploy-dev
• deploy.prod.yaml is the upper layer containing values specific to deploy-prod

The migration process isn't deterministic; there are design and aesthetic considerations. Here are some general tips:

• Start with the most similar examples to maximize base layer size
• Consider intermediate layers that make sense (e.g. deploy.yaml → deploy.frontend.yaml → deploy.frontend.prod.yaml)
• Split large configurations into logical subfiles; bkl can merge across file boundaries
• Iterate by diffing your evaluated layers against production (e.g. kubectl bkl diff -f), making small changes, then diffing again.
• Migrate as much as possible to the lowest layer to reduce duplication and complexity.
• Consider making your base layer match your production configuration then overriding values for dev/test configurations. This makes it very clear where you're drifting away from production.
• Remove duplication with $merge: but avoid chained merge paths.
• Avoid using hidden $output: false trees as template variables; prefer overriding values in-place.
• bkld doesn't know how to match entries within lists, so it may remove and replace large entries (e.g. K8s containers) that could be trivially patched. Use $match: to select the container and override values within it.

Comparison

bkl has some overlap with other configuration templating tools (e.g. Helm, Kustomize, Hiera). We believe that bkl has a combination of project goals that aren't fully served by any of the alternatives:

• Configuration and templating/layering should be generic, not service-specific.
• Configuration and templating/layering should be separate from deployment tooling.
• Configuration files should be written in standard formats (JSON, YAML, TOML) and parseable by their standard parsers.
• Basic functionality should be available without learning custom syntax.
• Non-templatized configuration (e.g. StackOverflow answers) should be usable without modification.
• File composition should be accomplished without meta configuration (e.g. manifest files).
• Templating/layering behavior should be intuitive and produce expected results.
• Documentation and examples should be excellent.