K8s migration

This example shows conversion of a realistic Kubernetes config to bkl. We'll use bkl best practices and explain the reasoning behind each choice.

Original files

We start with a list of config files like this:

original/
  dev/
    namespace.yaml
    api-service.yaml
    web-service.yaml
  prod/
    namespace.yaml
    api-service.yaml
    web-service.yaml
  staging/
    namespace.yaml
    api-service.yaml
    web-service.yaml

Create prep dir

Make a copy of your original configs into a prep directory for editing and comparison.

$ cp -r original prep

Prep

Transform patterns in the prep configs using bkl features to make them easier to work with.

Convert lists of items to maps with names plus $encode: values. This allows them to be easily referenced by upper layers.

ports: - port: 80 protocol: TCP - port: 443 protocol: TCP
ports: http: port: 80 protocol: TCP https: port: 443 protocol: TCP $encode: values

When the list items need to contain the key (e.g. ports: { name: http }), we can use $encode: values:KEY to reduce duplication.

ports: - name: http port: 80 protocol: TCP - name: https port: 443 protocol: TCP
ports: http: port: 80 protocol: TCP https: port: 443 protocol: TCP $encode: values:name
- resource: name: cpu target: averageUtilization: 70 type: Utilization type: Resource - resource: name: memory target: averageUtilization: 80 type: Utilization type: Resource
cpu: resource: target: averageUtilization: 70 type: Utilization type: Resource memory: resource: target: averageUtilization: 80 type: Utilization type: Resource $encode: values:resource.name

Convert environment variable lists to maps for easier overriding and use $encode: values:KEY:VALUE to transform them back.

env: - name: DB_HOST value: postgres - name: DB_PORT value: "5432"
env: DB_HOST: postgres DB_PORT: "5432" $encode: values:name:value

Lists with valueFrom use $encode: values:KEY instead.

env: - name: ANALYTICS_KEY valueFrom: secretKeyRef: key: analytics-key name: web-secrets - name: API_URL value: https://api.example.com
env: ANALYTICS_KEY: valueFrom: secretKeyRef: key: analytics-key name: web-secrets API_URL: value: https://api.example.com $encode: values:name

Use keys that make sense for the values. Remember that unusual characters are permitted in keys, but sometimes must be quoted.

paths: - backend: service: name: api-service path: /api(/|$)(.*) pathType: Prefix - backend: service: name: auth-service path: /auth(/|$)(.*) pathType: Prefix - backend: service: name: frontend-service path: /static pathType: Prefix
paths: /static: backend: service: name: frontend-service pathType: Prefix /api(/|$)(.*): backend: service: name: api-service pathType: Prefix /auth(/|$)(.*): backend: service: name: auth-service pathType: Prefix $encode: values:path

Convert argument lists to maps with names plus $encode: flags. If there are non-flag initial arguments, move them from args to command. Values are converted to strings automatically, so you don't need to quote them.

args: - --config=/app/config/app.properties - --feature=auth - --feature=metrics - --feature=rate-limiting - --log-format=json - --log-level=2 - --verbose
args: config: /app/config/app.properties feature: - auth - metrics - rate-limiting log-format: json log-level: 2 verbose: "" $encode: flags

Convert embedded JSON/YAML strings to structures with $encode: json-pretty or $encode: yaml.

app.config.json: | { "apiUrl": "http://api-service", "debugMode": true, "environment": "development", "features": { "analytics": false, "caching": false } }
app.config.json: apiUrl: http://api-service debugMode: true environment: development features: analytics: false caching: false $encode: json-pretty
prometheus.yml: | global: evaluation_interval: 15s scrape_interval: 15s
prometheus.yml: global: evaluation_interval: 15s scrape_interval: 15s $encode: yaml

For Java-style properties files, use $encode: properties.

app.properties: | cache.ttl=3600 environment=production logging.format=json logging.level=warn logging.output=stdout
app.properties: cache.ttl: 3600 environment: production logging.format: json logging.level: warn logging.output: stdout $encode: properties

Convert embedded base64 strings to $encode: base64.

database-url: cG9zdGdyZXNxbDovL3Byb2R1c2VyOnByb2RwYXNzQHByb2QtZGItY2x1c3Rlci5yZHMuYW1hem9uYXdzLmNvbTo1NDMyL3Byb2RkYg==
database-url: $value: postgresql://produser:prodpass@prod-db-cluster.rds.amazonaws.com:5432/proddb $encode: base64

Remove duplication between string fields with $"".

labels: environment: prod namespace: api-prod
labels: environment: prod namespace: $"api-{labels.environment}"

Validate

Validate the prepped files against the original configs before continuing.

$ bklc --sort=kind --color original/prod/namespace.yaml prep/prod/namespace.yaml $ bklc --sort=kind --color original/prod/api-service.yaml prep/prod/api-service.yaml $ bklc --sort=kind --color original/prod/web-service.yaml prep/prod/web-service.yaml $ bklc --sort=kind --color original/staging/namespace.yaml prep/staging/namespace.yaml $ bklc --sort=kind --color original/staging/api-service.yaml prep/staging/api-service.yaml $ bklc --sort=kind --color original/staging/web-service.yaml prep/staging/web-service.yaml $ bklc --sort=kind --color original/dev/namespace.yaml prep/dev/namespace.yaml $ bklc --sort=kind --color original/dev/api-service.yaml prep/dev/api-service.yaml $ bklc --sort=kind --color original/dev/web-service.yaml prep/dev/web-service.yaml

Target layout

Plan a file/layer structure for the bkl configs (since bkl uses filenames to determine layer structure).

Keep the original namespace/api-service/web-service structure to match the existing user expectations.

The namespace configs are small and self-contained; keep those independent. Compare the api-service and web-service configs to see if they should have a common base layer.

$ bkli --selector=kind prep/prod/api-service.yaml prep/prod/web-service.yaml

The configs are similar enough to have a common base layer. Use the following bkl config structure:

namespace.yaml
  namespace.staging.yaml
    namespace.staging.dev.yaml
base.yaml
  base.api-service.yaml
    base.api-service.staging.yaml
      base.api-service.staging.dev.yaml
  base.web-service.yaml
    base.web-service.staging.yaml
      base.web-service.staging.dev.yaml

Use the production version of each config as the base layer. This means that all other environments are expressed as differences from production, encouraging minimizing those differences.

Create a directory to put these in.

$ mkdir -p bkl

namespace

Use bkl to sort keys in plain YAML files. Use bkld to produce a diff of two plain YAML or bkl files.

$ bkl prep/prod/namespace.yaml --output=bkl/namespace.yaml $ bkld --selector=kind bkl/namespace.yaml prep/staging/namespace.yaml --output=bkl/namespace.staging.yaml $ bkld --selector=kind bkl/namespace.yaml prep/dev/namespace.yaml --output=bkl/namespace.staging.dev.yaml

base

Use bkli to create a minimal base layer for the services.

$ bkli --selector=kind prep/prod/api-service.yaml prep/prod/web-service.yaml --output=bkl/base.yaml

api-service

Use bkld to produce a diff of two plain YAML or bkl files.

$ bkld --selector=kind bkl/base.yaml prep/prod/api-service.yaml --output=bkl/base.api-service.yaml $ bkld --selector=kind bkl/base.api-service.yaml prep/staging/api-service.yaml --output=bkl/base.api-service.staging.yaml $ bkld --selector=kind bkl/base.api-service.staging.yaml prep/dev/api-service.yaml --output=bkl/base.api-service.staging.dev.yaml

web-service

Use bkld to produce a diff of two plain YAML or bkl files.

$ bkld --selector=kind bkl/base.yaml prep/prod/web-service.yaml --output=bkl/base.web-service.yaml $ bkld --selector=kind bkl/base.web-service.yaml prep/staging/web-service.yaml --output=bkl/base.web-service.staging.yaml $ bkld --selector=kind bkl/base.web-service.staging.yaml prep/dev/web-service.yaml --output=bkl/base.web-service.staging.dev.yaml

Validate

Validate the bkl file layers against the original configs.

$ bklc --sort=kind --color original/prod/namespace.yaml bkl/namespace.yaml $ bklc --sort=kind --color original/staging/namespace.yaml bkl/namespace.staging.yaml $ bklc --sort=kind --color original/dev/namespace.yaml bkl/namespace.staging.dev.yaml $ bklc --sort=kind --color original/prod/api-service.yaml bkl/base.api-service.yaml $ bklc --sort=kind --color original/staging/api-service.yaml bkl/base.api-service.staging.yaml $ bklc --sort=kind --color original/dev/api-service.yaml bkl/base.api-service.staging.dev.yaml $ bklc --sort=kind --color original/prod/web-service.yaml bkl/base.web-service.yaml $ bklc --sort=kind --color original/staging/web-service.yaml bkl/base.web-service.staging.yaml $ bklc --sort=kind --color original/dev/web-service.yaml bkl/base.web-service.staging.dev.yaml

Polish

Apply additional simplifications to the bkl configs.

Merge together common overrides into a single document. When merging things like labels, you may find cases where the values weren't consistently applied before to all objects, but could be.

kind: Service metadata: labels: environment: prod --- kind: Deployment metadata: labels: environment: prod
kind: Service --- kind: Deployment --- $match: {} metadata: labels: environment: prod

If overrides must only apply to a subset of objects, use $matches to select them.

kind: Ingress metadata: labels: app: frontend --- kind: Service metadata: labels: app: frontend --- kind: Secret metadata: labels: app: backend
kind: Ingress --- kind: Service --- kind: Secret --- $matches: - kind: Ingress - kind: Service metadata: labels: app: frontend --- $matches: - kind: Secret metadata: labels: app: backend

Validate

Validate the polished bkl file layers against the original configs.

$ bklc --sort=kind --color original/prod/namespace.yaml bkl/namespace.yaml $ bklc --sort=kind --color original/staging/namespace.yaml bkl/namespace.staging.yaml $ bklc --sort=kind --color original/dev/namespace.yaml bkl/namespace.staging.dev.yaml $ bklc --sort=kind --color original/prod/api-service.yaml bkl/base.api-service.yaml $ bklc --sort=kind --color original/staging/api-service.yaml bkl/base.api-service.staging.yaml $ bklc --sort=kind --color original/dev/api-service.yaml bkl/base.api-service.staging.dev.yaml $ bklc --sort=kind --color original/prod/web-service.yaml bkl/base.web-service.yaml $ bklc --sort=kind --color original/staging/web-service.yaml bkl/base.web-service.staging.yaml $ bklc --sort=kind --color original/dev/web-service.yaml bkl/base.web-service.staging.dev.yaml