Diffs

A diff compares a configuration with the state its last apply saved, and reports what applying that configuration would do. Running one does nothing: no entity is created, changed or destroyed, and no state is saved. Each entity that is still configured is checked through its provider’s read and change checks, as an apply would do, so a diff also shows changes made to the real infrastructure outside xcl.

Running a diff

Config.Diff takes the same paths as Config.Apply and fails in the same cases, such as a configuration that does not parse or validate:

import "github.com/jumppad-labs/xcl/highlight"

d, err := c.Diff([]string{"./config"})
if err != nil {
  return err
}

if d.Changed() == 0 {
  fmt.Println("nothing to do")
  return nil
}

os.Stdout.Write(highlight.Text.Render(d.Pieces()))

d = cfg.diff("./config")

if d.changed() == 0:
    print("nothing to do")
    return

s = d.summary
print(f"{s.create} to create, {s.update} to update, {s.replace} to replace, {s.delete} to delete")
for resource in d.resources:
    print(resource.address, resource.action)

The result is plain data. d.Summary counts the entities for each action and the ones that would stay unchanged, and d.Changed() adds up the four actions. d.Resources lists every entity an apply would change, sorted by address, and each one carries its Changes, one for each configured value that would change:

for _, resource := range d.Resources {
  fmt.Println(resource.Address, resource.Action)

  for _, change := range resource.Changes {
    fmt.Println("  ", change.Path, change.Before, change.After)
  }
}

for resource in d.resources:
    print(resource.address, resource.action)

    for change in resource.changes:
        print("  ", change.path, change.before, change.after_value)

A change’s Path locates the value within the entity, such as image, ports[0].host or env["LOG_LEVEL"]. Before is the saved value and After the configured one. An added value has no Before and a removed one has no After.

What each action means

Each entity an apply would change gets exactly one action:

An entity is replaced for one of three reasons, and the diff records which in the entity’s Reason:

A provider decides between leaving an entity alone, updating it and replacing it by comparing the saved and configured values, and it is told which of the entities it depends on an apply will update or replace. See Unchanged, update or replace for how providers make that decision.

An entity whose provider reports no change is unchanged. It is counted in the summary but not listed. Only entities an apply hands to a provider take part: variables, outputs, modules and disabled blocks are neither listed nor counted.

An update can have no changes of its own. Most often that is because an entity it depends on is updated or replaced, and its provider answered update for it, such as a container reattached to a replaced network. Its Dependencies then lists those entities, sorted by address, each with its Address and Action, and the rendering names them. Only an update with neither changes nor dependencies is one where the provider reports that the real entity differs from what was saved, but no configured value changed: something changed it outside xcl, and an apply would update it.

Known after apply

Some values only exist once an apply has run, such as an id or an address a provider assigns when it creates an entity. A value that depends on one is marked Unknown, with no After. An entity that is already saved and depends on such a value is reported as an update, since whether an apply would really change it cannot be known until then.

A diff assumes that updating an entity can change the values its provider fills in. So an entity that refers to a value of an entity being updated is also reported as an update, with that value known after apply, just as it would be for an entity being created.

Sensitive values

A change to a sensitive value is reported with Sensitive set and with neither its old nor its new value, so the secret never reaches the result, its JSON, or the rendering. The change is still listed, because knowing that a secret changed is useful in itself.

Pass diff.RevealSensitive() to include both values:

d, err := c.Diff([]string{"./config"}, diff.RevealSensitive())

d = cfg.diff("./config", reveal_sensitive=True)

The result then holds the secrets in plain form, and so does anything made from it, so print or store it with the same care as the secrets themselves. See Sensitive values for marking a field sensitive.

Reading the rendered output

d.Pieces() lays a result out as text in the style of a git diff, and highlight.Text.Render(d.Pieces()) gives that text plain, ready to print or log. This result has an entity for each action, and a replacement for each reason:

  # docker.container.api will be updated
  ~ docker "container" "api" {
      ~ ports[0].host      = 8080 -> 9090
      + ports[2]           = { host = 443, local = 8443 }
      ~ env["DB_PASSWORD"] = (sensitive value)
    }

  # docker.container.cache changed outside xcl and will be updated
  ~ docker "container" "cache" {}

  # docker.container.proxy will be updated because docker.network.app is replaced
  ~ docker "container" "proxy" {}

  # docker.container.web will be created
  + docker "container" "web" {
      + image   = "nginx"
      + db_host = (known after apply)
    }

  # docker.container.worker will be replaced because docker.network.app is replaced
-/+ docker "container" "worker" {}

  # docker.network.app will be replaced, it cannot be updated in place
-/+ docker "network" "app" {
      ~ subnet = "10.42.0.0/24" -> "10.42.0.0/23"
    }

  # docker.network.backup will be replaced, its last apply failed
-/+ docker "network" "backup" {}

  # database.postgres.old will be deleted
  - database "postgres" "old" {}

Diff: 1 to create, 3 to update, 3 to replace, 1 to delete, 4 unchanged.

Each entity starts with a comment saying what an apply would do with it, followed by its block header, marked with the action. For a replacement the comment also says why: its last apply failed, it cannot be updated in place, or the entities it depends on that are replaced, named one by one, as for docker.container.worker above. An update with no changes of its own names the dependencies behind it in the same way, as for docker.container.proxy, or A, B are replaced and C is updated when there are several. Only an update with neither changes nor dependencies says it changed outside xcl and will be updated, as for docker.container.cache. Inside the block there is one line for each change, with the = signs lined up. Entities appear in the result’s order, which is sorted by address. An entity with no changes is written as a header with an empty body.

A value only known once an apply has run is written as (known after apply). When it replaces a saved value, the line shows both, as "old" -> (known after apply). A changed sensitive value is written as (sensitive value). Its values appear only when the result holds them, which happens only when the diff ran with diff.RevealSensitive().

The last line sums up the diff. When nothing would change, it is the only line:

Diff: no changes, 9 unchanged.

Colour

The pieces are labelled, so colour is only a matter of the renderer. Render them with the same renderer you use to highlight configuration text:

import "github.com/jumppad-labs/xcl/highlight"

os.Stdout.Write(highlight.ANSI.Render(d.Pieces()))

Added lines use the theme’s inserted colour, removed ones its deleted colour, and changed and replaced ones its changed colour. Block headers and values are coloured as configuration. With no theme, the terminal renderer uses the terminal’s standard green, red and yellow. Given a VS Code colour theme with highlight.NewANSI(highlight.WithThemeFile(...)), it uses the colours that theme gives diffs. A theme that sets no diff colours leaves the markers in the terminal’s default colour.

xcl never checks whether output is a terminal, so whether to ask for colour is your decision. Removing the colour codes gives back exactly the plain rendering, and sensitive values are hidden the same way in both.

A renderer of your own sees the markers and paths labelled highlight.ScopeInserted, highlight.ScopeDeleted and highlight.ScopeChanged, comment lines labelled as comments, and headers and values labelled as configuration text. Every other piece, such as indentation, placeholders and the summary line, has an empty scope, so a renderer that escapes text sees every byte.

The JSON form

The result is also the machine-readable form of a diff. json.Marshal(d) writes the summary and the entities with their changes, each path in its written form, such as "ports[0].host". Values that are absent, such as the values of a sensitive change that was not revealed, are left out. A replacement carries its reason, and one replaced because of its dependencies also lists them in replaced_dependencies. Both are left out for every other action. An update lists the dependencies behind it in dependencies, each with its address and action, left out when there are none:

{
  "summary": { "create": 1, "update": 1, "replace": 2, "delete": 0, "unchanged": 4 },
  "resources": [
    { "address": "docker.container.proxy", "action": "update",
      "dependencies": [ { "address": "docker.network.app",
                          "action": "replace" } ] },
    { "address": "docker.container.web", "action": "create",
      "changes": [ { "path": "image", "after": "nginx" },
                   { "path": "db_host", "unknown": true } ] },
    { "address": "docker.container.worker", "action": "replace",
      "reason": "dependency",
      "replaced_dependencies": [ "docker.network.app" ] },
    { "address": "docker.network.app", "action": "replace",
      "reason": "provider",
      "changes": [ { "path": "subnet", "before": "10.42.0.0/24",
                     "after": "10.42.0.0/23" } ] }
  ]
}

In Python, Diff.raw holds the same JSON document as a dict.