template Block
Placement | job -> group -> task -> template |
The template
block instantiates an instance of a template renderer. This
creates a convenient way to ship configuration files that are populated from
environment variables, Consul data, Vault secrets, or just general
configurations within a Nomad task.
job "docs" {
group "example" {
task "server" {
template {
source = "local/redis.conf.tpl"
destination = "local/redis.conf"
change_mode = "signal"
change_signal = "SIGINT"
}
}
}
}
Nomad utilizes Go template and a tool called Consul Template, which adds a set of new functions that can be used to retrieve data from Consul and Vault. Nomad templates can reference Nomad's runtime environment variables, node attributes and metadata, Nomad service registrations, and Nomad variables. Templates can also be used to provide environment variables to your workload.
For a full list of the API template functions, please refer to the Consul Template documentation. For a an introduction to Go templates, please refer to the Learn Go Template Syntax guide.
template
Parameters
change_mode
(string: "restart")
- Specifies the behavior Nomad should take if the rendered template changes. Nomad will always write the new contents of the template to the specified destination. The following possible values describe Nomad's action after writing the template to disk.change_signal
(string: "")
- Specifies the signal to send to the task as a string like"SIGUSR1"
or"SIGINT"
. This option is required if thechange_mode
issignal
.change_script
(
- Configures the script triggered on template change. This option is required if theChangeScript
: nil)change_mode
isscript
.data
(string: "")
- Specifies the raw template to execute. One ofsource
ordata
must be specified, but not both. This is useful for smaller templates, but we recommend usingsource
for larger templates.destination
(string: <required>)
- Specifies the location where the resulting template should be rendered, relative to the task working directory. Only drivers without filesystem isolation (ex.raw_exec
) or that build a chroot in the task working directory (ex.exec
) can render templates outside of theNOMAD_ALLOC_DIR
,NOMAD_TASK_DIR
, orNOMAD_SECRETS_DIR
. For more details on howdestination
interacts with task drivers, see the Filesystem internals documentation. Note that where possible, theNOMAD_SECRETS_DIR
is mountednoexec
, so rendered templates can't be used as self-executing scripts.env
(bool: false)
- Specifies the template should be read back in as environment variables for the task (example). To update the environment on changes, you must setchange_mode
torestart
. Settingenv
when thechange_mode
issignal
will return a validation error. Settingenv
when thechange_mode
isnoop
is permitted but will not update the environment variables in the task.error_on_missing_key
(bool: false)
- Specifies how the template behaves when attempting to index a map key that does not exist in the map.When
true
, the template engine will return an error, which will cause the task to fail.When
false
, the template engine will do nothing and continue executing the template. If printed, the result of the index operation is the string "<no value>".
left_delimiter
(string: "{{")
- Specifies the left delimiter to use in the template. The default is "{{" for some templates, it may be easier to use a different delimiter that does not conflict with the output file itself.perms
(string: "644")
- Specifies the rendered template's permissions. File permissions are given as octal of the Unix file permissionsrwxrwxrwx
.uid
(int: nil)
- Specifies the rendered template owner's user ID. If negative or not specified (nil
) the ID of the Nomad agent user will be used.Caveat: Works only on Unix-based systems. Be careful when using containerized drivers, such as
docker
orpodman
, as groups and users inside the container may have different IDs than on the host system. This feature will also not work with Docker Desktop.gid
(int: nil)
- Specifies the rendered template owner's group ID. If negative or not specified (nil
) the ID of the Nomad agent group will be used.Caveat: Works only on Unix-based systems. Be careful when using containerized drivers, such as
docker
orpodman
, as groups and users inside the container may have different IDs than on the host system. This feature will also not work with Docker Desktop.right_delimiter
(string: "}}")
- Specifies the right delimiter to use in the template. The default is "}}" for some templates, it may be easier to use a different delimiter that does not conflict with the output file itself.source
(string: "")
- Specifies the path to the template to be rendered. One ofsource
ordata
must be specified, but not both. This source can be fetched using anartifact
resource. The template must exist in the task working directory prior to starting the task; it is not possible to reference a template whose source is inside a Docker container, for example.splay
(string: "5s")
- Specifies a random amount of time to wait between 0 ms and the given splay value before invoking the change mode. This is specified using a label suffix like "30s" or "1h", and is often used to prevent a thundering herd problem where all task instances restart at the same time.wait
(Code: nil)
- Defines the minimum and maximum amount of time to wait for the Consul cluster to reach a consistent state before rendering a template. This is useful to enable in systems where network connectivity to Consul is degraded, because it will reduce the number of times a template is rendered. This setting can be overridden by theclient.template.wait_bounds
. If the template configuration has amin
lower thanclient.template.wait_bounds.min
or amax
greater thanclient.template.wait_bounds.max
, the client's bounds will be enforced, and the templatewait
will be adjusted before being sent to the template engine.wait { min = "5s" max = "10s" }
vault_grace
(string: "15s")
- Deprecated
template
Examples
The following examples only show the template
blocks. Remember that the
template
block is only valid in the placements listed above.
Inline Template
This example uses an inline template to render a file to disk. This file watches various keys in Consul for changes:
template {
data = "---\nkey: {{ key \"service/my-key\" }}"
destination = "local/file.yml"
}
It is also possible to use heredocs for multi-line templates, like:
template {
data = <<EOH
---
bind_port: {{ env "NOMAD_PORT_db" }}
scratch_dir: {{ env "NOMAD_TASK_DIR" }}
node_id: {{ env "node.unique.id" }}
service_key: {{ key "service/my-key" }}
EOH
destination = "local/file.yml"
}
Remote Template
This example uses an artifact
block to download an input template
before passing it to the template engine:
artifact {
source = "https://example.com/file.yml.tpl"
destination = "local/file.yml.tpl"
}
template {
source = "local/file.yml.tpl"
destination = "local/file.yml"
}
Task meta
values
To render values from a task's meta
config, use the environment variable form
of the meta variable name.
meta {
mykey = "some_value"
}
template {
data = <<EOH
{{ env "NOMAD_META_mykey" }}
EOH
}
Node Variables
Use the env
function to access the Node's attributes and metadata inside a
template. Note the meta.
syntax here applies only to node meta fields.
template {
data = <<EOH
---
node_dc: {{ env "node.datacenter" }}
node_cores: {{ env "attr.cpu.numcores" }}
meta_key: {{ env "meta.node_meta_key" }}
EOH
destination = "local/file.yml"
}
Environment Variables
Templates may be used to create environment variables for tasks. These templates
work exactly like other templates except once the templates are written, they
are parsed as KEY=value
pairs. Those key value pairs are included in the
task's environment.
For example the following template block:
template {
data = <<EOH
# Lines starting with a # are ignored
# Empty lines are also ignored
LOG_LEVEL="{{key "service/geo-api/log-verbosity"}}"
API_KEY="{{with secret "secret/geo-api-key"}}{{.Data.value}}{{end}}"
EOH
destination = "secrets/file.env"
env = true
}
The task's environment would then have environment variables like the following:
LOG_LEVEL=DEBUG
API_KEY=12345678-1234-1234-1234-1234-123456789abc
This allows 12factor app style environment variable based configuration while keeping all the familiar features and semantics of Nomad templates.
Secrets or certificates may contain a wide variety of characters such as newlines, quotes, and backslashes which may be difficult to quote or escape properly.
Whenever a templated variable may include special characters, use the toJSON
function to ensure special characters are properly parsed by Nomad.
CERT_PEM={{ file "path/to/cert.pem" | toJSON }}
The parser will read the JSON string, so the $CERT_PEM
environment variable
will be identical to the contents of the file.
Likewise, when evaluating a password that may contain quotes or #
, use the
toJSON
function to ensure Nomad passes the password to the task unchanged.
# Passwords may contain any character including special characters like:
# \"'#
# Use toJSON to ensure Nomad passes them to the environment unchanged.
{{ with secret "secrets/data/application/backend" }}
DB_PASSWD={{ .Data.data.DB_PASSWD | toJSON }}
{{ end }}
For more details see go-envparser's README.
Template Destinations
Templates are rendered into the task working directory. Drivers without
filesystem isolation (such as raw_exec
) or drivers that build a chroot in
the task working directory (such as exec
) can have templates rendered to
arbitrary paths in the task. But task drivers such as docker
can only access
templates rendered into the NOMAD_ALLOC_DIR
, NOMAD_TASK_DIR
, or
NOMAD_SECRETS_DIR
. To work around this restriction, you can create a mount
from the template destination
to another location in the task.
task "task" {
driver = "docker"
config {
image = "redis:6.0"
mount {
type = "bind"
source = "local"
target = "/etc/redis.d"
}
}
template {
destination = "local/redis.conf"
}
}
Dependencies
For templates that read from Vault, Consul, or Nomad, each item read is called a
"dependency". All the template
blocks share the same internal runner which
de-duplicates dependencies requesting the same item. You should avoid having
large numbers of dependencies for a given task, as each dependency requires at
least one concurrent request (a possibly blocking query) to the upstream
server. If a task has more than 128 dependencies, a warn-level log will appear
in the Nomad client logs which reports "watching this many dependencies could
DDoS your servers", referring to the Vault, Consul, or Nomad cluster being
queried.
Nomad Integration
Nomad Services
Nomad service registrations can be queried using the nomadService
and
nomadServices
functions. The requests are tied to the same namespace as the
job which contains the template block.
template {
data = <<EOF
# Configuration for a single NGINX upstream service.
upstream my_app {
{{- range nomadService "my-app" }}
server {{ .Address }}:{{ .Port }};{{- end }}
}
# Configuration for all services registered in Nomad as an NGINX upstream
# service.
{{ range nomadServices }}
# Configuration for service {{ .Name }}.
upstream {{ .Name | toLower }} {
{{- range nomadService .Name }}
server {{ .Address}}:{{ .Port }};{{- end }}
}
{{ end -}}
EOF
destination = "local/nginx.conf"
}
Simple Load Balancing with Nomad Services
The nomadService
function now supports simple load balancing by selecting
instances of a service via rendezvous hashing.
To enable simple load balancing, the nomadService
function requires 3 arguments.
- The number of services to select
- The hashing key (should be unique, but consistent per requester)
- The service name
By using NOMAD_ALLOC_ID
as the hashing key, the selected instances will remain
mostly stable for the allocation. Each time the template is run, nomadService
will return the same set of instances for each allocation - unless N instances of
the service are added or removed, in which case there is a 1/N chance of a selected
instance being replaced. This helps maintain a more consistent output when rendering
configuration files, triggering fewer restarts and signaling of Nomad tasks.
template {
data = <<EOH
# Configuration for 1 redis instances, as assigned via rendezvous hashing.
{{$allocID := env "NOMAD_ALLOC_ID" -}}
{{range nomadService 1 $allocID "redis"}}
server {{ .Address }}:{{ .Port }};
{{- end}}
EOH
}
Nomad Variables
Warning
Using the Nomad integration results in an implicit dependency on the Nomad
servers. If the Nomad servers are unavailable (for example, a client is
disconnected), then after some retries, running allocations will be
stopped. Refer to the template.nomad_retry
client configuration option for
more details and to adjust the retry behaviour.
Nomad variables can be queried using the nomadVar
, nomadVarList
,
nomadVarListSafe
, and nomadVarExists
functions.
nomadVarList
and nomadVarListSafe
These functions can be used to list the paths of Nomad variables available to the task based on its workload identity. You can provide an optional prefix to filter the path list by as a parameter.
The list functions return a slice of NomadVarMeta
type:
type NomadVarMeta struct {
Namespace, Path string
CreateIndex, ModifyIndex uint64
CreateTime, ModifyTime nanoTime
}
The nanoTime
type contains Unix nanoseconds since the epoch. Its string method
prints it out as a formatted date/time value. It also has a Time
method that
can be used to retrieve a Go time.Time
for further manipulation.
NomadVarMeta
objects print their Path
value when used as a string. For
example, these two template blocks produce identical output.
template {
data = <<EOH
{{ range nomadVarList }}
{{ . }}
{{ end }}
EOH
}
template {
data = <<EOH
{{ range nomadVarList }}
{{ .Path }}
{{ end }}
EOH
}
You can provide a prefix to filter the path list as an optional parameter to the
nomadVarList
function.
template {
data = <<EOH
{{ range nomadVarList "path/to/filter" }}
{{ . }}
{{ end }}
EOH
}
By default, the nomadVarList
will list variables in the same namespace as the
task. The path filter can change the namespace by adding a suffix separated by
the @
character:
template {
data = <<EOH
{{ range nomadVarList "path/to/filter@example_namespace" }}
{{ . }}
{{ end }}
EOH
}
The nomadVarListSafe
function works identically to nomadVarList
, but refuses
to render the template if the variable list query returns blank/empty data.
nomadVar
These functions can be used to a read Nomad variable, assuming the task has
access rights to the variable based on the task's workload identity. If the
path does not exist or the caller does not have access to it, the template
renderer will block until the path is available. To avoid blocking, wrap
nomadVar
calls with nomadVarExists
.
The nomadVar
function returns a map of string
to NomadVarItems
structs. Each member of the map corresponds to a member of the variable's
Items
collection.
For example, given a variable exists at the path nomad/jobs/redis
, with a
single key/value pair in its Items collection—maxconns
:15
template {
data = <<EOH
{{ with nomadVar "nomad/jobs/redis" }}{{ .maxconns }}{{ end }}
EOH
}
renders
15
Each map value also contains helper methods to get the variable metadata and a link to the parent variable:
Keys
: produces a sorted list of keys to thisNomadVarItems
map.Values
: produces a key-sorted list.Tuples
: produces a key-sorted list of K,V tuple structs, with accessors.K
and.V
for the key and value respectively.Metadata
: returns this collection's parent metadata as aNomadVarMeta
Parent
: returns a parent object that has a Metadata field referring to theNomadVarMeta
and an Items field that refers to thisNomadVarItems
object.
For example, given a variable exists at the path nomad/jobs/redis
, you could
render some of its metadata as follows:
template {
data = <<EOH
{{ with nomadVar "nomad/jobs/redis" }}
Path: {{ .Metadata.Path }}
Namespace: {{ .Metadata.Namespace }}
CreateTime: {{ .Metadata.CreateTime }}
ModifyTime: {{ .Metadata.ModifyTime }}
{{ end }}
EOH
}
By default, the nomadVar
function reads a variable in the same namespace as
the task. The path filter can change the namespace by adding a suffix separated
by the @
character.
template {
data = <<EOH
{{ with nomadVar "nomad/jobs/redis@example_namespace" }}{{ .maxconns }}{{ end }}
EOH
}
nomadVarExists
This function can be used to check if a Nomad variable exists at the provided
path, assuming the task has access rights to the variable based on the task's
workload identity. If a variable exists, this will return true, false
otherwise. Unlike nomadVar
, this function will not block if the
variable does not exist, which can be useful for controlling flow.
For example:
template {
data = <<EOH
{{ if nomadVarExists "app/beta_active" }}
# ...
{{ else }}
# ...
{{ end }}
EOH
}
Consul Integration
Warning
Using the Consul integration results in an implicit dependency on Consul. If
Consul is unavailable, then after some retries, running allocations will be
stopped. Refer to the template.consul_retry
client configuration option
for more details and to adjust the retry behaviour.
Consul KV
Consul KV values can be accessed using the key
function to
retrieve a single value from a key path. The ls
function can be
used to retrieve all keys in a path. For deeply nested paths, use the
tree
function.
template {
data = <<EOF
# Read single key from Consul KV.
APP_NAME = "{{key "app/name"}}"
# Read all keys in the path `app/environment` from Consul KV.
{{range ls "app/environment"}}
{{.Key}}={{.Value}}
{{end}}
EOF
destination = "local/env"
env = true
}
Consul Services
The Consul service catalog can be queried using the service
and services
functions. For Connect-capable services, use
the connect
function.
template {
data = <<EOF
# Configuration for a single upstream service.
upstream my_app {
{{- range service "my-app" }}
server {{ .Address }}:{{ .Port }};{{- end }}
}
# Configuration for all services in the catalog.
{{ range services }}
# Configuration for service {{ .Name }}.
upstream {{ .Name | toLower }} {
{{- range service .Name }}
server {{ .Address}}:{{ .Port }};{{- end }}
}
{{ end -}}
EOF
destination = "local/nginx.conf"
}
Vault Integration
Warning
Using the Vault integration results in an implicit dependency on Vault. If
Vault is unavailable, then after some retries, running allocations will be
stopped. Refer to the template.vault_retry
client configuration option
for more details and to adjust the retry behaviour.
PKI Certificate
Vault is a popular open source tool for managing secrets. In addition to acting as an encrypted KV store, Vault can also generate dynamic secrets, like PKI/TLS certificates.
When generating PKI certificates with Vault, the certificate, private key, and any intermediate certs are all returned as part of the same API call. Most software requires these files be placed in separate files on the system.
As individual files
For templates, all dependencies are mapped into a single list. This means that multiple templates watching the same path return the same data.
template {
data = <<EOH
{{ with pkiCert "pki/issue/foo" "common_name=foo.service.consul" "ip_sans=127.0.0.1" }}
{{- .Cert -}}
{{ end }}
EOH
destination = "${NOMAD_SECRETS_DIR}/certificate.crt"
change_mode = "restart"
}
template {
data = <<EOH
{{ with pkiCert "pki/issue/foo" "common_name=foo.service.consul" "ip_sans=127.0.0.1" }}
{{- .CA -}}
{{ end }}
EOH
destination = "${NOMAD_SECRETS_DIR}/ca.crt"
change_mode = "restart"
}
template {
data = <<EOH
{{ with pkiCert "pki/issue/foo" "common_name=foo.service.consul" "ip_sans=127.0.0.1" }}
{{- .Key -}}
{{ end }}
EOH
destination = "${NOMAD_SECRETS_DIR}/private_key.key"
change_mode = "restart"
}
These are three different input templates, but when run under the Nomad job, they are compressed into a single call, sharing the resulting data.
As a PEM formatted file
This example acquires a PKI certificate from Vault in PEM format, concatenates the elements into a bundle, and stores it into your application's secret directory.
template {
data = <<EOH
{{ with pkiCert "pki/issue/foo" "common_name=foo.service.consul" "ip_sans=127.0.0.1" "format=pem" }}
{{ .Cert }}
{{ .CA }}
{{ .Key }}{{ end }}
EOH
destination = "${NOMAD_SECRETS_DIR}/bundle.pem"
change_mode = "restart"
}
Vault KV API v1
Under Vault KV API v1, paths start with secret/
, and the response returns the
raw key/value data. This secret was set using
vault kv put secret/aws/s3 aws_access_key_id=somekeyid
.
template {
data = <<EOF
AWS_ACCESS_KEY_ID = "{{with secret "secret/aws/s3"}}{{.Data.aws_access_key_id}}{{end}}"
EOF
}
Note that if the name of a secret includes the -
character, you must access
it by index. This secret was set using vault kv put secret/app db-password=somepassword
.
template {
data = <<EOF
DB_PASSWORD = "{{with secret "secret/app"}}{{index .Data "db-password"}}{{end}}"
EOF
}
Vault KV API v2
Under Vault KV API v2, paths start with secret/data/
, and the response returns
metadata in addition to key/value data. This secret was set using
vault kv put secret/aws/s3 aws_access_key_id=somekeyid
.
template {
data = <<EOF
AWS_ACCESS_KEY_ID = "{{with secret "secret/data/aws/s3"}}{{.Data.data.aws_access_key_id}}{{end}}"
EOF
}
Notice the addition of data
in both the path and the field accessor string.
Additionally, when using the Vault v2 API, the Vault policies applied to your
Nomad jobs will need to grant permissions to read
under secret/data/...
rather than secret/...
.
Like KV API v1, if the name of a secret includes the -
character, you must
access it by index. This secret was set using
vault kv put secret/app db-password=somepassword
.
template {
data = <<EOF
DB_PASSWORD = "{{with secret "secret/data/app"}}{{index .Data.data "db-password"}}{{end}}"
EOF
}
Client Configuration
The template
block has the following client configuration
options:
function_denylist
([]string: ["plugin"])
- Specifies a list of template rendering functions that should be disallowed in job specs. By default, theplugin
function is disallowed as it allows running arbitrary commands on the host as root (unless Nomad is configured to run as a non-root user),executeTemplate
is disallowed to prevent accidental or malicious infinitely recursive execution, andwriteToFile
is disallowed.disable_file_sandbox
(bool: false)
- Allows templates access to arbitrary files on the client host via thefile
function. By default, templates can access files only within the task working directory.