Job Actions¶

Lava can perform specified actions on the completion of a job depending on whether the job:

completed successfully (on_success); or
failed and will be retried (on_retry); or
failed and won't be retried (on_fail).

Actions are not executed if the job does not start (e.g. if the job specification is malformed).

These actions are defined as part of the job specification or can be inherited from the realm specification. An action specification at the job level will override one at the realm level.

A typical action specification within a job or realm specification looks like:

{
  "on_fail": [
    {
      "action": "sns",
      "message": "{{job.job_id}}@{{realm.realm}} ({{job.run_id}}) failed: {{result.error}}",
      "topic": "arn:aws:sns:ap-southeast-2:990565496141:lava-proto-notices"
    }
  ],
  "on_retry": [
    {
      "action": "log",
      "message": "Failed on attempt {{ globals.lava.iteration }} but will try again!"
    }
  ],
  "on_success": [
    {
      "action": "log",
      "message": "It worked!"
    },
    {
      "action": "dispatch",
      "job_id": "downstream_job"
    }
  ]
}

Actions in an action list are executed in the order specified.

Apart from the mandatory action field, action specification parameters are dependent on the action type.

Some actions support Jinja rendering of some of the action parameters.

Actions are implemented using a simple plugin mechanism. Adding new actions is relatively straightforward.

Action type: dispatch¶

The dispatch action type dispatches another lava job.

Any globals available in the job that triggers the action will also be passed in the dispatch request to the job being dispatched.

Field	Type	Required	Description
action	String	Yes	The action type: `dispatch`.
delay	String	No	Dispatch message sending delay in the form `nnX` where `nn` is a number and `X` is `s` (seconds) or `m` (minutes). The maximum allowed value is 15 minutes.
job_id	String	Yes	The job to be dispatched. This is Jinja rendered prior to use.
parameters	Map[String,*]	No	A map of parameters that will be sent with the dispatch. These are Jinja rendered prior to use. The dispatched job will receive these in the `parameters` element of its job specification.

The Jinja rendered action parameters have the following variables injected.

Name	Type	Description
globals	dict[str,*]	The globals map for the job that triggered the action.
job	dict[str,*]	The augmented job specification for the job that triggered the action.
realm	dict[str,*]	The realm specification.
result	dict[str,*]	The result object from the job that triggered the action.
start	datetime	The local time when the job run started.
state	dict[str,*]	A dictionary of the state items imported into the job, keyed on state_id. The default values are updated at run-time with any current values obtainable from the state table.
ustart	datetime	The UTC time when the job run started.
utils	dict[str,runnable]	A dictionary of utility functions that can be used in the Jinja markup.

Refer to Jinja Rendering in Lava for more information.

Action type: email¶

The email action uses either the email or a bare-metal AWS Simple Email Service (SES) default to send an email message.

Field	Type	Required	Description
action	String	Yes	The action type: `email`.
attachments	List[string]	No	A list of file names to add as attachments. If specified, `email_conn` must also be specified, as the default bare-metal AWS SES sender does not support attachments. Files can be local or in S3 (`s3://...`). Each filename is individually Jinja rendered. Be careful with this to avoid using untrusted globals in filenames.
email_conn	String	No	The name of an email connector to use for sending email. If not specified, then bare-metal AWS SES is used.
from	String	No	The source email address. If not specified, lava will defer to the `email_conn` connector if specified, or else look for a value in the realm specification (see below) or construct a default by scanning the SES configuration for validated domains.
message	String	Yes	The message body. If the message text begins with `<HTML>` (case-insensitive), it is sent as a HTML message body, otherwise as text. This is Jinja rendered prior to use.
region	String	No	The AWS region name for the SES service. If not specified, lava will look for a value in the realm specification (see below) or use the default of `us-east-1`. Ignored if `email_conn` is specified.
subject	String	Yes	The message subject. This is Jinja rendered prior to use.
to	String or List[String]	Yes	The destination email address(es).

The Jinja rendered action parameters have the following variables injected.

Name	Type	Description
globals	dict[str,*]	The globals map for the job that triggered the action.
job	dict[str,*]	The augmented job specification for the job that triggered the action.
realm	dict[str,*]	The realm specification.
result	dict[str,*]	The result object from the job that triggered the action.
start	datetime	The local time when the job run started.
state	dict[str,*]	A dictionary of the state items imported into the job, keyed on state_id. The default values are updated at run-time with any current values obtainable from the state table.
ustart	datetime	The UTC time when the job run started.
utils	dict[str,runnable]	A dictionary of utility functions that can be used in the Jinja markup.

Refer to Jinja Rendering in Lava for more information.

Default values for some of the fields required for the email action to use AWS SES can be set in the realms table by setting entries in the config realm entry, thus:

{
  "...": "... other realm related elements ...",
  "config": {
    "ses_region": "... SES region name ...",
    "ses_from": "... Default source email address"
  }
}

Action type: event¶

The event action type sends an event to AWS EventBridge.

Field	Type	Required	Description
action	String	Yes	The action type: `event`.
detail_type	String	No	The event detail type. This is Jinja rendered prior to use. Defaults to `Lava Job Action`.
detail	String or Map[String,*]	No	Event detail. The string, or the map values, are Jinja rendered prior to use. Defaults to a map containing realm, worker, job_id, run_id and exit_status.
event_bus	String	No	The event bus name. This is Jinja rendered prior to use. Defaults to the default event bus.
resources	List[String]	No	A list of resources. This is Jinja rendered prior to use. Defaults to an empty list.
source	String	No	The event source. This is Jinja rendered prior to use. Defaults to `lava.<REALM>`.

Info

The realm CloudFormation template only provides workers with permission to put events on the default event bus.

The Jinja rendered action parameters have the following variables injected.

Name	Type	Description
globals	dict[str,*]	The globals map for the job that triggered the action.
job	dict[str,*]	The augmented job specification for the job that triggered the action.
realm	dict[str,*]	The realm specification.
result	dict[str,*]	The result object from the job that triggered the action.
start	datetime	The local time when the job run started.
state	dict[str,*]	A dictionary of the state items imported into the job, keyed on state_id. The default values are updated at run-time with any current values obtainable from the state table.
ustart	datetime	The UTC time when the job run started.
utils	dict[str,runnable]	A dictionary of utility functions that can be used in the Jinja markup.

Refer to Jinja Rendering in Lava for more information.

This is the minimal action specification:

{
    "on_success": [
        {
            "action": "event"
        }
    ]
}

It will produce an event in the default event bus like so:

{
  "version": "0",
  "id": "1bdf1ead-95ef-79fd-1342-8f8ff1e57765",
  "detail-type": "Lava Job Action",
  "source": "lava.dev",
  "account": "123456789012",
  "time": "2021-07-25T05:12:23Z",
  "region": "ap-southeast-2",
  "resources": [],
  "detail": {
    "realm": "dev",
    "worker": "core",
    "job_id": "action/event/default",
    "run_id": "05175b7c-8b0d-40ed-af31-0bc846bbe400",
    "exit_status": 0
  }
}

This is a sample, custom event:

{
    "on_success": [
        {
            "action": "event",
            "detail": {
                "job_id": "{{ job.job_id }}",
                "realm": "{{ realm.realm }}",
                "result": "{{result | tojson}}",
                "run_id": "{{job.run_id }}",
                "status": "success"
            }
        }
    ]
}

Action type: log¶

The log action type writes a message to the logger local to the worker with log level info. The worker log configuration is specified as command line arguments so it depends on these where the message ends up.

For more information on worker logging arguments run:

lava-worker --help

Field	Type	Required	Description
action	String	Yes	The action type: `log`.
message	String	Yes	The message to log. This is Jinja rendered.

The Jinja rendered action parameters have the following variables injected.

Name	Type	Description
globals	dict[str,*]	The globals map for the job that triggered the action.
job	dict[str,*]	The augmented job specification for the job that triggered the action.
realm	dict[str,*]	The realm specification.
result	dict[str,*]	The result object from the job that triggered the action.
start	datetime	The local time when the job run started.
state	dict[str,*]	A dictionary of the state items imported into the job, keyed on state_id. The default values are updated at run-time with any current values obtainable from the state table.
ustart	datetime	The UTC time when the job run started.
utils	dict[str,runnable]	A dictionary of utility functions that can be used in the Jinja markup.

Refer to Jinja Rendering in Lava for more information.

Action type: slack¶

The slack action sends a message to a Slack channel. It relies on the slack connector which uses Slack webhooks.

Field	Type	Required	Description
action	String	Yes	The action type: `slack`.
colour	String	No	As per the `colour` field of the slack connector. If not specified, the value specified in the connector is used.
from	String	No	An arbitrary source identifier for display in the Slack message. If not specified, any default value specified in the connector will be used.
message	String	Yes	The message body. This is Jinja rendered prior to use. Only the first 3,000 characters of the rendered value are used.
preamble	String	No	As per the `preamble` field of the slack connector. If not specified, the value specified in the connector is used.
slack_conn	String	Yes	The name of a slack connector to use.
style	String	No	As per the `style` field of the slack connector. If not specified, the value specified in the connector is used.
subject	String	No	The message subject. This is Jinja rendered prior to use. Only the first 250 characters of the rendered value are used.

The Jinja rendered action parameters have the following variables injected.

Name	Type	Description
globals	dict[str,*]	The globals map for the job that triggered the action.
job	dict[str,*]	The augmented job specification for the job that triggered the action.
realm	dict[str,*]	The realm specification.
result	dict[str,*]	The result object from the job that triggered the action.
start	datetime	The local time when the job run started.
state	dict[str,*]	A dictionary of the state items imported into the job, keyed on state_id. The default values are updated at run-time with any current values obtainable from the state table.
ustart	datetime	The UTC time when the job run started.
utils	dict[str,runnable]	A dictionary of utility functions that can be used in the Jinja markup.

Refer to Jinja Rendering in Lava for more information.

Action type: sns¶

The sns action type sends a message to an SNS topic.

Field	Type	Required	Description
action	String	Yes	The action type: `sns`.
message	String or Map[String,*]	Yes	The message body. This can be either a string or an object. If it is an object it will be JSON encoded before sending. This is Jinja rendered.
subject	String	No	The message subject. This is Jinja rendered. The rendered value will be truncated to 100 characters.
topic	String	Yes	The topic ARN.

The Jinja rendered action parameters have the following variables injected.

Name	Type	Description
globals	dict[str,*]	The globals map for the job that triggered the action.
job	dict[str,*]	The augmented job specification for the job that triggered the action.
realm	dict[str,*]	The realm specification.
result	dict[str,*]	The result object from the job that triggered the action.
start	datetime	The local time when the job run started.
state	dict[str,*]	A dictionary of the state items imported into the job, keyed on state_id. The default values are updated at run-time with any current values obtainable from the state table.
ustart	datetime	The UTC time when the job run started.
utils	dict[str,runnable]	A dictionary of utility functions that can be used in the Jinja markup.

Refer to Jinja Rendering in Lava for more information.

This is an example of how a JSON formatted object can be sent:

{
    "on_success" : [
        {
            "action": "sns",
            "message": {
                "key1": "Hello world",
                "key2": {
                    "key2a": "{{ job.globals.a_global }}",
                    "key2b": 42
                }
            },
            "topic": "arn:aws:sns:ap-southeast-2:123456789012:mytopic"
        }
    ]
}

Action type: sqs¶

The sqs action type sends a message to an SQS queue.

Field	Type	Required	Description
action	String	Yes	The action type: `sqs`.
dedup_id	String	No	The token used for deduplication of sent messages. This parameter applies only to FIFO queues. This is Jinja rendered.
delay	String	No	Message sending delay in the form `nnX` where `nn` is a number and `X` is `s` (seconds) or `m` (minutes). The maximum allowed value is 15 minutes.
group_id	String	No	The tag that specifies that a message belongs to a specific message group. This parameter applies only to FIFO queues. This is Jinja rendered.
message	String or Map[String,*]	Yes	The message body. This can be either a string or an object. If it is an object it will be JSON encoded before sending. This is Jinja rendered.
queue	String	Yes	The name or URL of the queue (not the ARN!).

The Jinja rendered action parameters have the following variables injected.

Name	Type	Description
globals	dict[str,*]	The globals map for the job that triggered the action.
job	dict[str,*]	The augmented job specification for the job that triggered the action.
realm	dict[str,*]	The realm specification.
result	dict[str,*]	The result object from the job that triggered the action.
start	datetime	The local time when the job run started.
state	dict[str,*]	A dictionary of the state items imported into the job, keyed on state_id. The default values are updated at run-time with any current values obtainable from the state table.
ustart	datetime	The UTC time when the job run started.
utils	dict[str,runnable]	A dictionary of utility functions that can be used in the Jinja markup.

Refer to Jinja Rendering in Lava for more information.

This is an example of how a JSON formatted object can be sent:

{
    "on_success" : [
        {
            "action": "sqs",
            "message": {
                "key1": "Hello world",
                "key2": {
                    "key2a": "{{ job.globals.a_global }}",
                    "key2b": 42
                }
            },
            "topic": "myqueue"
        }
    ]
}

Action type: state¶

The state action type allows jobs to post state items that can be subsequently retrieved by other jobs or authorised external actors.

Field	Type	Required	Description
action	String	Yes	The action type: `state`.
kms_key	String	No	The secure state item type supports KMS encryption of the value. This field specifies the KMS key to use, either as a KMS key ARN or a key alias in the form `alias/key-id`. Defaults to the `sys` key for the lava realm. Ignored for other state item types.
publisher	String	No	An arbitrary identifier for the entity posting the event item. Not used by lava itself. The default is the job ID. This is Jinja rendered prior to use.
state_id	String	Yes	The state entry ID. This is Jinja rendered prior to use.
ttl	String	No	Time to live for the state item specified as a duration in the form `nnX` where `nn` is a number and `X` is `s` (seconds), `m` (minutes), `h` (hours) or `d` (days). If greater than the maximum specified for the realm, it will be silently reduced to that value. A default value is provided by lava which can be overridden at the realm level.
type	String	No	The state item type. If not specified, the `json` type is used.
value	*	Yes	A JSON encodable object. This is Jinja rendered prior to use.

Info

Do not create state items with a state_id starting with lava. This prefix is reserved.

The Jinja rendered action parameters have the following variables injected.

Name	Type	Description
globals	dict[str,*]	The globals map for the job that triggered the action.
job	dict[str,*]	The augmented job specification for the job that triggered the action.
realm	dict[str,*]	The realm specification.
result	dict[str,*]	The result object from the job that triggered the action.
start	datetime	The local time when the job run started.
state	dict[str,*]	A dictionary of the state items imported into the job, keyed on state_id. The default values are updated at run-time with any current values obtainable from the state table.
ustart	datetime	The UTC time when the job run started.
utils	dict[str,runnable]	A dictionary of utility functions that can be used in the Jinja markup.

Refer to Jinja Rendering in Lava for more information.