Processors

Processors transform or enrich metadata records after extraction and before they reach a sink. They are executed sequentially in the order defined in the recipe — the output of one processor becomes the input of the next.

Enrich

enrich

Append custom attributes to each asset's data. Useful for adding metadata that does not exist in the source system.

Config

Key	Value	Example	Description
attributes	map	{team: platform}	Key-value pairs to merge into the asset's attributes	required

Sample usage

processors:
  - name: enrich
    config:
      attributes:
        team: data-platform
        environment: production

More details

Labels

labels

Append key-value labels to each asset's labels field. Labels are useful for categorization, filtering, and routing in downstream systems.

Config

Key	Value	Example	Description
labels	map[string]string	{source: meteor}	Key-value pairs to append to the asset's labels	required

Sample usage

processors:
  - name: labels
    config:
      labels:
        source: meteor
        classification: internal

More details

Script

script

Transform each asset using a user-defined script. Currently, Tengo is the only supported script engine. The script processor gives you full control over asset transformation, including the ability to make HTTP calls to external services.

Config

Key	Value	Example	Description
engine	string	tengo	Script engine to use (currently only tengo)	required
script	string	see below	Inline Tengo script	required

Sample usage

processors:
  - name: script
    config:
      engine: tengo
      script: |
        asset.labels["processed"] = "true"

More details

Chaining Processors

Processors execute sequentially in the order they appear in the recipe. Each processor receives the output of the previous one. This allows you to build transformation pipelines:

processors:
  - name: enrich          # Step 1: add attributes
    config:
      attributes:
        domain: payments
  - name: labels          # Step 2: add labels
    config:
      labels:
        source: meteor
  - name: script          # Step 3: custom transform
    config:
      engine: tengo
      script: |
        asset.name = asset.name + " (processed)"

If a processor fails, the entire recipe execution fails. There is no skip-on-error behavior — fix the processor configuration to resolve errors.