data.map

Transform each row. Add columns, rename fields, compute values.

Input: Table | Output: Table

Minimal example

Add a full_name column from first and last name:

nodes:
  add-name:
    type: data.map
    config:
      include_original: true
      fields:
        full_name: "{{ first_name }} {{ last_name }}"

Config reference

Field	Type	Required	Description
fields	map	yes	Output column name to template expression
include_original	boolean	no	Keep all input columns alongside mapped fields. Default: false

When include_original is true, the output contains every input column plus the mapped fields. If a mapped field name matches an existing column, the mapped value replaces it. When false, the output contains only the mapped fields.

Expression syntax

Expressions use {{ ... }} delimiters. Inside them, you reference columns by name and apply operations.

Column references

fields:
  email_copy: "{{ email }}"

Arithmetic

fields:
  total: "{{ price * quantity }}"
  margin: "{{ revenue - cost }}"
  tax: "{{ subtotal * 0.19 }}"

String concatenation

Multiple {{ }} blocks in one expression produce string concatenation:

fields:
  full_name: "{{ first_name }} {{ last_name }}"
  greeting: "Hello {{ first_name }}!"

Pipe filters

Filters transform values. Syntax: {{ field | filter }}.

Filter	Result type	Description
lower	string	Lowercase
upper	string	Uppercase
trim	string	Strip whitespace
length	number	String length
round	number	Round to nearest integer
ceil	number	Round up
floor	number	Round down
abs	number	Absolute value
to_string	string	Cast to string
to_number	number	Cast to number (null on failure)
strip_currency	number	Remove currency symbols, parse as number
extract_number	number	Pull first numeric value from string
parse_date_dmy	date	Parse dd/mm/yyyy
parse_date_mdy	date	Parse mm/dd/yyyy
parse_date_ymd	date	Parse yyyy-mm-dd

fields:
  email_lower: "{{ email | lower }}"
  price_cents: "{{ price_text | strip_currency }}"
  joined: "{{ join_date | parse_date_dmy }}"

Filters with arguments

replace and regexp_replace take arguments:

fields:
  clean_phone: '{{ phone | replace("+", "00") }}'
  no_tags: '{{ body | regexp_replace("<[^>]+>", "") }}'

Coalesce (null fallback)

fields:
  display_name: '{{ nickname ?? "Anonymous" }}'

Conditional (ternary)

fields:
  tier: '{{ score > 0.7 ? "high" : "low" }}'

Quoted identifiers

For column names with special characters, use double quotes inside the expression:

fields:
  amount: '{{ "Debit/Credit" | to_number }}'

Progressive examples

Computed columns with originals

nodes:
  enrich-orders:
    type: data.map
    config:
      include_original: true
      fields:
        total: "{{ price * quantity }}"
        discount_amount: "{{ total * discount_pct }}"
        status_label: '{{ status == "paid" ? "Paid" : "Pending" }}'

Projection (select + transform)

nodes:
  clean-contacts:
    type: data.map
    config:
      fields:
        name: "{{ first_name }} {{ last_name }}"
        email: "{{ email | lower | trim }}"
        score: "{{ raw_score | round }}"

Since include_original defaults to false, the output contains only name, email, and score.

String operations

nodes:
  normalize:
    type: data.map
    config:
      include_original: true
      fields:
        email: "{{ email | lower | trim }}"
        domain: '{{ email | lower }}'
        name_upper: "{{ name | upper }}"

Type casting

nodes:
  parse-amounts:
    type: data.map
    config:
      include_original: true
      fields:
        amount: "{{ amount_text | strip_currency }}"
        quantity: "{{ qty_string | to_number }}"
        label: "{{ id | to_string }}"

Casts use TRY_CAST internally — unparseable values become null instead of errors.

Date formatting

nodes:
  parse-dates:
    type: data.map
    config:
      include_original: true
      fields:
        signup_date: "{{ raw_date | parse_date_mdy }}"
        birth_date: "{{ dob | parse_date_dmy }}"

Math with conditionals

nodes:
  compute-pricing:
    type: data.map
    config:
      include_original: true
      fields:
        final_price: '{{ is_member == true ? price * 0.9 : price }}'
        tax: "{{ price * 0.19 | round }}"

Edge cases

NULL propagation. Any arithmetic or filter applied to a NULL value produces NULL. Use the coalesce operator (??) to provide fallback values.

Type changes. Mapped fields can change the type of an existing column. If score was a string and you map it with to_number, the output schema reflects the new type.

Empty input. An empty input table produces an empty output table with the mapped schema.