Node Spec Format

Every custom node has a node-spec.yaml file. It defines the node’s contract: what data it accepts, what it produces, and what parameters it requires. The runtime enforces this contract. AI agents use this spec to generate compatible nodes.

Top-level schema

# node-spec.yaml — complete schema
# ─────────────────────────────────────────────

id: enrich-leads            # string, required. Unique node identifier.
                             # Lowercase, hyphens only. Must match the
                             # node ID in pipeline.rf.yaml.

type: custom                # string, required. Node type.
                             # custom | deterministic | source | service

description: >              # string, optional. What this node does.
  Enrich lead records with company data
  from the Clearbit API.

inputs:                      # object, required. Map of port name → port spec.
  # ... (see Port schema below)

outputs:                     # object, required. Map of port name → port spec.
  # ... (see Port schema below)

params:                      # object, optional. Map of param name → param spec.
  # ... (see Param schema below)

sandbox:                     # object, optional. Sandbox overrides.
  network: false             # boolean. Allow network access. Default: false.
  timeout: 30000             # integer. Max execution time in ms. Default: 30000.

Port schema

Ports are the typed connection points on a node. Each input port declares the data type it accepts. Each output port declares the data type it produces.

inputs:
  leads:                     # string (port name). Lowercase, hyphens.
    type: Table              # string, required. One of: Value, Record, Table, Stream.

    schema:                  # object, required for Table and Record.
      name:                  # field name
        type: string         # string, required. Field type:
                             #   string, number, integer, boolean, null,
                             #   array, object
        required: true       # boolean, optional. Default: true.
        description: ""      # string, optional. Field description.

      email:
        type: string

      score:
        type: number
        required: false      # Optional field — may be absent in input data.

    description: >           # string, optional. Port description.
      Leads table with contact info and scores.

outputs:
  enriched:
    type: Table
    schema:
      name: { type: string }
      email: { type: string }
      score: { type: number }
      company: { type: string }
      industry: { type: string }
      employee_count: { type: integer }

Port types

Type	Schema required	Description
Value	No	Single scalar: string, number, boolean, null.
Record	Yes	Single JSON object with named fields.
Table	Yes	Ordered collection of records (rows).
Stream	Yes	Unbounded sequence of records.

Value ports do not have a schema — they carry a single typed scalar. Record, Table, and Stream ports require a schema defining their fields.

Param schema

Parameters configure the node’s behavior. They are set in the pipeline YAML and passed to the node at execution time.

params:
  api_key:                   # string (param name). Lowercase, underscores.
    type: string             # string, required. Param type:
                             #   string, number, integer, boolean
    required: true           # boolean, optional. Default: false.
    description: >           # string, optional.
      Clearbit API key for enrichment lookups.
    secret: true             # boolean, optional. Stored encrypted. Default: false.

  batch_size:
    type: integer
    required: false
    default: 100             # any, optional. Default value if not provided.
    description: >
      Number of records to process per API call.

  include_funding:
    type: boolean
    required: false
    default: false
    description: >
      Include funding data in enrichment results.

Param fields

Field	Type	Required	Description
type	string	yes	string, number, integer, boolean
required	boolean	no	Whether the param must be provided. Default: false.
default	any	no	Default value when param is not set.
description	string	no	Human-readable description.
secret	boolean	no	Whether to store encrypted. Default: false.
enum	array	no	Allowed values. Validation rejects others.

Complete example

A node that reads leads from a Table, enriches them via an external API, and outputs an enriched Table:

id: enrich-leads
type: custom
description: >
  Look up company data for each lead using the Clearbit API.
  Adds company name, industry, and employee count.

inputs:
  leads:
    type: Table
    schema:
      name: { type: string }
      email: { type: string }
      score: { type: number, required: false }
    description: Raw leads with at least name and email.

outputs:
  enriched:
    type: Table
    schema:
      name: { type: string }
      email: { type: string }
      score: { type: number, required: false }
      company: { type: string }
      industry: { type: string }
      employee_count: { type: integer }
    description: Leads enriched with Clearbit company data.

  errors:
    type: Table
    schema:
      email: { type: string }
      error: { type: string }
    description: Leads that failed enrichment lookup.

params:
  api_key:
    type: string
    required: true
    secret: true
    description: Clearbit API key.

  batch_size:
    type: integer
    required: false
    default: 50
    description: Records per API batch.

  timeout_ms:
    type: integer
    required: false
    default: 5000
    description: Per-request timeout in milliseconds.

sandbox:
  network: true
  timeout: 60000

File layout

Custom nodes live in the nodes/ directory of the pipeline workspace:

nodes/
  enrich-leads/
    node-spec.yaml          # this file — the contract
    main.ts                 # implementation (generated or hand-written)
    schemas/
      leads.schema.json     # input schema (auto-generated from spec)
      enriched.schema.json  # output schema (auto-generated from spec)
    artifacts/
      enriched.ndjson       # output data (written at execution time)
      errors.ndjson         # error output (written at execution time)

The runtime reads node-spec.yaml, validates inputs against the declared schemas, executes main.ts (or main.sql, main.py, run.sh), and validates outputs before passing them downstream.