rf run

rf run executes the pipeline defined in pipeline.rf.yaml. It resolves the execution order from the graph, validates schemas, and runs each node in dependency order.

Usage

rf run [options]

Options

Flag	Description
--file <path>	Path to the pipeline file. Default: pipeline.rf.yaml.
--node <id>	Run only this node and its upstream dependencies.
--dry-run	Validate and show execution plan without running any nodes.
--parallel	Run independent nodes concurrently. Default: sequential.
--verbose	Show detailed output including stdout/stderr from each node.

Execution flow

1. Parse. Read and parse pipeline.rf.yaml.
2. Validate. Check schema compatibility, resolve ref() bindings, detect cycles.
3. Plan. Compute topological execution order. Identify parallelizable groups.
4. Execute. Run each node in order. Pass output Tables/Records to downstream inputs.
5. Report. Print summary with row counts, timings, and any warnings.

If --node is specified, only the target node and its transitive upstream dependencies run. Downstream nodes are skipped.

Example

$ rf run

[read-leads]     Reading data/leads.csv
[read-leads]     5 rows read
[score-leads]    Running SQL: SELECT *, score * 1.5 AS weighted FROM rows
[score-leads]    5 rows produced
[filter-top]     Running SQL: SELECT * FROM rows WHERE weighted >= 80
[filter-top]     3 rows matched
[write-output]   Writing output/qualified.json
[write-output]   3 rows written

Pipeline completed. 4 nodes executed in 0.6s.

Run a single node

$ rf run --node filter-top

[read-leads]     5 rows read          (dependency)
[score-leads]    5 rows produced      (dependency)
[filter-top]     3 rows matched       (target)

3 nodes executed in 0.4s.

Dry run

$ rf run --dry-run

Execution plan:
  1. read-leads      source     file.read
  2. score-leads     deterministic  sql.query
  3. filter-top      deterministic  sql.query
  4. write-output    deterministic  file.write

4 nodes would execute. No data was processed.

Parallel execution

$ rf run --parallel

[read-leads]     5 rows read
[read-config]    Config loaded              # runs in parallel with read-leads
[score-leads]    5 rows produced
[filter-top]     3 rows matched
[write-output]   3 rows written

Pipeline completed. 5 nodes executed in 0.3s (parallel).

Independent nodes — those with no edges between them — run concurrently when --parallel is set.

Exit codes

Code	Meaning
0	All nodes completed successfully.
1	One or more nodes failed.
2	Validation error (schema mismatch, cycle, missing ref).

Error handling

When a node fails, rf run prints the error, skips all downstream nodes, and exits with code 1. Nodes on independent branches continue if --parallel is set.

$ rf run

[read-leads]     5 rows read
[call-api]       ERROR: HTTP 401 Unauthorized
[write-output]   SKIPPED (upstream failure)

Pipeline failed. 1 error.