rf run

Synopsis

rf run [options]

Description

rf run executes the pipeline defined in flow.yaml. It resolves execution order from the graph, validates schemas, and runs each node in dependency order.

Options

Flag	Default	Description
--dry-run	—	Validate and show the execution plan without running any nodes.
--node <id>	—	Run only this node and its upstream dependencies.
--verbose	—	Show detailed output including stdout/stderr from each node.
--timeout <ms>	300000	Maximum execution time for the entire pipeline, in milliseconds.
--parallel	—	Run independent nodes concurrently. Default: sequential.

Examples

Full run

$ rf run

[read-leads]     Reading data/leads.csv
[read-leads]     5 rows read
[score-leads]    Running SQL: SELECT *, score * 1.5 AS weighted FROM rows
[score-leads]    5 rows produced
[filter-top]     Running SQL: SELECT * FROM rows WHERE weighted >= 80
[filter-top]     3 rows matched
[write-output]   Writing output/qualified.json
[write-output]   3 rows written

Pipeline completed. 4 nodes executed in 0.6s.

Dry run

Shows the execution plan without processing any data.

$ rf run --dry-run

Execution plan:
  1. read-leads      source         file.read
  2. score-leads     deterministic  sql.query
  3. filter-top      deterministic  sql.query
  4. write-output    deterministic  file.write

4 nodes would execute. No data was processed.

Single node

Runs the target node and its transitive upstream dependencies. Downstream nodes are skipped.

$ rf run --node filter-top

[read-leads]     5 rows read          (dependency)
[score-leads]    5 rows produced      (dependency)
[filter-top]     3 rows matched       (target)

3 nodes executed in 0.4s.

Verbose output

$ rf run --verbose

[read-leads]     Reading data/leads.csv
[read-leads]     Parsed 5 rows, 3 columns (name, email, score)
[read-leads]     Schema: {name: string, email: string, score: number}
[read-leads]     Duration: 12ms
...

Output format

Each node prints its name in brackets, followed by a status line. The final summary shows total nodes executed, elapsed time, and any errors or warnings.

When --parallel is set, independent nodes (those with no edges between them) run concurrently.

$ rf run --parallel

[read-leads]     5 rows read
[read-config]    Config loaded              # parallel with read-leads
[score-leads]    5 rows produced
[filter-top]     3 rows matched
[write-output]   3 rows written

Pipeline completed. 5 nodes executed in 0.3s (parallel).

Error handling

When a node fails, rf run prints the error, skips all downstream nodes, and exits with code 1. Nodes on independent branches continue if --parallel is set.

$ rf run

[read-leads]     5 rows read
[call-api]       ERROR: HTTP 401 Unauthorized
[write-output]   SKIPPED (upstream failure)

Pipeline failed. 1 error.

Exit codes

Code	Meaning
0	All nodes completed successfully.
1	Validation error (schema mismatch, cycle, missing ref).
2	One or more nodes failed during execution.
3	Pipeline timed out.