Data Pipelines

Fetch: Retrieve and Filter

Retrieve data with optional filtering, ordering, and limiting. Results are always typed:

(* Fetch with filtering and ordering *)
<Fetch> the <active-users: List<User>> from the <users>
    where <status> is "active"
    order by <name> asc
    limit 100.

Filter: Subset Existing Collections

Filter an existing collection with a predicate:

(* Filter for premium users *)
<Filter> the <premium-users: List<User>> from the <users>
    where <tier> is "premium".

Map: Transform to Different Types

Transform a collection to a different OpenAPI-defined type. The runtime automatically maps matching field names:

(* Map List<User> to List<UserSummary> *)
<Map> the <summaries: List<UserSummary>> from the <users>.

Reduce: Aggregate to Single Value

Aggregate a collection using sum, count, avg, min, max, and more:

(* Sum all order amounts *)
<Reduce> the <total: Float> from the <orders>
    with sum(<amount>).

(* Count orders *)
<Reduce> the <order-count: Integer> from the <orders>
    with count().

(* Average price of electronics *)
<Reduce> the <avg-price: Float> from the <products>
    where <category> is "electronics"
    with avg(<price>).

Comparison Operators

Use these operators in where clauses:

Aggregation Functions

Complete Example

Here's a sales analytics feature set using all pipeline operations:

(Sales Report: Analytics) {
    (* Fetch recent orders *)
    <Fetch> the <recent-orders: List<Order>> from the <orders>
        where <created-at> > now().minus(30.days)
        order by <created-at> desc.

    (* Get total revenue *)
    <Reduce> the <total-revenue: Float> from the <recent-orders>
        with sum(<amount>).

    (* Count pending orders *)
    <Reduce> the <pending-count: Integer> from the <recent-orders>
        where <status> is "pending"
        with count().

    (* Map to summaries *)
    <Map> the <summaries: List<OrderSummary>> from the <recent-orders>.

    (* Filter high-value orders *)
    <Filter> the <high-value: List<Order>> from the <recent-orders>
        where <amount> > 1000.

    <Return> an <OK: status> with {
        orders: <summaries>,
        total: <total-revenue>,
        pending: <pending-count>,
        high-value-count: <high-value>.count()
    }.
}

Why This Approach?

Type Safety

All results are typed via OpenAPI schemas. <users: List<User>> is guaranteed to contain User objects as defined in your schema.

Readability

Operations read naturally. "Fetch the active users from users where status is active" is close to how you'd describe the operation in English.

Simplicity

No complex SQL. No JOINs, subqueries, or CTEs. Just straightforward pipeline operations that are easy to understand and maintain.