DemandFlow Support Centre

Reference for DFQL, the streaming query language behind the DemandFlow client. Covers the request envelope, the NDJSON response, and all nine query types including single and batch POST, MULTI chaining, PARENTOBJECT traversal and CASCADE.

DFQL, the DemandFlow query language

DFQL is the query language behind every read and write the DemandFlow client makes. A DFQL request is a single JSON object describing what you want, and the response is a stream of NDJSON records, one JSON object per line, written as each record is read rather than assembled and returned at the end. That is what lets a screen start painting rows while the rest of the result set is still being fetched.

DFQL is lower level than the public REST API. Where /v1/objects and /v1/entities give you one well-formed resource per call, DFQL gives you the primitives the platform itself uses: read by ID, batch read, prefix query on a comboKey, parallel multi-query with chaining, ancestor traversal, and full descendant cascade.

Endpoint

All DFQL queries are sent to a single endpoint:

POST https://stream.demandflow.com/dfql

The response is application/x-ndjson, written progressively as records are read.

Authentication

Authenticate with a personal access token, the opaque pat_-prefixed string you generate for scripts and integrations, sent as a bearer token in the Authorization header:

Authorization: Bearer pat_<your-token>

The token resolves to a user, the user resolves to a subscription, and the subscription is the tenant table, so a DFQL query can only ever see one tenant's data. A missing token is 401, an invalid one 403. See Getting an API Token (PAT) for how to create one.

Request shape

The body is an object with a single query property. The type inside it selects the operation:

{
  "query": {
    "type": "ENTITY_COMBO",
    "entity": "PJ",
    "key": "SUB:<your-sub-id>",
    "load": "id,name,status"
  }
}

Omitting query or query.type is a 400. An unrecognised type is reported as an error record on the stream.

Response shape

One JSON object per line. Ordinary records are the objects themselves. Two other kinds of line can appear:

Meta records, marked {"_meta": true, "_type": "count" | "scan" | "post", ...}. A count record carries entity and count. A scan record carries scanned and matched and is emitted whenever a text filter ran, so you can see how much was read to produce the matches. A post record closes a batch create with created and failed totals.
Error records, {"_type": "error", "message": "..."}. Because the response has already begun streaming, per-query failures arrive as records rather than HTTP status codes. Always check _type on each line.

Records from different query lines in a MULTI are interleaved. Route each one by its entity field, not by arrival order.

Query types

GET, read one object by ID

{ "query": { "type": "GET", "id": "<object-id>", "entity": "PJ" } }

id is required. entity is optional and used for routing, for example sending LOG reads to the tenant log table. Emits a single record.

BATCHGET, read many objects by ID

{ "query": { "type": "BATCHGET", "id": ["<id-1>", "<id-2>"], "entity": "PJ" } }

id must be a non-empty array. IDs are fetched in batches of 100 behind the scenes, so you can pass more than 100 and the endpoint handles the chunking. Records stream as each batch returns, so ordering does not match the order of the IDs you sent. IDs that do not exist simply produce no record.

ENTITY_COMBO, prefix query on a comboKey

The workhorse. Queries the entity-comboKey-index for one entity where the key begins with a prefix, which is how the DemandFlow hierarchy is navigated.

{
  "query": {
    "type": "ENTITY_COMBO",
    "entity": "PJ",
    "key": "SUB:<your-sub-id>|PORT:<portfolio-id>",
    "load": "id,name,status,startDate",
    "limit": 500
  }
}

entity, required, uppercase entity code.
key, required, the comboKey prefix to match. Broader prefixes return more, so SUB:<sub> returns every record of that entity in the tenant, and adding segments narrows it to one branch of the hierarchy.
load, optional, comma-separated field projection. Applied after the record is read, so it saves bandwidth rather than database work.
limit, optional, hard cap on records returned.
tsStart, tsEnd, for the LOG entity only, an epoch-millisecond range against a timestamp-suffixed comboKey. LOG also reads newest first, where every other entity reads in ascending key order.

Paging is automatic. Without a limit the endpoint will follow up to 100 pages of results.

MULTI, many queries in parallel, optionally chained

Takes an array of query lines, runs them concurrently, and writes them all to one stream. This is how a dashboard loads everything it needs in one round trip.

{
  "query": {
    "type": "MULTI",
    "queries": [
      { "entity": "PPL",
        "comboKey": "comboKey",
        "query": "SUB:<sub>",
        "load": "id,name,email" },
      { "entity": "ACTION",
        "comboKey": "comboKey",
        "query": "SUB:<sub>",
        "countOnly": true,
        "tag": "actionCount" },
      { "entity": "RISK",
        "comboKey": "comboKey",
        "query": "SUB:<sub>",
        "filter": { "field": "description", "term": "supplier" } }
    ]
  }
}

Each query line accepts:

entity, required.
comboKey, the name of the key attribute to match on. Normally the literal string "comboKey". Entities with a secondary key index, such as comboKey2, can be queried by naming it here.
query, required, the prefix to match on that key.
load, limit, tsStart, tsEnd, as for ENTITY_COMBO.
countOnly, when true emits a single count meta record instead of the data.
filter, a case-insensitive text filter { "field": "message", "term": "timeout" }. Use "field": "*" to match anywhere in the record. Filtering happens after the read, so it always emits a scan meta record telling you how many were scanned to produce the matches.
tag, an arbitrary value echoed back on this line's meta records, so counts and scan stats can be matched to the query line that produced them.

Chained queries. A query line can depend on the results of another. Give the producing line a queryId, then reference it from a dependent line with parentQuery, placing that same token inside the query prefix where the parent's ID should be substituted:

{
  "query": {
    "type": "MULTI",
    "queries": [
      { "queryId": "pjs",
        "entity": "PJ",
        "comboKey": "comboKey",
        "query": "SUB:<sub>",
        "load": "id,name" },
      { "parentQuery": "pjs",
        "parentEntity": "PJ",
        "entity": "WTASK",
        "comboKey": "comboKey",
        "query": "SUB:<sub>|PJ:pjs" }
    ]
  }
}

The chained line is expanded once per parent record, with the token replaced by that record's ID. Root lines run first, then chained lines run in waves as their parents complete, to a maximum of 10 waves, in batches of 10 expansions at a time. parentEntity is optional and narrows the parent set to records of that entity. A parentQuery that names a queryId nothing produced is skipped, with a warning in the logs.

PARENTOBJECT, walk up to an ancestor

{
  "query": {
    "type": "PARENTOBJECT",
    "sourceId": "<task-id>",
    "sourceEntity": "WTASK",
    "parentEntity": "PORT"
  }
}

Returns the source object first, then every intermediate ancestor it loads, ending with the record whose entity matches parentEntity. It walks the comboKey one segment at a time, from nearest parent outward, ignoring the SUB segment and the record's own trailing ENT or ID segment, and continues from the topmost ancestor when the target is not present in the current key. Traversal stops after 20 steps and reports an error record if the target was never reached.

CASCADE, stream a whole subtree

Returns a root object and every descendant, derived from the entity definitions rather than a hardcoded map, so it follows the real parent and child structure including any tenant-specific definition overrides.

{
  "query": {
    "type": "CASCADE",
    "id": "<project-id>",
    "entities": ["WTASK", "PJMS", "RISK"],
    "filter": [
      { "entity": "WTASK", "prop": "status", "value": "Open" }
    ],
    "projection": [
      { "entity": "WTASK", "fields": ["id", "name", "status"] }
    ],
    "operations": [
      { "entity": "WTASK",
        "type": "sum",
        "property": "totalCost",
        "fields": ["labourCost", "materialCost"] }
    ]
  }
}

id, required, the root object. It is always returned first.
entities, optional allowlist of descendant entity codes. Omit it to return everything below the root.
filter, optional array of {entity, prop, value}. Conditions for the same entity are ANDed. A record that fails its filter is excluded from the output and is not descended into, so filtering prunes the tree rather than just the result.
projection, optional array of {entity, fields}, restricting output fields per entity.
operations, optional array of computed fields applied before projection. Type sum adds property as the total of the named fields. If you project, remember to include the computed property in fields or it will be stripped again.

Visited IDs are tracked, so a cycle in the data cannot loop the cascade.

POST, create an object

{
  "query": {
    "type": "POST",
    "fields": {
      "entity": "PPL",
      "level": 220,
      "comboKey": "SUB:<your-sub-id>|ENT:",
      "name": "Ada Lovelace"
    }
  }
}

fields is the new record, and three system fields are required within it:

entity, the uppercase entity code.
level, the numeric level from that entity's definition.
comboKey, the hierarchical key, which must end with the literal suffix |ENT:. The backend appends the new ID directly onto it, so a comboKey without that trailing terminator produces a malformed key. For a child record, include the parent before the terminator, for example SUB:<sub>|PJ:<project-id>|ENT:.

Everything else is entity-specific. id, ref, created, updated, ownerId, subscription and _df_version are all assigned by the server, so any values you send for them are ignored. The stream carries the created record, complete with its new id and ref, which you can then use for a DFQL GET, PATCH or DELETE. Creation fires the same pusher, automation and activity-stream side effects as creating the record in the UI.

Creating many at once. fields also accepts an array of objects, up to 500 per request, each validated and created independently:

{
  "query": {
    "type": "POST",
    "fields": [
      { "entity": "PPL",
        "level": 220,
        "comboKey": "SUB:<sub>|ENT:",
        "name": "Ada Lovelace" },
      { "entity": "PPL",
        "level": 220,
        "comboKey": "SUB:<sub>|ENT:",
        "name": "Grace Hopper" }
    ]
  }
}

Each created record is written to the stream as it completes, so results do not necessarily arrive in the order you sent them. Creates run in chunks of ten at a time rather than all at once, because every create increments its entity's reference counter, which is a single hot item.

A batch is not a transaction. One bad entry does not roll back or halt the rest: it produces an error record carrying the index of the entry that failed, and the remaining entries still go through. The batch closes with a summary record so you can reconcile without counting lines yourself:

{ "_type": "error", "index": 1, "message": "fields.entity is required" }
{ "_meta": true, "_type": "post", "created": 1, "failed": 3 }

The index field and the summary record appear only when you send an array. Passing a single object behaves exactly as before, one record out and nothing else.

The same operation is available on the REST API as POST /v1/objects, documented separately, if you are not already holding a DFQL stream open. That route creates one object per call.

PATCH, update an object

{
  "query": {
    "type": "PATCH",
    "id": "<object-id>",
    "entity": "PJ",
    "fields": { "status": "In Progress", "raglStatus": "Amber" }
  }
}

fields must be a non-empty object of the fields to change. id, created, updated, ownerId and entity are protected and rejected with an error record naming them. A successful patch also fires the automation runner, writes the activity stream entry, and pushes the change to other connected clients, exactly as an edit made in the UI does.

DELETE, remove an object

{ "query": { "type": "DELETE", "id": "<object-id>" } }

The object is loaded first and its subscription checked against yours, so a delete cannot cross tenants. On success the stream carries {"_type": "ok", "id": "..."}. The automation runner and client push notifications fire as they would for a UI delete. This deletes one object only, it does not cascade to children.

Practical notes

Parse line by line. Read the response as a stream and handle each newline-delimited object as it arrives. Buffering the whole body before parsing throws away the main benefit and, on large result sets, the memory headroom.
Check every line's _type. Errors and meta records share the stream with data.
Project with load. Result sets in the tens of thousands are normal; asking for four fields instead of the whole record is usually the single biggest win.
Prefer one MULTI to many calls. Parallel query lines in one request beat sequential round trips, and chaining removes the need to wait for a first response before issuing the second.
Use countOnly for badges and totals. It counts in the database and returns one line instead of the records.
Filters are post-read. A filter narrows what is written to you, not what is read from the database. Narrow the comboKey prefix first, then filter.