Skip to content

Bound the input at the framing seam, not the projected output #826

Description

@ddeboer

Fan-out (ADR 26, #798) turns one qualified edge into the cartesian product of its weldable leaves. That product grows with the edge’s own values, which ADR 12 says is no bound at all. It needs one; ADR 26 deliberately does not supply it.

Why not cap the entries

The first attempt did exactly that – a maxEntries option on the inline reference, default 100 – and it was removed before merge. Not because it failed, but because it guards the wrong end.

By the time the projection runs, every cost has already been paid. The CONSTRUCT matched those values and the endpoint paid for it, they crossed the wire, buildSubjectIndex holds them, and frameSubjects has materialised them into one framed node. Capping the product declines the last and cheapest step while keeping all the expensive ones – and it bounds nothing that framing did not already hold.

It also missed the case that actually predates fan-out: a wide edge, or a display-only nesting of ten thousand entries, is linear rather than multiplicative and was never bounded at all. An entry cap does nothing for either.

What to do instead

Cap values per leaf where the data enters memory – buildSubjectIndex, per (subject, predicate) – so one bound protects the subject index, the framed node, the projection, the product and the engine import.

The product then becomes k^(weldable leaves). Leaf count is a schema constant, so that is a bound in the schema’s own units, rather than a number written against the data’s. Realistic k is small: a role stated per language is 2–5, endpoints per edge 1–3, so a cap in the tens is generous.

What needs deciding

  • k is a shared-seam decision. Framing serves every consumer, not just the weld, so truncating there shortens display lists and any other multi-valued field. That is a semantic change, not a tuning knob, and wants its own ADR.
  • Whether it can move further upstream. Bounding in the CONSTRUCT itself would avoid fetching the values at all, but SPARQL has no per-group LIMIT without a subquery per property, which carries its own cost. Worth measuring before ruling it out.
  • Whether truncation should be reported. The projection has no diagnostic channel today; the framing seam may be a better place for one.

Until then

Fan-out is unbounded, exactly as the entry list it replaced always was. What changed is that the growth can now be multiplicative rather than linear – which is the reason to do this rather than assume it is fine.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions