Pipelines

Say a run gave you something you did not expect: a summary with no branches in it, or a route that paired with nothing. Nearly always, one step between your files and the output is the cause, and you should be able to find that step without reading the source. Each command below is traced end to end, with a run at every stage for you to compare yours against.

Architecture has the static package picture, and Cross-boundary checking explains what a finding means.

suss extract

suss extract turns a project into BehavioralSummary[].

The CLI parses the flags and hands off. @suss/adapter-typescript builds a ts-morph Project from the given tsconfig, walks the source files, and looks for discovery matches from each pack it was given. A discovery match identifies one code unit: a handler, a client call site, a loader.

Each matched unit then goes through the five extraction steps in Extraction algorithm. The adapter finds the terminals, enumerates the paths that reach each one, reads the conditions along each path, turns those conditions into predicates, and assembles a branch. It reads the contract too, when the pack declares one. What comes out per unit is a RawCodeStructure: plain data with no AST references, ready to be serialized or tested against a fixture.

@suss/extractor.assembleSummary then normalizes each RawCodeStructure into a BehavioralSummary. It wraps un-decomposed conditions as opaque and records a gap wherever the contract and the code disagree, or wherever a return didn't match any terminal pattern. Then it assesses confidence and assembles the summary. The CLI collects the array, parses it back through the IR validator as a sanity check, and writes it to disk.

Python and Ruby take the same route through @suss/adapter-python and @suss/adapter-ruby, which parse with tree-sitter instead of ts-morph and emit the same RawCodeStructure. --lang says which one to use, and when you leave the flag off suss works it out from what the directory contains.

Here it is over an Express and Prisma API, gothinkster/node-express-realworld-example-app, with --explain to print the counts behind the total:

suss extract -p tsconfig.app.json -f express -f axios -f prisma --explain -o summaries/code.json

  Where these came from:
    26  files in the tsconfig
    13  files read
     7  files importing express
    21  boundaries recognized by express
    21  summaries from express
    21  of those, summaries saying what express does
     0  files importing axios
     0  boundaries recognized by axios
     0  summaries from axios
     0  of those, summaries saying what axios does
    12  files importing @prisma/client
    21  unit bodies prisma could look inside
    79  effects prisma recognized

Pack health (1):
  no-files  axios  axios is installed and no file in this run imports it, directly or through a module of the project's own. Either this code does not use the library, or it reaches it some way the import gate does not follow.

Above that, extract prints one success line, Wrote 52 summaries to <path> in 1.48s, with the absolute path of the file it wrote and an elapsed time that moves from run to run.

Read the funnel from the top. 26 files were in the tsconfig and 13 survived the pre-filter, which skips a file when it imports nothing any pack is looking for. Seven of those import express, and express found 21 routes in them. The axios column is all zeroes, in the shape a pack that found nothing always prints, because this repository lists axios in its package.json and never calls it. Pack health says the same thing in one line. Prisma discovers no boundaries of its own, because it is made of recognizers: it looked inside the 21 units express found and recognized 79 database calls in them.

--timing shows where the time went, one row per phase, ordered by cost. The milliseconds differ on every run and the rows below a millisecond swap places, so read the shares rather than the numbers:

Timing:
     360ms   41.6%  preFilter
     355ms   41.1%  extract per-file
      82ms    9.5%  expandReachableClosure
      34ms    3.9%  loadImportGraphs
      16ms    1.9%  lazyProjectInit
       6ms    0.7%  mountPrefix
       5ms    0.6%  enrichRethrows
       2ms    0.2%  readTsconfigFileList
       2ms    0.2%  stampGraphqlClientRefs
       1ms    0.2%  deriveBoundaryEffects
       1ms    0.1%  stampModuleImports
       0ms    0.0%  synthesizeSubUnits
       0ms    0.0%  liftSchemasOntoDocuments
       0ms    0.0%  expandWrapperCallers
       0ms    0.0%  project.getSourceFiles
       0ms    0.0%  cache.merge
       0ms    0.0%  cache.write
       0ms    0.0%  emitLibraryEnvReadMarkers
       0ms    0.0%  warmExportChains
       0ms    0.0%  cache.lookup
     865ms  100.0%  total
  cache: miss (no-manifest)

In this run, deciding which files a pack could match costs about as much as extracting from the files that pass. The pre-filter reads the imports of all 26 files. When a file's own imports name nothing a pack looks for, it also follows that file's re-export chains, because a local barrel file can re-export the package a pack is looking for. The last line says whether the run reused a previous one; --no-cache forces the miss shown here.

The same run as a sketch, from the command down to the file it writes:

User
 │  suss extract -p tsconfig.json -f ts-rest -o out.json
 ▼
@suss/cli
 │
 ▼
@suss/adapter-typescript
 │  build ts-morph Project from tsconfig
 │
 │  for each source file:
 │    for each pack (ts-rest, axios, …):
 │      discovery.patterns → matched code units
 │      for each unit:
 │        run the 5 extraction steps (terminals, paths, conditions,
 │                                    predicates, assembly)
 │        produce RawCodeStructure
 │        │
 │        ▼
 │      @suss/extractor.assembleSummary(raw)
 │        wraps opaque predicates, detects gaps, assesses confidence
 │        ▼
 │      BehavioralSummary
 │
 ▼
BehavioralSummary[] → write out.json

The whole pipeline depends on the split between the two. The adapter does everything that touches the AST, and the extractor never sees a node. Adding a language means writing a new adapter that emits RawCodeStructure, and the extractor does not change. Python and Ruby were added that way.

suss inspect

suss inspect renders summaries for people to read.

suss inspect summaries.json loads the file, runs it through safeParseSummaries so any malformed JSON fails with a path-pointed error before anything renders, then formats each summary as a tree of transitions with their conditions, outputs and gaps. One summary out of a three-summary file, with the other two cut:

src/handler.ts
└─ GET /invoices/{id}  (ts-rest handler | line 8)
     Contract: 200, 404, 500
       if  !findInvoice()
         -> 404 { error }
       elif  findInvoice().voidedAt
         -> 200 { id, total, state }
       else
         -> 200 { id, total, state }
           + src/db.findInvoice →

       !! Declared response 500 is never produced by the handler

Three things in that block are notation, and they appear in every rendering.

A line starting + is an effect: something the branch does besides producing its output. + src/db.findInvoice → says this branch calls findInvoice, and the arrow says that callee has a summary of its own in the same run, so you can go and read it. The name has a path in front when the callee is in another file, so this one reads src/db.findInvoice instead of plain findInvoice. A query the branch runs itself prints the same way, as + reads postgresql:invoices.

A Reaches block appears under a handler when something it calls touches a store, a bus or another service. reads postgresql:invoices through findInvoice says the read happens inside findInvoice, so you can see what a request touches without reading down the chain of calls.

A line starting !! is a gap: something suss could not work out, written down where you can see it. This one is the contract promising a 500 that no branch produces. Gaps separate "there is nothing here" from "suss could not tell", so you do not mistake an empty answer for an all-clear.

suss inspect --diff before.json after.json and suss inspect --dir summaries/ are variants over the same load-and-parse plumbing. The first uses diffSummaries to compute added, removed and changed transitions per summary pair. The second uses pairSummaries to show which summaries face which, and which ones matched nothing.

Use --diff on a pull request. Add one branch to an Express route so admins get an extra field, read the route again into after/api.json, then compare it against the file from before the change:

suss inspect --diff before/api.json after/api.json

1 boundary changed: 2 outcomes.

~ serves GET /users/{id}  src/routes/users.ts::getUser  (2 outcomes)
  outcomes
    + responds 200 { id, name, role, admin }  when  db.findById() && db.findById().role === "admin"
    ~ was  responds 200 { id, name, role }  when  db.findById()
      now  responds 200 { id, name, role }  when  db.findById() && !(db.findById().role === "admin")

Changes by file

src/routes/users.ts
  ~ getUser

One added if made two changes. The admin case is new, and the plain 200 is the same response under a narrower condition: it is now the case where the user exists and is not an admin. A ~ line is a path that moved. When the status and the test stayed the same, it prints as one line with a marker on each field of the body that moved: a - on a field that went, a + and its type on one that arrived, and a ~ with both types on one whose type changed. Anything else prints as a was line and a now line, so a reader can tell a narrowed branch from a branch that went away.

The report is organized by boundary. It opens with how many boundaries moved and how many of their outcomes and effects went with them, then gives one block per route, queue consumer or Lambda. Inside a block, outcomes is what it returns and under what test, and effects is what a request touches on its way through: a line such as + reads postgresql:users through loadUser means this route now gets to a table it did not before, whichever function down the chain does the reading. The files with units that moved come last, with a unit's own lines written out when there are a few of them and counted when there are more.

A handler is paired with the one before it by its route, so a renamed handler on the same route pairs, and a route that moved to another path prints as removed and added. A client is paired by its own name under the route it calls, since several callers share one route, and its key reads client:GET /pet/{petId}::getPetById.

All three modes share one failure path. If safeParseSummaries reports issues, the CLI prints Invalid summary file <path>: <issue paths> and exits non-zero, and nothing renders.

suss check

suss check pairs providers with consumers and emits findings.

The CLI loads the files through safeParseSummaries, the same validation path inspect uses, then calls checkPair(provider, consumer). That runs seven independent check functions one after another and concatenates their findings:

• checkProviderCoverage: does the consumer handle every status code the provider produces? It also looks at sub-cases within one status, where the provider's branches are told apart by a predicate the consumer never tests.
• checkResponseMisread: does a consumer path read a body field off a response that does not include it, with nothing on that path telling that response apart from the one that does? The read comes back undefined and nothing throws.
• checkConsumerSatisfaction: does the consumer handle status codes the provider never produces? Those are dead branches.
• checkContractConsistency: is the handler's behavior consistent with the contract attached to it, a ts-rest responses block or an OpenAPI schema?
• checkConsumerContract: does the consumer read fields the declared contract never promised, so it depends on an undeclared implementation detail?
• checkBodyCompatibility: do the consumer's body-field reads line up with the bodies the provider produces, per status?
• checkSemanticBridging: does the provider produce a distinguishing literal or a field-presence discriminator that the consumer collapses into one branch?

Each check is pure over (provider, consumer), emits Finding[], and runs without reference to the other six. The findings are then rendered, human-readable or JSON, and the exit code comes from --fail-on: error, warning, info or none.

suss check --dir summaries/ is the same flow with a step in front. pairSummaries groups every summary by its boundary key and by its role from BOUNDARY_ROLE[kind], and produces matched pairs plus buckets of unmatched providers, unmatched consumers, and summaries that took no part. checkPair runs on each matched pair.

Over the same Express and Prisma API, with the schema read into the same folder by suss contract --from prisma src/prisma/schema.prisma -o summaries/prisma.json:

suss check --dir summaries/

Compared 7 of 37 boundaries.

  20 provider-side boundaries have no client to compare against.
  10 boundaries had nothing to pair with, so nothing was checked across them.
  Run the same command with --all to list them.

1 other summary is internal code with no boundary, so nothing pairs with it.

────────────────────────────────────────────────────────────
[WARNING] boundaryFieldUnused
  Comment declares "articleId" and code here writes to it, but no query reads it. suss counts a column as read only when a query selects it or filters on it, so before you treat the write as pointless, look for code that takes "articleId" off a record it already fetched.
  provider: src/prisma/schema.prisma::Comment (src/prisma/schema.prisma:1)
  consumer: src/prisma/schema.prisma::Comment (src/prisma/schema.prisma:1)
  boundary: prisma (postgresql)
────────────────────────────────────────────────────────────
[WARNING] boundaryFieldUnused
  Comment declares "authorId" and code here writes to it, but no query reads it. suss counts a column as read only when a query selects it or filters on it, so before you treat the write as pointless, look for code that takes "authorId" off a record it already fetched.
  provider: src/prisma/schema.prisma::Comment (src/prisma/schema.prisma:1)
  consumer: src/prisma/schema.prisma::Comment (src/prisma/schema.prisma:1)
  boundary: prisma (postgresql)
────────────────────────────────────────────────────────────
[WARNING] boundaryFieldUnused
  _ArticleToTag declares "A" and code here writes to it, but no query reads it. suss counts a column as read only when a query selects it or filters on it, so before you treat the write as pointless, look for code that takes "A" off a record it already fetched.
  provider: src/prisma/schema.prisma::_ArticleToTag (src/prisma/schema.prisma:1)
  consumer: src/prisma/schema.prisma::_ArticleToTag (src/prisma/schema.prisma:1)
  boundary: prisma (postgresql)
────────────────────────────────────────────────────────────
[WARNING] boundaryFieldUnused
  _ArticleToTag declares "B" and code here writes to it, but no query reads it. suss counts a column as read only when a query selects it or filters on it, so before you treat the write as pointless, look for code that takes "B" off a record it already fetched.
  provider: src/prisma/schema.prisma::_ArticleToTag (src/prisma/schema.prisma:1)
  consumer: src/prisma/schema.prisma::_ArticleToTag (src/prisma/schema.prisma:1)
  boundary: prisma (postgresql)
────────────────────────────────────────────────────────────
[WARNING] boundaryFieldUnused
  _UserFavorites declares "A" and code here writes to it, but no query reads it. suss counts a column as read only when a query selects it or filters on it, so before you treat the write as pointless, look for code that takes "A" off a record it already fetched.
  provider: src/prisma/schema.prisma::_UserFavorites (src/prisma/schema.prisma:1)
  consumer: src/prisma/schema.prisma::_UserFavorites (src/prisma/schema.prisma:1)
  boundary: prisma (postgresql)
────────────────────────────────────────────────────────────
[WARNING] boundaryFieldUnused
  _UserFavorites declares "B" and code here writes to it, but no query reads it. suss counts a column as read only when a query selects it or filters on it, so before you treat the write as pointless, look for code that takes "B" off a record it already fetched.
  provider: src/prisma/schema.prisma::_UserFavorites (src/prisma/schema.prisma:1)
  consumer: src/prisma/schema.prisma::_UserFavorites (src/prisma/schema.prisma:1)
  boundary: prisma (postgresql)
────────────────────────────────────────────────────────────
[WARNING] boundaryFieldUnused
  _UserFollows declares "A" and code here writes to it, but no query reads it. suss counts a column as read only when a query selects it or filters on it, so before you treat the write as pointless, look for code that takes "A" off a record it already fetched.
  provider: src/prisma/schema.prisma::_UserFollows (src/prisma/schema.prisma:1)
  consumer: src/prisma/schema.prisma::_UserFollows (src/prisma/schema.prisma:1)
  boundary: prisma (postgresql)
────────────────────────────────────────────────────────────
[WARNING] boundaryFieldUnused
  _UserFollows declares "B" and code here writes to it, but no query reads it. suss counts a column as read only when a query selects it or filters on it, so before you treat the write as pointless, look for code that takes "B" off a record it already fetched.
  provider: src/prisma/schema.prisma::_UserFollows (src/prisma/schema.prisma:1)
  consumer: src/prisma/schema.prisma::_UserFollows (src/prisma/schema.prisma:1)
  boundary: prisma (postgresql)
────────────────────────────────────────────────────────────
8 findings: 0 error, 8 warning, 0 info

suss met a call it could not follow in 19 units, of 59, so those are described in part. `suss inspect` says which calls.

The seven boundaries compared are the four Prisma models and the three tables Prisma adds for many-to-many relations, each against every query that reads or writes it. The 20 uncompared providers are the HTTP routes: the front end for this API is in another repository, so nothing in this run is on the other side of them. Nine of the ten that paired with nothing are function-call:reachable helpers, functions a route or the seed script reaches and no file imports, so no boundary key addresses them. The tenth is the app.use call in main.ts. Each of those lines gives a different reason nothing was compared, and the run reports each one separately. The eight warnings are foreign-key columns that code writes and no query selects or filters on.

The grouping in pairSummaries reads the method and the path and nothing else, so a store, a queue and a runtime's configuration all come back unpaired from it. Each of those has a pass of its own, and each records what it compared into the same pairs list. checkAll then drops those from the unmatched buckets, so one table is never reported as compared and unpaired in the same run.

A boundary's semantics is what kind of meeting point it is: a REST route, a GraphQL field, a queue subject, a database table. Each kind declares two things: the key its two sides pair by, and what counts as those two sides agreeing. The pairing code gets both from the semantics and does not branch on the protocol, so adding a kind of boundary never touches it. See Boundary semantics.

suss contract --from openapi

suss contract --from openapi turns an OpenAPI 3.x document into BehavioralSummary[] marked confidence.source: "derived". The output is in the same form suss extract produces, and it pairs with extracted consumers.

@suss/contract-openapi walks every (path, operation) in the document. For each operation it emits one handler summary with one transition per declared response, and each transition has the status code plus the body schema converted to a TypeShape. The summary gets metadata.http.declaredContract, so checkContractConsistency can cross-check a provider you extract later, and confidence.source: "derived", so a downstream reader can tell where it came from.

Over the Petstore document in examples/petstore-axios-openapi, suss contract --from openapi openapi.json -o out/provider.json writes 19 summaries, one per operation. Reading them back:

suss inspect out/provider.json

The first two of the nineteen, with the rest cut:

openapi:openapi.json
├─ PUT /pet  (openapi handler | line 0)
│    Contract: 200, 400, 404, 422, default
│      -> 200 { id, name, category, photoUrls, ... }
│      -> 400
│      -> 404
│      -> 422
│      -> default
│
├─ POST /pet  (openapi handler | line 0)
│    Contract: 200, 400, 422, default
│      -> 200 { id, name, category, photoUrls, ... }
│      -> 400
│      -> 422
│      -> default

The run ends with 19 summaries. When a route goes missing, compare that number against the operation count in your own document.

The lines with no shape after the status are the responses Petstore declares with no schema. They still become transitions, because a caller has to handle a 400 whether or not anybody wrote down what is in it. These summaries came out of a document rather than out of source, so there is no line in a source file to point at and the location reads line 0.

The CLI writes the result to disk after round-tripping it through safeParseSummaries, to catch any drift in the structure.

suss contract --from cloudformation

This is the most layered of the contract readers, because one physical API can be written several ways in CloudFormation and all of them should produce the same summaries.

It has three layers, kept separate on purpose. Parsing the raw template, turning a file on disk into plain data and resolving CloudFormation's intrinsic YAML tags, lives in @suss/manifest-aws, shared with the manifest-driven framework packs. On top of it, the manifest-reader layer in @suss/contract-cloudformation walks the parsed tree and builds normalized RestApiConfig and HttpApiConfig values: which API this is, which endpoints, which authorizer, CORS, throttle, and integration config. That happens in buildRestApiConfigs, buildHttpApiConfigs, readSamApiEvents, readSamHttpApiEvents and readCors, and it is pure grouping. It handles AWS::ApiGateway::RestApi plus AWS::ApiGateway::Method, AWS::ApiGatewayV2::Api plus Route plus Integration, the SAM AWS::Serverless::Api and AWS::Serverless::HttpApi shorthand, and SAM Events.Api and Events.HttpApi blocks. It also handles an inline OpenAPI body on a RestApi.

The resource-semantics layer turns each normalized config into BehavioralSummary[] with the transitions the platform injects: 401 and 403 from an authorizer, 403 from an API key, 400 from a request validator, 429 from a throttle, 502 and 504 from the integration, and an OPTIONS preflight for CORS. That logic is restApiToSummaries and httpApiToSummaries in @suss/contract-aws-apigateway, which you can use on its own, so a hand-authored API Gateway path with no CloudFormation involved would go straight into the semantics layer. When the manifest has an inline OpenAPI body, the CloudFormation reader hands that part to @suss/contract-openapi instead.

Over the SAM template in fixtures/aws-lambda:

suss contract --from cloudformation fixtures/aws-lambda/template.yaml -o cfn.json
suss inspect cfn.json

That template gives 31 summaries, and inspect prints all of them, about a hundred lines. Six are routes, and each looks like this:

cloudformation:fixtures/aws-lambda/template.yaml:ListWidgetsFunction:List
└─ GET /widgets  (apigateway handler | line 0)
     Contract:
       if  aws:apigateway:status-504
         -> 504  !! undeclared
       elif  aws:apigateway:status-502
         -> 502  !! undeclared

Neither status is in the Lambda. API Gateway produces the 504 on an integration timeout and the 502 on an integration failure, and a caller receives both the same as any other response. The template is the only place they are written down, and this reader exists to read them from there. Contract: is empty because this route declares no responses of its own, and !! undeclared on each line says the same thing from the other side: the declaration never mentions the status.

The other 25 are the template's own resources, printed as one tree under the template's name. Here is where that tree starts:

cloudformation:fixtures/aws-lambda/template.yaml
├─ runtime-config:ConfirmTokenFunction  (cloudformation library | line 17)
│
├─ runtime-config:ListWidgetsFunction  (cloudformation library | line 28)
│
├─ runtime-config:WidgetItemFunction  (cloudformation library | line 40)

and where it ends:

├─ MixedTriggerFunction.Drain → aws_sqs JobsQueue  (cloudformation consumer | line 162)
│
└─ MixedTriggerFunction.Sweep → eventbridge schedule:MixedTriggerFunction.Sweep  (cloudformation consumer | line 162)

31 summaries.

A Lambda becomes a library summary for the environment the template gives it, printed as runtime-config:<function>, and a queue becomes one too. The wiring between them becomes a consumer summary that says which function reads which queue, and one of those reads OrderIndexerFunction.Orders → aws_sqs default#order.placed: the template routes that EventBridge subject into a queue, and that queue feeds that Lambda, so the reader followed the rule to the queue to the function. The rule's delivery into each queue is a summary of its own, OrderPlacedRule.indexer#order.placed, which writes bus:aws_sqs OrderIndexerQueue. Each summary's line is the line its resource starts on. The 31 summaries. line at the end is the count to check when a resource you expected goes missing.

The three layers, and why each one is separate:

See Contract sources for the reasoning behind this split and the opaque-predicate naming convention for transcribed external contracts.

Internal: RawCodeStructure → BehavioralSummary

assembleSummary runs one level below suss extract. It reads the raw branches and produces one Transition per branch. Structured predicates pass through, and a condition it could not take apart is wrapped as opaque, which keeps the source text and marks the branch as one suss read but did not understand.

Each transition gets an ID built from (function, terminal kind, status, conditionHash). Because the condition is hashed into the ID, reordering two branches changes nothing, and rewriting one condition counts as a change. The --diff above shows that behaviour.

Then it looks for gaps, and there are two kinds. An unhandledCase gap means the contract and the code disagree, in either direction: a declared response the handler never produces, or a produced response the contract never declared. It describes the code, and the checker reports it as an error. An unreadOutcome gap means a return matched none of the pack's terminal patterns, so suss could not tell what that path produces. It describes what suss could read. It forces confidence to low, and the checker reports it as info, because failing a build over what the analyzer could not read would punish working code.

With no unreadOutcome gap, confidence comes from the ratio of opaque predicates to structured ones. Finally it assembles the summary object, nesting any HTTP-scoped metadata under metadata.http.* per the Boundary semantics namespacing convention.

Every step here is pure over RawCodeStructure and testable on its own, so the extractor test suite runs in milliseconds and takes no compiler dependency.

Internal: cross-boundary pairing

Before suss check --dir runs checkPair, it has to work out which summaries face each other. pairSummaries does that in three passes.

1. Bucket each summary. pairingKey(binding) gives the bucket. A summary with no boundary binding at all goes to unmatched.unpairable with the reason noBoundary. A binding whose semantics declares no key goes there too, with unnamedBoundary. BOUNDARY_ROLE[summary.kind] then says which side it is on: provider for a handler, loader, action, middleware, resolver, worker, component or hook, and consumer for a client or a consumer. An unrecognized kind goes to unpairable with unknownKind, and that guard stays until the zod IR migration makes it unreachable.

2. Within a bucket, decide the rest. Sharing a key is not enough on its own: two REST sides can share a normalized path and use different methods, and two message-bus sides can share a subject and use different buses. semanticsAgree decides that, per semantics variant. A consumer that several services all serve produces an ambiguousProvider finding and no pair, because pairing it with one of them would compare a caller against a handler it may never reach.

3. Collect what is left. A key with providers and no consumers populates unmatched.providers, and the other way round for unmatched.consumers.

The result is { pairs, unmatched }. checkPair runs on each pair, and the unmatched lists reach the CLI output. That is where the "20 provider-side boundaries have no client" line in the run above comes from.

The key function and the agreement check both come from the binding's semantics variant, imported from @suss/ir-core. REST buckets by path and decides the method inside the bucket, GraphQL pairs by the parent type name plus the field, and message-bus pairs by the channel's subject. See Boundary semantics.