Speed
Where the time goes between a message and the first command, and how to measure it.
Getting from a message to the first command takes four steps: accept the message, start the model, get a sandbox ready and run the command. Pistis times each step, so you can see where the time goes on your own deployment and check it with one command.
/metrics: accepting the message, passing it between servers, starting the model, the first output, getting the sandbox ready, the round trip to the sandbox, the command and the update in the browser. Watch the p95, not averages.What is timed
| Metric | Measures |
|---|---|
pistis_admit_seconds | A message received, to saved |
pistis_fleet_hop_seconds | A call passed to the server handling the session |
pistis_dispatch_seconds | A message saved, to the model request |
pistis_model_first_token_seconds | The model request, to its first output |
pistis_sandbox_lease_seconds | Getting the session's sandbox ready, by how: spare, already connected, resumed, restarted, replaced or new |
pistis_hands_rtt_seconds | One round trip from Pistis to the sandbox |
pistis_tool_call_seconds | A command, end to end |
pistis_follow_lag_seconds | A change saved, to its appearing in the browser |
More sessions don't slow it down
Accepting a message reads only that session's record. We timed 30 messages with 3 sessions in the database and again with 303: median 20.2 ms → 17.4 ms.
Spare sandboxes
sandboxes.max.Set sandboxes.warm to keep that many sandboxes started and connected per server. A new session takes one at once, and Pistis starts a replacement.
In a test with two spares: 12 sessions leased warm; 12/12 under 300 ms (p95 < 300 ms); the pool refilled to 2. All twelve fell in the fastest bucket: pistis_sandbox_lease_seconds_bucket{path="warm",le="0.1"} 12. After a restart, Pistis kept the same spares: same 14 containers, none made or lost.
Postgres and SQLite
One server on SQLite saves in-process. Several servers on Postgres pay a network round trip per statement, so handing a message to the model is slower there. On a two-server deployment it measured dispatch p95 ≤ 0.1s, within its target.