Audit
Audit API
Trace model, query API, signed decision receipts, offline verification, compliance evidence bundles.
13 min read
How the forensic audit vault records every request, how to query it, and how to verify a decision receipt offline on an auditor's laptop.
TL;DR
- Every request becomes a
SessionTrace— an ordered list of typedTraceSteps, plus aggregate counters, plus an HMAC-SHA256 signature. - Traces are queryable via REST: list with filters, fetch by ID, verify signature, dashboard analytics.
- Every
BLOCKorMODIFYsecurity verdict can be turned into a signed decision receipt — a small JSON blob an auditor can verify offline with a one-line CLI. - Compliance evidence bundles map Verosek's checks onto NIST AI RMF and EU AI Act articles, populated from your actual trace data.
Trace model
A SessionTrace row lives in Postgres (session_traces table). Each row captures one request lifecycle end-to-end.
Row-level fields:
id "vsk_tr_..." + 16 hex chars
agent_id String — agent identifier for this key
key_id String — secret key id (never returned in API responses; only the tenant sees it indirectly via key_ref)
user_context_hash SHA-256 of the user context object, never plaintext
model_provider String — which provider backend the request ran on
started_at / ended_at datetime (UTC)
steps JSONB array of TraceStep objects
peak_risk_score float — max step-level score
total_interventions int — count of MODIFY / BLOCK / TERMINATE decisions
decisions_allow / _modify / _hold / _block / _terminate int counters
total_prompt_tokens / total_completion_tokens / total_tokens / llm_rounds int
total_tool_calls / total_tool_duration_ms / total_duration_ms int
cost_usd float (USD)
verdict COMPLETED | WITH_INTERVENTIONS | BLOCKED | TERMINATED | IN_PROGRESS
signature HMAC-SHA256 hex digest over the trace
Step types
Every trace step is one of the following types. Each carries the subset of fields relevant to its kind.
| Type | When it's emitted | Key fields |
|---|---|---|
REQUEST_RECEIVED | Start of each request | input_messages, request_model, has_tools, tool_count, tool_names |
LLM_CALL | Each tool-loop round that called the LLM | model, round_number, prompt_tokens, completion_tokens, total_tokens, finish_reason, provider_url |
TOOL_PROPOSED | Model proposed a tool call | tool_name, tool_call_id, arguments_hash, arguments_preview, connector_id |
POLICY_DECISION | Tool-access rule was evaluated | policy_decision, access_config_snapshot, connection_alias, original_tool_name |
SECURITY_SCAN_INPUT | Shield pre-LLM scan produced verdicts | policy_decision (verdicts packed as triggered_checks), duration_ms |
SECURITY_SCAN_OUTPUT | Shield post-LLM scan produced verdicts | Same shape as SECURITY_SCAN_INPUT |
SECURITY_BLOCKED | Shield blocked the request | Same shape; policy_decision.decision = BLOCK, block_reason populated |
TOOL_EXECUTED | MCP tool ran | rows_returned, data_volume_bytes, success, connector_mode, result_preview, is_error |
LLM_RESPONSE | Final text returned to the caller | response_preview, response_tokens |
SESSION_END / ERROR | Session closed / errored | error_message |
arguments_preview is a sanitized 200-char preview; the full arguments are hashed into arguments_hash and never stored in plaintext. result_preview is the first 500 chars of the tool result. response_preview is the first 300 chars of the final text.
policy_decision on SECURITY_* steps is a PolicyDecision whose triggered_checks array contains every Shield verdict — including ALLOW verdicts — so a shadow-mode scan that found nothing is still visible as a scan that ran.
Query API
All endpoints live under /api/v1/traces.
List traces
GET /api/v1/traces
Query parameters:
| Name | Type | Default | Description |
|---|---|---|---|
agent_id | string | — | Filter by agent id |
key_id | string | — | Filter by virtual-key internal id |
start, end | datetime | — | Window on started_at |
verdict | string | — | COMPLETED / WITH_INTERVENTIONS / BLOCKED / TERMINATED |
min_score | float | — | Minimum peak_risk_score |
limit | int | 50 (max 200) | Page size |
offset | int | 0 (max 100_000) | Paging offset |
Response is a JSON array of summary rows (no step-level detail):
[
{
"id": "vsk_tr_abc123...",
"agent_id": "agt_...",
"key_id": "vsk_...",
"model_provider": "openai",
"started_at": "2025-01-01T12:00:00Z",
"ended_at": "2025-01-01T12:00:01.234Z",
"peak_risk_score": 0.0,
"total_interventions": 1,
"decisions_allow": 4,
"decisions_modify": 1,
"decisions_block": 0,
"total_prompt_tokens": 320,
"total_completion_tokens": 128,
"total_tokens": 448,
"llm_rounds": 2,
"total_tool_calls": 1,
"total_duration_ms": 1234,
"cost_usd": 0.0042,
"verdict": "WITH_INTERVENTIONS",
"signature": "a1b2c3..."
}
]
Fetch a trace by id
GET /api/v1/traces/{trace_id}
Returns the full trace record including every TraceStep in order. 404 if not found.
Verify signature
GET /api/v1/traces/{trace_id}/verify
Recomputes the HMAC over the stored trace and compares to the stored signature. Response:
{
"trace_id": "vsk_tr_abc123...",
"valid": true,
"detail": "Signature valid"
}
Analytics summary
GET /api/v1/traces/analytics/summary?hours=24
Aggregate KPIs for the dashboard. Either pass hours (1–720, default 24) or start+end for a custom window.
Response shape:
{
"period_hours": 24,
"total_requests": 1234,
"verdict_distribution": { "COMPLETED": 1200, "WITH_INTERVENTIONS": 30, "BLOCKED": 4 },
"tokens": { "prompt": 500000, "completion": 120000, "total": 620000 },
"decisions": { "allow": 5400, "modify": 120, "hold": 0, "block": 4, "terminate": 0 },
"tools": { "total_calls": 340, "total_duration_ms": 9120, "total_llm_rounds": 1480 },
"avg_latency_ms": 1045.3,
"risk_distribution": { "low": 1210, "medium": 20, "high": 3, "critical": 1 }
}
Decision receipts
A decision receipt is a small JSON blob produced for every security verdict that blocked or modified a request (i.e. decision != "ALLOW"). It is HMAC-signed. An auditor can verify it offline without talking to Verosek.
What triggers a receipt
- Any Shield verdict where
decision ∈ { BLOCK, MODIFY, TERMINATE, HOLD }produces one receipt. ALLOWverdicts do not produce receipts.
Downloading receipts for a trace
GET /api/v1/security/receipts/{trace_id}
Response:
{
"trace_id": "vsk_tr_abc123...",
"count": 2,
"receipts": [
{
"receipt_version": 1,
"verosek_gateway_version": "1.0.0",
"receipt_id": "rcpt_...",
"trace_id": "vsk_tr_abc123...",
"key_id_masked": "vsk_****_1234",
"timestamp_utc": "2025-01-01T12:00:00.123Z",
"decision": "BLOCK",
"check_id": "CHK-013",
"confidence": 0.9672,
"scanner_name": "prompt_injection",
"input_hash": "sha256:e3b0c44...",
"signature_algorithm": "HMAC-SHA256",
"signature": "a1b2c3d4..."
},
{
"receipt_version": 1,
"verosek_gateway_version": "1.0.0",
"receipt_id": "rcpt_...",
"trace_id": "vsk_tr_abc123...",
"key_id_masked": "vsk_****_1234",
"timestamp_utc": "2025-01-01T12:00:00.167Z",
"decision": "MODIFY",
"check_id": "CHK-015",
"confidence": 0.912,
"scanner_name": "pii",
"input_hash": "sha256:e3b0c44...",
"signature_algorithm": "HMAC-SHA256",
"signature": "e4f5g6h7..."
}
]
}
Optional fields that appear when provided: model_name, model_version_sha, matched_span_hash, policy_snapshot_hash. Raw matched text is never included — only a hash.
HMAC signature design
Every trace and every decision receipt is signed with HMAC-SHA256. The key is the tenant master key (HMAC key) that we establish during onboarding.
What the signature covers. For both traces and receipts, the signable payload is a deterministic JSON serialisation of the object with the signature field omitted. The serialiser is json.dumps(payload, sort_keys=True, separators=(",", ":")), which makes the signature reproducible in any language with a stdlib JSON + HMAC-SHA256 implementation.
Algorithm. HMAC-SHA256. Constant-time comparison on verify (hmac.compare_digest).
Versioning. Receipts carry a receipt_version field (currently 1). If we ever change the signable field set or the algorithm we bump the version and maintain legacy verifiers.
Key rotation. Performed under our operational control. On Verosek Cloud rotation is handled by us. On on-prem deployments it is coordinated with your infrastructure team during an upgrade window.
Onboarding-only
Handled during onboarding — not public. The rotation procedure, the old-key overlap window, and the storage layout for the master key are intentionally not published on the public docs.
Offline verification
An auditor can verify a downloaded receipt without network access using the verosek-verify-receipt CLI. The CLI has no gateway dependency — it is stdlib-only and ships as a console script in the verosek Python package on PyPI.
Invocation
export VEROSEK_MASTER_KEY=<the master key issued at onboarding>
verosek-verify-receipt path/to/receipt.json
# Or pass the key inline:
verosek-verify-receipt path/to/receipt.json --key <master-key>
Exit codes
| Code | Meaning |
|---|---|
0 | Signature is valid |
1 | Signature mismatch or unsupported version |
2 | VEROSEK_MASTER_KEY not set and no --key provided |
3 | Receipt file could not be loaded (missing / malformed JSON) |
Sample output (valid receipt)
$ verosek-verify-receipt receipt.json
VALID
receipt_id: rcpt_abc123defg4567
trace_id: vsk_tr_abc123def456789
check: CHK-013 (BLOCK)
timestamp: 2025-01-01T12:00:00.123Z
Alternative — verify over HTTP
If an auditor has network access and prefers a web tool:
POST /api/v1/security/receipts/verify
Content-Type: application/json
<the receipt JSON>
Response:
{ "valid": true, "detail": "" }
Retention
Trace rows persist in Postgres on your tenant. The default retention window and any tier-specific policies are agreed during onboarding.
Onboarding-only
Handled during onboarding — not public. No on-disk retention-policy knob is exposed publicly.
Compliance evidence bundles
Two frameworks are supported today. Both draw on real audit data over a specified reporting window.
List frameworks
GET /api/v1/security/compliance/frameworks
Response:
{
"frameworks": [
{ "id": "nist_ai_rmf", "name": "NIST AI Risk Management Framework (AI RMF 1.0)", "description": "Govern / Map / Measure / Manage functions for AI risk management." },
{ "id": "eu_ai_act", "name": "EU AI Act", "description": "Articles 9, 10, 12, 13, 14, 15 coverage for high-risk AI systems." }
]
}
Generate a report
GET /api/v1/security/compliance/{framework_id}?hours=168&customer_name=Acme+Corp
Parameters:
framework_id—nist_ai_rmf|eu_ai_acthours— reporting window, 1–8760 (default 168 = 7 days)customer_name— shown in the report header (max 200 chars)
Response shape (NIST example — trimmed for brevity):
{
"framework_id": "nist_ai_rmf",
"framework_name": "NIST AI Risk Management Framework (AI RMF 1.0)",
"customer_name": "Acme Corp",
"reporting_period_hours": 168,
"generated_at_utc": "2025-01-08T12:00:00Z",
"functions": [
{
"id": "GV",
"name": "Govern",
"description": "Policies, processes, procedures, and practices for the mapping, measuring, and managing of AI risks.",
"verosek_contribution": "Admin UI provides per-key security profiles, tamper-evident audit traces, and signed decision receipts that demonstrate policy enforcement.",
"checks_covered": ["profile_system", "audit_vault", "decision_receipts"]
},
{
"id": "MP",
"name": "Map",
"description": "Context for AI system operation, including interrelationships and risk considerations.",
"verosek_contribution": "MCP connector catalog + per-connection security profiles document the data flows between the AI system and external tools. The audit trace captures the full lineage of each decision.",
"checks_covered": ["connector_catalog", "per_connection_profiles"]
},
{
"id": "MS",
"name": "Measure",
"description": "Analysis, assessment, benchmarking, and monitoring of AI risks and related impacts.",
"verosek_contribution": "12 active security checks scan every request, producing structured verdicts recorded in the audit vault with HMAC signatures.",
"checks_covered": ["CHK-013", "CHK-014", "CHK-015", "CHK-016", "CHK-017", "CHK-018", "CHK-019", "CHK-020", "CHK-021", "CHK-022", "CHK-023", "CHK-024"],
"metrics": {
"total_requests_audited": 1234,
"total_shield_scans": 4920,
"total_blocked": 4,
"total_modified": 30
}
},
{
"id": "MG",
"name": "Manage",
"description": "Allocation of risk resources to mapped and measured risks on a regular basis.",
"verosek_contribution": "Per-check enforcement modes (off / log_only / enforce), per-key profiles, and decision receipts enable incremental rollout of security controls. CHK-022 session drift actively manages ongoing attacks.",
"checks_covered": ["profile_modes", "CHK-022_drift_detection"]
}
],
"disclaimer": "Verosek is not a certified auditor. This report is an evidence bundle for review by a qualified compliance professional. The framework mappings reflect the security controls implemented by Verosek Shield — they do not constitute a compliance certification."
}
The EU AI Act response has the same envelope but maps onto Articles 9, 10, 12, 13, 14, 15 in place of functions.
What's next
Read Policy-as-code to see how to export your Shield profile to YAML, review it in git, and import it back with validation.