Operations

Webhooks, OpenTelemetry, status monitoring, config management, diagnostics.

Doctor

Run stockyard doctor before first start to verify your environment.

$ stockyard doctor

  ✓ Go runtime         go1.22.5
  ✓ Config             port=4200
  ✓ Data directory     ./stockyard-data (writable)
  ✓ SQLite database    connected (58 modules)
  ✓ Port 4200          available
  ✓ OpenAI API key     set
  ✓ Anthropic API key  set
  ⚠ Gemini API key     not set
  ✓ Ollama             detected (localhost:11434)
  ⚠ License key        not set (Community tier)
  ✓ Disk space         writable

  9 passed · 2 warnings · 0 failures

Use stockyard doctor --json for machine-readable output in CI pipelines.

Webhooks

Register HTTP endpoints to receive events from Stockyard: alerts firing, cost thresholds, trust policy violations, and more.

# Register a Slack webhook
curl -X POST localhost:4200/api/webhooks \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://hooks.slack.com/services/YOUR/SLACK/URL",
    "secret": "my-signing-secret",
    "events": "alert.fired,cost.threshold,trust.violation"
  }'

# List registered webhooks
curl localhost:4200/api/webhooks

# Send a test event
curl -X POST localhost:4200/api/webhooks/test

# Delete a webhook
curl -X DELETE localhost:4200/api/webhooks/1

Event Types

EventFired When
alert.firedAn Observe alert threshold is exceeded
cost.thresholdDaily cost exceeds a configured limit
trust.violationA Trust policy blocks or warns on a request
error.spikeError rate exceeds normal baseline
webhook.testManual test via POST /api/webhooks/test

HMAC Signatures

If you set a secret, every delivery includes an X-Stockyard-Signature header with sha256=HMAC of the JSON payload. Verify this on your server to confirm the webhook came from Stockyard.

Auto-disable

Webhooks that fail 10 consecutive deliveries are automatically disabled. Check fail_count in the list response.

OpenTelemetry Export

Export every proxy request as an OTLP trace span. Works with Jaeger, Grafana Tempo, Datadog, Honeycomb, or any OTLP-compatible backend.

# Set the collector endpoint
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

# Optional: service name (default: "stockyard")
export OTEL_SERVICE_NAME=my-stockyard

# Optional: auth headers
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer my-token"

# Start Stockyard — traces flow automatically
stockyard

Span Attributes

AttributeDescription
llm.modelModel name (e.g., gpt-4o-mini)
llm.providerProvider name (e.g., openai)
llm.tokens.totalTotal tokens used
llm.cost.usdEstimated cost in USD
llm.latency.msRequest latency in milliseconds
llm.statusok, error, or cache_hit

Spans are batched (100 per batch, 5s flush interval) and flushed gracefully on shutdown.

Status Monitoring

Real-time system health at GET /api/status:

curl localhost:4200/api/status | jq .
{
  "status": "healthy",
  "uptime": "2d 5h 30m",
  "total_requests": 12453,
  "error_rate": 0.002,
  "avg_latency_ms": 245.3,
  "memory": { "alloc_mb": 8.2, "sys_mb": 24.1 },
  "goroutines": 12,
  "components": {
    "database": { "status": "healthy", "detail": "1.2ms" },
    "modules": { "status": "healthy", "detail": "58 registered" },
    "observe": { "status": "healthy", "detail": "12,453 traces" },
    "trust": { "status": "healthy", "detail": "12,453 events" }
  }
}

The web status page at /status/ auto-refreshes every 30 seconds.

Config Export & Import

Snapshot your entire Stockyard configuration and restore it on another instance.

# Export current config
curl localhost:4200/api/config/export > stockyard-config.json

# Preview changes before applying
curl -X POST localhost:4200/api/config/diff \
  -d @stockyard-config.json

# Import on another instance
curl -X POST localhost:4200/api/config/import \
  -d @stockyard-config.json

The export includes module states, webhooks, and trust policies. Provider API keys are never included in exports.

API Key Rotation

Rotate API keys without downtime — the old key is immediately revoked and a new one generated:

# Rotate your own key
curl -X POST localhost:4200/api/auth/me/keys/1/rotate \
  -H "Authorization: Bearer sk-sy-current-key"
# Returns: { "status": "rotated", "new_key": { "key": "sk-sy-new-...", ... } }

# Admin: rotate any user's key
curl -X POST localhost:4200/api/auth/users/1/keys/1/rotate