Token Usage Ingestion

Endpoint contract and operator workflow for importing local agent token usage into ruchern.dev.

The token usage ingestion workflow parses local coding-agent logs, prices the token usage, folds events into daily aggregates, and writes those aggregates into the token_usage table.

Parsing happens locally because agent logs live on the machine that ran the agents. Production writes happen through the deployed API route, so the production DATABASE_URL never has to be present on a local machine.

Endpoint

POST /api/usage/ingest

The endpoint is implemented in the web app and upserts rows into whichever database that deployment is configured to use.

apps/web/src/app/api/usage/ingest/route.ts

Authentication

Requests must be authorised by either:

Authorization: Bearer ${BLOG_MCP_AUTH_TOKEN}
An authenticated Better Auth session whose user role is admin

The static bearer token is the expected path for production ingestion.

Request Body

The body must match UsageIngestPayload from @workspace/usage/ingest.

{
  "rows": [
    {
      "date": "2026-06-02",
      "agent": "claude",
      "provider": "anthropic",
      "model": "claude-sonnet-4-5",
      "inputTokens": 1200,
      "outputTokens": 420,
      "cacheReadTokens": 3000,
      "cacheWriteTokens": 250,
      "reasoningTokens": 0,
      "totalTokens": 5870,
      "costUsd": "0.012345",
      "messages": 3
    }
  ]
}

Row Contract

Each row represents one daily aggregate for a specific (date, agent, provider, model) tuple.

Field	Type	Notes
`date`	`YYYY-MM-DD` string	Local calendar date for the aggregate.
`agent`	string	Coding tool that produced the logs, such as `claude`, `codex`, or `opencode`.
`provider`	string	Inference provider that billed the usage, such as `anthropic`, `openai`, or `fireworks-ai`.
`model`	string	Model identifier from the source log.
`inputTokens`	non-negative integer	Input token count.
`outputTokens`	non-negative integer	Output token count.
`cacheReadTokens`	non-negative integer	Cache-read input token count.
`cacheWriteTokens`	non-negative integer	Cache-write input token count.
`reasoningTokens`	non-negative integer	Reasoning output token count.
`totalTokens`	non-negative integer	Sum used by the usage page for token totals.
`costUsd`	decimal string or `null`	Fixed-point USD cost. `null` means the model could not be priced, not a real `$0`.
`messages`	non-negative integer	Number of source usage events folded into the row.

The request must include at least one row and at most 20,000 rows.

Idempotency

Ingestion is idempotent. Rows are upserted by the composite key:

date + agent + provider + model

Re-running ingestion recomputes the aggregates from local logs and overwrites existing rows for the same key. This keeps local logs as the source of truth.

Cache Revalidation

After a successful upsert, the route revalidates the usage cache tag so the public usage page can pick up the new data.

Local Ingestion

Use local ingestion to write directly to the database in DATABASE_URL.

pnpm usage:ingest

This is intended for a local development branch/database.

Production Ingestion

Use production ingestion to parse locally, then POST rows to the deployed endpoint.

pnpm usage:ingest:prod

Required environment variables:

BLOG_MCP_AUTH_TOKEN
VERCEL_PROJECT_PRODUCTION_URL or VERCEL_URL

The production database connection string stays inside the deployment environment. The local machine only sends the bearer token and JSON rows.

Example Request

curl -X POST "https://ruchern.dev/api/usage/ingest" \
  -H "Authorization: Bearer $BLOG_MCP_AUTH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "rows": [
      {
        "date": "2026-06-02",
        "agent": "claude",
        "provider": "anthropic",
        "model": "claude-sonnet-4-5",
        "inputTokens": 1200,
        "outputTokens": 420,
        "cacheReadTokens": 3000,
        "cacheWriteTokens": 250,
        "reasoningTokens": 0,
        "totalTokens": 5870,
        "costUsd": "0.012345",
        "messages": 3
      }
    ]
  }'

Success Response

{
  "ok": true,
  "upserted": 1
}

Failure Modes

Status	Cause
`401`	Missing or invalid bearer token, or non-admin session.
`400`	Request body does not match the ingest schema.
`500`	Database write or server-side ingestion failure.

Source Files

packages/usage/src/ingest.ts defines the wire schema.
apps/web/src/scripts/ingest-usage.ts parses, prices, folds, and posts rows.
apps/web/src/app/api/usage/ingest/route.ts authenticates and upserts rows.
apps/web/src/lib/queries/usage.ts performs the chunked upsert and usage profile aggregation.

On this page