guarded-api-sample

A small, self-contained sample for the Featherless guarded API. It shows the two pieces a customer builds around the guard:

 client.ts ──/v1/chat/completions──▶ shim ──▶ api.featherless.ai
 (openai SDK)                         │
                                      └─ pre/post webhooks ─▶ webhook-server.ts
                                         ◀── allow │ block ──┘   (your guard)

• client.ts — a stock OpenAI SDK client pointed at the shim. Its key, shim URL, and model are hardcoded in a SETTINGS block; runs a single message and a full tool-calling cycle (npm run client / client:single / client:tools).
• webhook-server.ts — a minimal customer webhook whose decision on each hook is hardcoded in its SETTINGS block (the deny/allow matrix below).
• start-webhook.sh — runs the webhook server + ngrok, then prints the public URL to register.

The shim itself is a separate service — the guarded-API proxy (the guarded-api-cloudflare repo). This repo is just the client and the webhook guard that sit on either side of it. Point the client at wherever the shim runs with SHIM_BASE_URL.

The webhook returns one decision per hook, fixed by two env vars:

A deny becomes { "action": "block", "reason": … } on the wire; an allow becomes { "action": "allow" }. The shim turns a block into a 400 carrying the reason. (The contract also supports a modify action; this sample keeps to allow/deny.)

npm install

You also need:

• A Featherless API key (rc_…) with access to a tool-capable model.
• ngrok installed and authed: brew install ngrok then ngrok config add-authtoken <token> (free account at ngrok.com).
• A running shim to point at (the guarded-api-cloudflare service).

Three terminals: the webhook (+ngrok), the shim, and the client.

npm run webhook

# or pin a reserved ngrok domain so the URL never changes between runs:
NGROK_DOMAIN=your-name.ngrok-free.dev npm run webhook

The allow/deny decisions are hardcoded in the SETTINGS block at the top of webhook-server.ts — edit PRE_DECISION / POST_DECISION (and restart) to exercise the matrix.

It prints:

 PUBLIC WEBHOOK URL  →  https://xxxx.ngrok-free.app/webhook

Create webhook-config.json from webhook-config.example.json, pasting that URL in (the script prints the exact JSON to use).

Run the shim from the guarded-api-cloudflare repo, pointed at real Featherless and reading the config file you just wrote:

INFERENCE_URL='https://api.featherless.ai/v1/chat/completions' \
CONFIG_STORE=local \
CONFIG_STORE_PATH='/absolute/path/to/guarded-api-sample/webhook-config.json' \
PORT=8001 \
npm run shim

The shim resolves the webhook config from that file on every request, so editing the URL (or enabled) takes effect immediately — no restart.

Paste your key into the SETTINGS block at the top of client.ts (and set the shim URL / model there), then:

npm run client          # interactive menu — pick 1 / 2 / 3 to send a request
npm run client:single   # one-shot: just the single message
npm run client:tools    # one-shot: just the tool cycle

The menu loops until you press q:

guarded-API client → https://guarded-api.featherless.ai/v1  (model: zai-org/GLM-5.1)
  1) single message  — "What is the capital of Japan?"
  2) tool cycle      — weather in Tokyo (uses get_weather)
  3) custom message  — type your own
  q) quit
>

• With allow/allow you get normal completions; the tool flow prints the model's tool call, the local tool result, and the final answer.
• Set PRE_DECISION = 'deny' in webhook-server.ts (then restart the webhook in terminal 1) and the client prints HTTP 400 … blocked — the model never ran. POST_DECISION = 'deny' blocks after the model runs instead.

Client — settings are hardcoded in the SETTINGS block of client.ts (edit and re-run), not env vars:

Webhook — settings are hardcoded in the SETTINGS block of webhook-server.ts (edit and restart), not env vars:

ngrok wrapper (start-webhook.sh) reads one env var:

• Why ngrok? So a shim — including a remotely deployed one — can reach your local webhook over a public HTTPS URL. For a purely local shim you could point the config straight at http://127.0.0.1:3000/webhook and skip ngrok.
• Reserved domain. A free ngrok account includes one static domain. Pass it as NGROK_DOMAIN and the public URL is identical every run, so you set webhook-config.json once instead of re-pasting a fresh URL each time. Without it, ngrok hands out a new random URL on each launch.
• Streaming isn't supported by the shim (the post hook needs the full response); the client uses non-streaming calls.