Webhooks: receive, verify, and process job events

Connect an endpoint to your application

Add a public HTTPS endpoint in Dashboard → Webhooks and keep its signing secret in your server’s environment. Subscribe this example to job.completed and job.failed. A scan or classification job can produce either event.

The complete flow is: submit a job, save its ID, receive and verify the notification, commit the event to an inbox, acknowledge it, then fetch the result in a worker. This keeps slow report processing outside the delivery request.

For a batch integration, connect each saved job ID to the corresponding item in your batch-processing workflow.

A notification tells you which job to retrieve

The body is the event payload itself, with no outer data wrapper. These are the routing fields from a scan completion:

{
  "job_id": "job_example",
  "request_id": "req_example",
  "type": "scan",
  "status": "completed",
  "callback_metadata": {
    "item_id": "api-comparison"
  }
}

Completions also include completed_at and timings with queue_wait_ms, execution_ms, and total_wall_ms. Failures include failed_at, error.code, error.message, a one-based attempt, and will_retry. Terminal failure notifications have will_retry: false.

The full report is retrieved separately. Omit format to receive the default for the job’s tier; Lite and Standard/Deep use different report schemas.

Verify the exact bytes that arrived

OnPage-Signature: <64-character hexadecimal HMAC>
OnPage-Timestamp: <Unix seconds>
OnPage-Delivery-Id: <delivery ID>
OnPage-Event: job.completed
User-Agent: OnPage-Webhooks/1.0

1. Read the raw request body before JSON middleware changes it.
2. Validate the timestamp and signature shape. The examples allow five minutes of clock skew; keep your server clock synchronized.
3. Compute HMAC-SHA256 over timestamp + "." + raw_body, using the signing-secret string as UTF-8 text.
4. Compare equal-length signature bytes with a constant-time comparison.
5. Only then parse JSON and persist the event.

The signed message includes the timestamp and body. The delivery ID and event-name headers are not part of that signature. These examples derive routing and duplicate protection from the signed job_id and status; delivery IDs are useful for tracing.

The timestamp is refreshed for each delivery attempt, so a valid retry several hours later still passes the freshness check. Repeated requests within the five-minute window are handled by the inbox’s unique event key.

Run the Node.js or Python integration

Choose Node.js 24+ or Python 3.10+. Each download has three commands: serve verifies and stores events, test sends a locally signed message, and work drains the saved inbox once. SQLite commits an event before the receiver returns 202.

import { createServer } from "node:http";
import { createHmac, timingSafeEqual, createHash, randomUUID } from "node:crypto";
import { DatabaseSync } from "node:sqlite";
import { mkdirSync, writeFileSync, renameSync } from "node:fs";
import { join } from "node:path";

// Node.js 24+. Keep the database on persistent local storage.
const mode = process.argv[2] ?? "serve";
const secret = process.env.ONPAGE_WEBHOOK_SECRET;
const dbPath = process.env.ONPAGE_WEBHOOK_DB ?? "webhook-inbox.sqlite";
const port = Number(process.env.PORT ?? 8787);
const maxBytes = 1024 * 1024;
process.umask(0o077);
function sign(timestamp, bytes) {
  return createHmac("sha256", secret).update(timestamp + ".").update(bytes).digest("hex");
}
export function verify(bytes, timestamp, signature, now = Date.now() / 1000) {
  if (!secret || typeof timestamp !== "string" || !/^\d{1,12}$/.test(timestamp) ||
      typeof signature !== "string" || !/^[a-fA-F0-9]{64}$/.test(signature) ||
      Math.abs(now - Number(timestamp)) > 300) return false;
  return timingSafeEqual(Buffer.from(signature, "hex"), Buffer.from(sign(timestamp, bytes), "hex"));
}
function inbox() {
  const db = new DatabaseSync(dbPath);
  db.exec("PRAGMA journal_mode=WAL; PRAGMA synchronous=FULL; PRAGMA busy_timeout=5000;");
  db.exec("CREATE TABLE IF NOT EXISTS inbox (event_key TEXT PRIMARY KEY, job_id TEXT, kind TEXT NOT NULL, payload TEXT NOT NULL, state TEXT NOT NULL DEFAULT 'pending')");
  return db;
}
function identity(body, bytes) {
  if (body.test === true) return { key: "test:" + createHash("sha256").update(bytes).digest("hex"), jobId: null, kind: "test" };
  if (!/^job_[A-Za-z0-9_-]+$/.test(body.job_id ?? "") ||
      !["scan", "classify"].includes(body.type) || !["completed", "failed"].includes(body.status)) throw new Error("Unsupported job event");
  if (body.status === "failed" && body.will_retry !== false) throw new Error("Expected a terminal failure");
  // Identity and routing come from the signed body, not unsigned event headers.
  return { key: body.job_id + ":" + body.status, jobId: body.job_id, kind: body.status };
}
if (mode === "test") {
  if (!secret) throw new Error("Set ONPAGE_WEBHOOK_SECRET.");
  const bytes = Buffer.from(JSON.stringify({ test: true, nonce: randomUUID() }));
  const timestamp = String(Math.floor(Date.now() / 1000));
  const response = await fetch("http://127.0.0.1:" + port + "/webhooks/onpage", {
    method: "POST", headers: { "Content-Type": "application/json", "OnPage-Timestamp": timestamp, "OnPage-Signature": sign(timestamp, bytes) }, body: bytes,
    signal: AbortSignal.timeout(10_000),
  });
  console.log(response.status, await response.text());
  if (!response.ok) process.exitCode = 1;
} else if (mode === "serve") {
  if (!secret) throw new Error("Set ONPAGE_WEBHOOK_SECRET.");
  const db = inbox();
  const insert = db.prepare("INSERT OR IGNORE INTO inbox(event_key,job_id,kind,payload) VALUES (?,?,?,?)");
  const server = createServer(async (req, res) => {
    const reply = (code, message) => { res.writeHead(code, { "Content-Type": "text/plain" }); res.end(message); };
    if (req.method !== "POST" || req.url !== "/webhooks/onpage") { reply(404, "Not found"); return; }
    try {
      const chunks = []; let length = 0;
      for await (const chunk of req) {
        length += chunk.length;
        if (length > maxBytes) { reply(413, "Body too large"); return; }
        chunks.push(chunk);
      }
      const bytes = Buffer.concat(chunks);
      if (!verify(bytes, req.headers["onpage-timestamp"], req.headers["onpage-signature"])) { reply(401, "Invalid signature or timestamp"); return; }
      let event;
      try { event = identity(JSON.parse(bytes.toString("utf8")), bytes); }
      catch { reply(400, "Unsupported payload"); return; }
      // SQLite commits the insert before we acknowledge the delivery.
      const result = insert.run(event.key, event.jobId, event.kind, bytes.toString("utf8"));
      reply(result.changes ? 202 : 200, result.changes ? "Queued" : "Already recorded");
    } catch { if (!res.headersSent) reply(503, "Inbox unavailable; retry delivery"); }
  });
  server.requestTimeout = 15_000;
  server.headersTimeout = 10_000;
  server.listen(port, "127.0.0.1", () => console.log("Webhook receiver listening on 127.0.0.1:" + port));
} else if (mode === "work") {
  const db = inbox();
  const output = process.env.ONPAGE_REPORT_DIR ?? "webhook-reports";
  mkdirSync(output, { recursive: true, mode: 0o700 });
  for (const row of db.prepare("SELECT * FROM inbox WHERE state='pending'").all()) {
    try {
      if (row.kind === "completed") {
        const key = process.env.ONPAGE_API_KEY;
        if (!key) throw new Error("Set ONPAGE_API_KEY to retrieve completed results.");
        const response = await fetch("https://api.on-page.ai/v1/jobs/" + encodeURIComponent(row.job_id) + "/result", {
          headers: { Authorization: "Bearer " + key }, signal: AbortSignal.timeout(30_000),
        });
        if (!response.ok) throw new Error("Result HTTP " + response.status + "; leave pending and retry later.");
        const report = await response.json();
        const destination = join(output, row.job_id + ".json"), temp = destination + "." + randomUUID() + ".tmp";
        writeFileSync(temp, JSON.stringify(report, null, 2), { mode: 0o600 });
        renameSync(temp, destination);
      }
      db.prepare("UPDATE inbox SET state='done' WHERE event_key=?").run(row.event_key);
      console.log(row.event_key, "done");
    } catch (error) { console.error(row.event_key, error.message); process.exitCode = 1; }
  }
  db.close();
} else { throw new Error("Usage: node onpage-webhook.mjs serve|test|work"); }

For a local test, set the same throwaway secret in two terminals. In the first terminal, start the receiver:

export ONPAGE_WEBHOOK_SECRET="local-demo-secret"
node onpage-webhook.mjs serve

In the second terminal, send a signed test and process the saved event. Both terminals must use the same working directory, or the same absolute ONPAGE_WEBHOOK_DB path.

export ONPAGE_WEBHOOK_SECRET="local-demo-secret"
node onpage-webhook.mjs test
node onpage-webhook.mjs work

The test returns 202 Queued; the worker prints a test event marked done. It makes no scan or result request. For real completions, supply ONPAGE_API_KEY to the worker and use your endpoint’s actual signing secret in the receiver.

Successful job results are written to webhook-reports/job_ID.json. Failed job notifications stay in the database as a record and are marked processed; the sample does not automatically purchase another scan. A result-fetch error leaves the event pending and makes the worker exit with code 1. Rerun it after resolving the cause, respecting rate limits and any Retry-After interval.

Run one inbox worker at a time. The report filename is stable, so a process restart between saving the report and marking the event done can safely overwrite that same file. For emails, CMS changes, or other external actions, use a separate unique action key and the destination service’s idempotency support.

For deployment, keep the inbox on persistent storage and put the receiver behind your application’s HTTPS proxy. The Python starter expects a buffered body with Content-Length. Use your production server framework for traffic handling, retaining the same raw-body verification and transaction order. The sample serves one endpoint and account; a multi-account app must map each job to the owning account before choosing credentials.

Handle delivery failures and duplicate events

There are up to eight delivery attempts: the initial attempt, then delays of 30 seconds, 2 minutes, 10 minutes, 1 hour, 6 hours, 24 hours, and 48 hours after successive failures. Those delays add up to about 79 hours, with processing and queue time additional. The delivery ID stays the same across retries.

Endpoint health becomes degraded after five consecutive failures and paused after twenty. New events are enqueued only for active endpoints. Check endpoint health and delivery history in the dashboard, resolve the failure, and re-enable the endpoint. Do not rely on a later successful delivery to restore its active state automatically.

Keep a reconciliation process for submitted jobs with no recorded outcome. Query their status and retrieve completed results using the same unique job record. This also covers events missed while an endpoint was inactive.

Test local handling, then public delivery

The local test above proves signature verification and inbox processing. To exercise public HTTPS delivery, the API also provides a test endpoint:

curl -X POST https://api.on-page.ai/v1/webhooks/test \
  -H "Authorization: Bearer $ONPAGE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://your-app.example/webhooks/onpage","event":"job.completed"}'

This call returns a fresh signing_secret and delivery_id for that test. Its body is a test marker with test: true and createdAt, not a real completed-job payload. Use a separate test receiver or capture the raw test request privately and verify it against the returned secret. Do not replace your production endpoint’s configured secret with this temporary one.

Before connecting real jobs, test a valid event twice, an altered body, a malformed signature, an old timestamp, and an unavailable inbox. Expect one saved work item, 401 for failed verification, and a retryable non-2xx when persistence fails.

Questions that come up in real integrations

You now have a path from a signed job notification to a recoverable report file. Use the report explorer to choose which findings your next processing step should consume.