A clear path
from request to result.

1. Create a key.

   Create your account, complete email confirmation when required, then create a named key in API keys. Save its secret when it appears; it cannot be retrieved later. Choose an active paid subscription before creating your first key.

2. Set two environment variables.

   Set WAGGLE_BASE_URL to the website origin you are using, without a trailing slash, and WAGGLE_API_KEY to your secret. Run these examples from your server or terminal.

3. Inspect the result.

   A successful page extraction returns HTTP 200, request_id, data, credits_used: 1 and elapsed_ms. Keep the request ID; it connects your response to Activity.

curl --fail-with-body --include --max-time 45 \
  -X POST "$WAGGLE_BASE_URL/api/pageflock/v1/scrape" \
  -H "Authorization: Bearer $WAGGLE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com"
}'

Python examples use the standard library. JavaScript examples target Node.js 20+. Go, Ruby and Java examples use their standard HTTP libraries. cURL examples include response headers and fail with the error body intact. The client timeout is 45 seconds; a lost response does not prove the request was never completed.

Send Authorization: Bearer YOUR_API_KEY and Content-Type: application/json with JSON POST requests. The base path is /api/pageflock/v1. Keys share their account’s credit balance, rate limits and concurrency allowance.

Existing API keys and the /api/waggle/v1 alias continue to work. Responses expose X-PageFlock-* headers alongside their X-Waggle-* compatibility names.

Follow the key handover guide

The seven operations below are synchronous. They return their result in the same HTTP response. Saving or configuring a request in the workbench does not run it.

Readable text, links and metadata from a public page.

Static HTML costs 1 credit. Optional JavaScript rendering costs 5 where available. Login, private and blocked sources are unsupported.

Request body

Make the request

curl --fail-with-body --include --max-time 45 \
  -X POST "$WAGGLE_BASE_URL/api/pageflock/v1/scrape" \
  -H "Authorization: Bearer $WAGGLE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "url": "https://example.com"
}'

Illustrative JSON with shortened content. Fields in the table below are inside data; source content and timing will vary.

{
  "request_id": "00000000-0000-4000-8000-000000000001",
  "data": {
    "url": "https://example.com/",
    "title": "Example Domain",
    "text": "Example Domain\nThis domain is for use in illustrative examples in documents.",
    "links": [
      {
        "text": "Learn more",
        "url": "https://iana.org/domains/example"
      }
    ],
    "metadata": {
      "description": null,
      "language": "en",
      "quality": {
        "score": 58,
        "grade": "usable",
        "mode": "static"
      }
    },
    "evidence": {
      "source_url": "https://example.com/",
      "retrieved_at": "2026-09-17T00:00:00+00:00",
      "content_sha256": "…"
    },
    "status_code": 200
  },
  "credits_used": 1,
  "elapsed_ms": 840
}

Published emails, phone contacts and social links, with sources.

One page per request, up to 50 contacts per category. Published addresses are not mailbox-verified. Suggested pages are never fetched automatically.

Request body

Make the request

curl --fail-with-body --include --max-time 45 \
  -X POST "$WAGGLE_BASE_URL/api/pageflock/v1/contacts" \
  -H "Authorization: Bearer $WAGGLE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "url": "https://www.w3.org/Consortium/contact"
}'

Illustrative JSON with shortened content. Fields in the table below are inside data; source content and timing will vary.

{
  "request_id": "00000000-0000-4000-8000-000000000001",
  "data": {
    "url": "https://example.com/contact",
    "checked_at": "2026-09-15T10:00:00+00:00",
    "pages_checked": 1,
    "emails": [
      {
        "value": "hello@example.com",
        "found_in": "mailto",
        "verification": "not_checked",
        "source_url": "https://example.com/contact"
      }
    ],
    "phones": [],
    "socials": [],
    "contact_pages": [],
    "found": true,
    "truncated": false,
    "scope": "Published contacts from this page's static HTML and organization markup. Mailboxes, phone ownership and social profiles are not verified. Linked pages are not fetched; each follow-up costs a separate request."
  },
  "credits_used": 1,
  "elapsed_ms": 840
}

Discover public sources with precise queries and local results.

Up to 10 results. Quoted phrases and site: operators are preserved. Extracting a result is a separate request.

Request body

Make the request

curl --fail-with-body --include --max-time 45 \
  -X POST "$WAGGLE_BASE_URL/api/pageflock/v1/search" \
  -H "Authorization: Bearer $WAGGLE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "query": "site:docs.python.org \"asyncio\"",
  "country": "us",
  "language": "en",
  "limit": 10
}'

Illustrative JSON with shortened content. Fields in the table below are inside data; source content and timing will vary.

{
  "request_id": "00000000-0000-4000-8000-000000000001",
  "data": {
    "query": "site:docs.python.org \"asyncio\"",
    "country": "us",
    "language": "en",
    "provider": "serper",
    "type": "web",
    "results": [
      {
        "position": 1,
        "title": "asyncio — Asynchronous I/O",
        "url": "https://docs.python.org/3/library/asyncio.html",
        "snippet": "A library to write concurrent code using the async/await syntax.",
        "date": null
      }
    ]
  },
  "credits_used": 1,
  "elapsed_ms": 840
}

Product details from published data on pages that allow direct access.

Reads published Product JSON-LD, or a selected Amazon, Walmart or Shopee product page. Some marketplaces reject direct requests; blocked or unavailable pages return an error without a credit charge. Missing values stay empty.

Request body

Make the request

curl --fail-with-body --include --max-time 45 \
  -X POST "$WAGGLE_BASE_URL/api/pageflock/v1/products" \
  -H "Authorization: Bearer $WAGGLE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "url": "https://scrapeme.live/shop/Bulbasaur/"
}'

Illustrative JSON with shortened content. Fields in the table below are inside data; source content and timing will vary.

{
  "request_id": "00000000-0000-4000-8000-000000000001",
  "data": {
    "url": "https://example.com/products/notebook",
    "products": [
      {
        "name": "Field notebook",
        "description": null,
        "sku": "NOTE-01",
        "brand": "Example",
        "offers": [
          {
            "price": "12.00",
            "currency": "USD",
            "availability": "https://schema.org/InStock",
            "url": "https://example.com/products/notebook"
          }
        ]
      }
    ],
    "source": "published_json_ld",
    "scope": "Up to 20 published products and 20 offers each. Missing fields stay null; prices are not independently verified."
  },
  "credits_used": 1,
  "elapsed_ms": 840
}

Public video details and exact engagement when published.

Public, embeddable videos only. Extended metadata, captions and related videos are returned when available and permitted. Missing engagement counts are labelled, not estimated.

Request body

Make the request

curl --fail-with-body --include --max-time 45 \
  -X POST "$WAGGLE_BASE_URL/api/pageflock/v1/youtube" \
  -H "Authorization: Bearer $WAGGLE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "url": "https://www.youtube.com/watch?v=jNQXAC9IVRw"
}'

Illustrative JSON with shortened content. Fields in the table below are inside data; source content and timing will vary.

{
  "request_id": "00000000-0000-4000-8000-000000000001",
  "data": {
    "url": "https://www.youtube.com/watch?v=jNQXAC9IVRw",
    "video_id": "jNQXAC9IVRw",
    "title": "Me at the zoo",
    "channel": {
      "name": "jawed",
      "url": "https://www.youtube.com/@jawed"
    },
    "thumbnail_url": "https://i.ytimg.com/vi/jNQXAC9IVRw/hqdefault.jpg",
    "source": "youtube_oembed",
    "scope": "Public embed metadata: title, channel and thumbnail. Extended public details were unavailable for this video."
  },
  "credits_used": 1,
  "elapsed_ms": 840
}

Find headlines, sources and snippets in your chosen market.

Up to 10 results. Article extraction is a separate request. Dates are supplied by the search provider.

Request body

Make the request

curl --fail-with-body --include --max-time 45 \
  -X POST "$WAGGLE_BASE_URL/api/pageflock/v1/search/news" \
  -H "Authorization: Bearer $WAGGLE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "query": "\"renewable energy\"",
  "country": "us",
  "language": "en",
  "limit": 10
}'

Illustrative JSON with shortened content. Fields in the table below are inside data; source content and timing will vary.

{
  "request_id": "00000000-0000-4000-8000-000000000001",
  "data": {
    "query": "\"renewable energy\"",
    "country": "us",
    "language": "en",
    "provider": "serper",
    "type": "news",
    "results": [
      {
        "position": 1,
        "title": "Example energy headline",
        "url": "https://example.com/news/energy",
        "snippet": "An illustrative news-result snippet.",
        "date": null
      }
    ]
  },
  "credits_used": 1,
  "elapsed_ms": 840
}

Discover videos, then collect metadata for supported YouTube links.

Up to 10 discovery results with bounded Serper fields. YouTube views, likes and comments are enriched from the public YouTube Data API when configured; dislikes remain private. Transcripts require a separate metadata request. No media downloads.

Request body

Make the request

curl --fail-with-body --include --max-time 45 \
  -X POST "$WAGGLE_BASE_URL/api/pageflock/v1/search/videos" \
  -H "Authorization: Bearer $WAGGLE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "query": "site:youtube.com \"design systems\"",
  "country": "us",
  "language": "en",
  "limit": 10
}'

Illustrative JSON with shortened content. Fields in the table below are inside data; source content and timing will vary.

{
  "request_id": "00000000-0000-4000-8000-000000000001",
  "data": {
    "query": "site:youtube.com \"design systems\"",
    "country": "us",
    "language": "en",
    "provider": "serper",
    "type": "videos",
    "results": [
      {
        "position": 1,
        "title": "Example design systems video",
        "url": "https://example.com/videos/design",
        "snippet": "An illustrative video-result snippet.",
        "date": null
      }
    ]
  },
  "credits_used": 1,
  "elapsed_ms": 840
}

Use the same Bearer key. This read-only request returns your account’s current balance and effective limits. Read these values instead of hard-coding the allowances of a pricing tier.

curl --fail-with-body --include --max-time 45 \
  -X GET "$WAGGLE_BASE_URL/api/pageflock/v1/usage" \
  -H "Authorization: Bearer $WAGGLE_API_KEY"

A batch processes a list of URLs. A crawl follows eligible links from one exact origin. Collections use static page extraction and continue after you close the browser. Search queries and dedicated contact/product/video output are separate synchronous operations.

curl --fail-with-body --include --max-time 45 \
  -X POST "$WAGGLE_BASE_URL/api/pageflock/v1/collections" \
  -H "Authorization: Bearer $WAGGLE_API_KEY" \
  -H 'Content-Type: application/json' \
  --data '{
  "name": "My research",
  "mode": "crawl",
  "urls": [
    "https://example.com"
  ],
  "max_pages": 5,
  "client_id": "my-unique-run-001"
}'

The response contains an id, status, counts, credits_used and expires_at. Keep its ID and poll every four seconds or slower. The collection body uses client_id for duplicate submissions; the synchronous Idempotency-Key header does not apply here.

Each successful page costs one credit. Failed or interrupted page reservations are refunded; pending cancelled pages are not charged. There are at most 50 saved collections per account. Results expire seven days after collection creation, so download them before expires_at.

All seven synchronous operations use the success envelope below. Errors use {"error":{"message":"…"}}. Do not expect a success-shaped data object when the HTTP status is an error.

Find the original operation by request ID in Activity. History stores request metadata and inputs; it does not ordinarily retain the whole synchronous response. Saved requests store configurations and never run automatically.

For the seven synchronous POST operations, send an optional Idempotency-Key before the first request. Generate a UUID for each new logical operation. Keep that key with its exact request inputs and reuse it if the connection drops.

Opting in allows pageflock to retain the request and its response for up to 24 hours: up to 2 MiB per response, 16 MiB of retained response data per account, and 1,000 unexpired receipts per account. The raw idempotency key is stored as a hash. Account closure removes these receipts and results.

Without this header, each successful submission is a new billable operation. A client timeout is ambiguous: inspect Activity before submitting again. Replays still count toward your account request-rate limit, but do not consume provider or destination acquisition budgets. Even with retry protection, use a bounded retry count and honor Retry-After; changing the key on every retry defeats duplicate protection.

Use GET usage for your effective limits. Rate is how many operations can begin in a time window; concurrency is how many can be running together. They are separate controls shared across account keys, the dashboard and collection workers.

Estimate a workload by mixing one-credit requests and five-credit searches. Count calls, not result rows. Paid plans renew monthly and add credits after each successful payment. Unused credits roll over while your account remains open. Cancel before the next renewal from Plans & billing. The estimator selects one monthly plan; larger workloads need an Enterprise quote. Existing purchased credits are retained without automatically subscribing you.

First inspect the HTTP status, error.message, and any request ID or retry header. An error message can describe the failed field or source policy; do not retry every 422 or rotate keys to work around a limit.

{
  "error": {
    "message": "Check these fields: limit"
  }
}

pageflock checks source decisions, network destinations, robots.txt for page extraction, and each redirect destination. It refuses local/private networks, credentials in URLs, unsupported ports and blocked sources. It does not bypass logins, paywalls or CAPTCHAs, or fetch LinkedIn profile pages. A search-result URL or a social link is not authorization to scrape that page.

Contacts carry source_url and found_in; keep their evidence and extraction timestamp with downstream records. Published contact information is not verified mailbox, phone or social ownership.

Settings provides account export and closure. Exporting does not extend a collection’s expiry or include every synchronous response. Keep your own copy of important results and choose an appropriate retention policy for contact information in your application.