INTEGRATIONS / HTTP RECIPE

pageflock, meet Python.

Use Python requests with the pageflock API. Read a public page, inspect the response and save the fields your project needs.

← All integrations

Your first connection.

01
Install requests in your project's virtual environment: python -m pip install requests.
02
Set WAGGLE_BASE_URL to your pageflock origin, without a trailing slash. Set WAGGLE_API_KEY in your server environment.
03
Run the Python example below. The response is JSON; title, text, links and metadata belong to the selected public page.

Build the request.

Set the environment variables in your terminal for these code examples. In automation tools, enter the URL and credentials in the tool’s own fields.

Public page URL

import json
import os
import urllib.request
import urllib.error

base_url = os.environ["WAGGLE_BASE_URL"].rstrip("/")
headers = {"Authorization": "Bearer " + os.environ["WAGGLE_API_KEY"]}
headers["Content-Type"] = "application/json"
payload = json.loads("{\n  \"url\": \"https://example.com\"\n}")
request = urllib.request.Request(
    base_url + "/api/pageflock/v1/scrape",
    data=json.dumps(payload).encode("utf-8"),
    headers=headers,
    method="POST",
)
try:
    with urllib.request.urlopen(request, timeout=45) as response:
        print("Credits this call:", response.headers.get("X-PageFlock-Credits-Used") or response.headers.get("X-Waggle-Credits-Used"))
        print(json.load(response))
except urllib.error.HTTPError as error:
    print("HTTP", error.code)
    print("Request ID:", error.headers.get("X-PageFlock-Request-Id") or error.headers.get("X-Waggle-Request-Id"))
    print(error.read().decode())
    raise

This edits a code example. It does not send a request or use credits.

Run a request in your workspace