THE PAGEFLOCK BLOG / FIELD NOTES

A little reading. A lot to build.

Useful ideas for working with the open web. Start with the basics, then make something of your own.

The basics3 min read

Search, extract or crawl? Start with the right tool.

Three different jobs. One useful workflow. A guide to choosing your starting point.

Read the field note ↗
Workflows3 min read

Build a dataset you can trace back to the source.

A practical collection workflow, from a small URL list to a useful export.

Read the field note ↗
Engineering3 min read

Plan your requests. Keep control of your credits.

How costs, concurrency and retries fit together in a pageflock workflow.

Read the field note ↗
YOUR NEXT STEP

Put an idea into practice.

Choose a paid plan and start gathering useful web data.

Start building
YOUR PRIVACY

Choose what works for you. You can reopen these settings from the footer at any time.