
firecrawl-crawl
#7 in Reading & ResearchBulk-extract many pages from one site or section.
Overview
Bulk-extract many pages from one site or section. Use for "crawl", "everything under /docs", or content spanning linked pages.From the project's GitHub description
Our editorial assessment of firecrawl-crawl is still in review. Facts below come from the repository and our catalogue.
Good for
Technical specifications
Each attribute is labelled with where it came from.
- Software type
- Agent / Skillevidence
- Agent ecosystem
- Skillevidence
- Interface
- No direct user interfaceevidence
- Runtime
- Agent runtimeevidence
- Interaction model
- Agent-operatedevidence
- Deployment model
- Local installationevidence
- License
- Not verified. GitHub could not identify this licence; check the repository before adopting.
Install
A skill is a set of instructions your agent loads. Install it into the agent you use; there is nothing to host.
$ npx skills add https://github.com/firecrawl/cli --skill firecrawl-crawl- Listing
- skills.shInstall counts come from skills.sh.
Alternatives
Also in Reading & Research
Same category, not necessarily equivalent.
GPT Researcher
An autonomous agent that researches a topic and writes a cited report.
firecrawl-search
Find web sources with query-relevant page excerpts and optional full-page content, and discover workflows, data APIs, and indexes.
web-search
Web search and content extraction with Tavily and Exa via inference.sh CLI.
Reviews
Community ratings are kept separate from Beav Scout rankings.
No reviews yet. Community reviews and comments are opening soon; you'll be able to rate firecrawl-crawl once signed in.