npm.io
0.18.3 • Published 5d ago

crw-sdk

Licence
MIT
Version
0.18.3
Deps
0
Size
53 kB
Vulns
0
Weekly
716

crw-sdk

TypeScript/JavaScript SDK for CRW — the fast, Firecrawl-compatible web scraper. Scrape, crawl, map, search, parse, and extract any website from Node.

Zero runtime dependencies (Node 18+ fetch). Dual ESM + CommonJS.

Install

npm install crw-sdk

Quick start — Cloud (default)

CRW is cloud-first. Sign up for 500 free credits — no payment, no monthly reset (GitHub/Google, ~10s) — then set CRW_API_KEY:

import { CrwClient } from "crw-sdk";

const crw = new CrwClient(); // reads CRW_API_KEY from the env
const res = await crw.scrape("https://example.com", { formats: ["markdown"] });
console.log(res.markdown);
// ...or pass the key explicitly
const crw = new CrwClient({ apiKey: "fc-..." });

Self-hosting

// A self-hosted server:
const crw = new CrwClient({ apiUrl: "http://localhost:3000" });
# Local zero-config engine (no server, no key): set CRW_LOCAL=1.
# Requires the `crw-mcp` binary on PATH (or set CRW_BINARY).
CRW_LOCAL=1 node app.js

Methods

Method Description Mode
scrape(url, opts?) Scrape one URL both
crawl(url, opts?) Crawl a site (async, polled) both
map(url, opts?) Discover URLs both
search(query, opts?) Web search (+ optional scrape) both¹
parseFile(bytes, opts?) PDF → markdown / structured JSON both
extract({urls, schema?}) Structured LLM extraction HTTP
batchScrape(urls, opts?) Scrape many URLs (async) HTTP
capabilities() Feature-detect the engine HTTP
changeTrackingDiff(cur, prev?) Diff vs a prior snapshot HTTP
close() Shut down the local subprocess

¹ Local search needs a SearXNG URL configured on the engine.

// Structured extraction:
const data = await crw.extract({
  urls: ["https://example.com"],
  schema: { type: "object", properties: { title: { type: "string" } } },
});

// Parse a PDF:
import { readFileSync } from "node:fs";
const doc = await crw.parseFile(readFileSync("invoice.pdf"), { formats: ["markdown"] });

Parity

This SDK mirrors the Python crw client method-for-method, and both are conformance-tested against the engine's OpenAPI spec.

Keywords