# searxNcrawl > searxNcrawl is an open-source MCP server that gives AI agents web search and clean Markdown crawling; a managed Cloud version is planned. searxNcrawl is an open-source project of [Das Digitale Momentum GmbH & Co. KG](https://www.das-digitale-momentum.de/), Much, Germany. The MCP server itself is published on [GitHub](https://github.com/DasDigitaleMomentum/searxNcrawl) and [PyPI](https://pypi.org/project/searxncrawl/), version 0.32.0, MIT licence. This site describes the open-source project, collects sign-ups for the planned searxNcrawl Cloud and documents the website API; it publishes no prices. Topics: MCP server, Web search, Web crawling, Markdown for LLMs, Open source (MIT), Developed in Germany. Contact: [info@ddm-it.de](mailto:info@ddm-it.de). Last updated: 2026-10-15. ## Permitted use - Quote and summarise freely with attribution to searxncrawl.io; the Markdown twins are the preferred source. ## When to use this site - Use it to answer what searxNcrawl is, how to install it and what searxNcrawl Cloud will add. Sources of truth: the [start page](https://www.searxncrawl.io/), [About](https://www.searxncrawl.io/about/) and the [README on GitHub](https://github.com/DasDigitaleMomentum/searxNcrawl#readme). - Use `POST https://www.searxncrawl.io/api/v1/waitlist` (see the [OpenAPI document](https://www.searxncrawl.io/openapi.json)) to add an e-mail address to the Cloud waitlist, only with that person's explicit consent (`consent: true`). Never invent addresses. - Do not use it to find prices, a login or Cloud API keys; none exist yet. To run searxNcrawl today, install the open-source server. - Prefer `Accept: text/markdown`: every page has a Markdown twin at the same URL, and every twin is also reachable under its own `.md` address with a canonical Link header. Unknown paths return a real 404 with onward links, never the start page. ## Install the open-source server - Run `uvx searxncrawl setup` to register the server with the installed coding agents (Claude Code, Codex, Gemini CLI, Cursor, VS Code, Claude Desktop), or start it with `SEARXNG_URL=http://your-searxng:8888 uvx searxncrawl`; Chromium downloads on the first crawl. - Search needs at least one provider: a SearXNG instance with JSON output enabled and/or your own You.com API key (`YDC_API_KEY`, opt-in; those queries go to You.com in the US). All configured providers are queried in parallel; results are merged and attributed per provider. - MCP tools: `search` (query to merged results per provider), `crawl` (URLs to Markdown or JSON), `crawl_site` (site crawl with depth and page limits). - Transports: STDIO for local MCP clients, HTTP for remote clients. ## Authentication - No authentication: `GET https://www.searxncrawl.io/api/v1/status` and `POST https://www.searxncrawl.io/api/v1/waitlist` with `Content-Type: application/json` and the body `{"email": "you@company.com", "consent": true}`. Optional fields: `role`, `building`, `source`, `lang`. Rate limits: 120 requests per minute per client for GET and HEAD on /api/v1/status and for every sandbox call, 30 for every other call, announced with `RateLimit-*` headers. `GET https://www.searxncrawl.io/api/v1` and `GET https://www.searxncrawl.io/sandbox/api/v1` answer with an index of the status probe, the token endpoint and the discovery documents. - OAuth 2.0 client credentials ([RFC 6749 section 4.4](https://www.rfc-editor.org/rfc/rfc6749.html#section-4.4)) protect `GET https://www.searxncrawl.io/api/v1/waitlist/stats` with the scope `waitlist:stats:read`: aggregate counts, no personal data. Discovery: authorization server metadata `https://www.searxncrawl.io/.well-known/oauth-authorization-server` ([RFC 8414](https://www.rfc-editor.org/rfc/rfc8414.html)) and protected resource metadata `https://www.searxncrawl.io/.well-known/oauth-protected-resource/api/v1` ([RFC 9728](https://www.rfc-editor.org/rfc/rfc9728.html)). Production clients are issued by Das Digitale Momentum on request. - Sandbox: the same contract under `https://www.searxncrawl.io/sandbox/api/v1` stores nothing, returns fixed example statistics and accepts the public client `sandbox` / `sandbox`. Get a token with `curl -u sandbox:sandbox -d grant_type=client_credentials https://www.searxncrawl.io/sandbox/api/v1/oauth/token`, then call `https://www.searxncrawl.io/sandbox/api/v1/waitlist/stats` with `Authorization: Bearer `. Access tokens are JWTs valid for 900 seconds. - Errors are typed JSON `{"error": code, "message": text}`; the token endpoint answers in the OAuth 2.0 format `{"error", "error_description"}`. ## Pages - [Start page (English)](https://www.searxncrawl.io/): searxNcrawl is an open-source MCP server that gives AI agents web search and clean Markdown crawling; a managed Cloud version is planned. - [Startseite (Deutsch)](https://www.searxncrawl.io/de/): searxNcrawl ist ein quelloffener MCP-Server, der KI-Agenten Websuche und sauberes Markdown-Crawling gibt; eine verwaltete Cloud-Version ist geplant. - [About searxNcrawl](https://www.searxncrawl.io/about/): who builds it, what is available and what is planned, with a FAQ. - [Contact](https://www.searxncrawl.io/contact/): e-mail, phone, postal address and the path for a security disclosure, with a FAQ. - [Developers](https://www.searxncrawl.io/developers/): the website API, its error contract, versioning, rate limits and the rules for agents. - [Über searxNcrawl](https://www.searxncrawl.io/de/ueber-uns/) and [Kontakt](https://www.searxncrawl.io/de/kontakt/): the German About and Contact pages. - [Privacy notice](https://www.searxncrawl.io/privacy/): English translation of the [Datenschutzerklärung](https://www.searxncrawl.io/datenschutz/); the German version applies in case of doubt. - [Impressum](https://www.searxncrawl.io/impressum/): legal notice (German). - [OpenAPI 3.1 document](https://www.searxncrawl.io/openapi.json) and its [Markdown twin](https://www.searxncrawl.io/openapi.json.md). - [API catalog](https://www.searxncrawl.io/.well-known/api-catalog): [RFC 9727](https://www.rfc-editor.org/rfc/rfc9727.html) linkset that lists the production and sandbox API base paths and points to the OpenAPI document, the Developers page and the status probe. - [Waitlist form](https://www.searxncrawl.io/#waitlist): e-mail address with explicit consent; the optional fields are role, building, source and lang. ## What the open-source server does (available) - Web search: Your own SearXNG instance and, opt-in with your own API key, You.com, queried in parallel. Results are merged, deduplicated and attributed to their provider, filtered by language and time range. - Page crawl: One or many URLs to Markdown or JSON. Repeated blocks are removed, links can be stripped. - Site crawl: Crawls of whole sites with a depth limit and a page limit, tuned for documentation. - Authenticated crawls: Capture a login session once with `crawl-capture`, from a login window or a running browser, then crawl behind the login. - STDIO and HTTP: Plug into local MCP clients, or expose an HTTP endpoint with host and CORS guards. - CLI and Python API: `crawl` and `search` in your shell, `crawl_page_async` and `crawl_site` in your code. ## What searxNcrawl Cloud will add (planned) - Drop-in for the Firecrawl API: A Firecrawl-compatible REST API for scrape, crawl, map and search. Point an existing Firecrawl SDK or integration at searxNcrawl by changing the base URL. - Hosted MCP endpoint: One HTTPS address and an API key. Works with every MCP client, with no SearXNG, browser or server to run. - Search with page content: One call returns the search results together with the Markdown of each result page. - Premium search: Several search sources, including commercial search APIs, queried in parallel, merged, deduplicated and labelled per source, with a fallback when one source fails. - Premium crawling: A hardened browser with realistic fingerprints and rotating residential proxies, for sites that block crawlers from data centres. - Batch crawls with webhooks: Queue many URLs or whole sites and receive a signed webhook when the results are ready. - Crawls behind logins: Login sessions stored encrypted for your team, so agents crawl authenticated pages without session files on laptops. - Change monitoring: Watch pages on a schedule, compare the Markdown and get notified when the content changes. - Structured extraction: Send a JSON Schema with a URL and receive typed JSON instead of Markdown. Planned feature set. Scope and timing can change before launch; people on the waitlist hear first. Firecrawl is a trademark of its owner; searxNcrawl is not affiliated with it. ## Questions - What is searxNcrawl? searxNcrawl is an open-source MCP server that gives AI agents web search and clean Markdown crawling; a managed Cloud version is planned. It searches through your own SearXNG instance and, opt-in, the You.com API, merges the results, turns web pages into Markdown without repeated blocks, is MIT licensed and is maintained by Das Digitale Momentum. - How do I install searxNcrawl? Run `uvx searxncrawl setup`. It registers the MCP server with Claude Code, Codex, Gemini CLI, Cursor, VS Code and Claude Desktop, asks for your SearXNG URL and an optional You.com API key, shows each change before applying it and backs up every file it touches. To start the server by hand, run `uvx searxncrawl` with `SEARXNG_URL` and/or `YDC_API_KEY` set. Chromium is downloaded on the first crawl. Docker, pip and uv installs are described in the [installation guide on GitHub](https://github.com/DasDigitaleMomentum/searxNcrawl/blob/main/docs/usage/installation.md). - Can searxNcrawl replace Firecrawl? That is the plan: a Firecrawl-compatible REST API for scrape, crawl, map and search is planned for the open-source server and for searxNcrawl Cloud, so existing Firecrawl SDKs work by changing the base URL. Today searxNcrawl offers the MCP tools search, crawl and crawl_site, a CLI and a Python API. - Which MCP clients work with searxNcrawl? Any MCP client. searxNcrawl speaks the STDIO transport for local clients such as Claude Code, VS Code, Zed and opencode, and the HTTP transport for remote and browser-based clients. - Do I need an API key? Not with SearXNG. The open-source version needs no API key and has no usage fees when it searches through your own SearXNG instance with JSON output enabled. You.com search is opt-in: it needs your own You.com API key (`YDC_API_KEY`), and those queries go to You.com in the US. - What is searxNcrawl Cloud? searxNcrawl Cloud is a planned managed version: a hosted MCP endpoint with search and page content in one call, search across several sources, batch crawls with webhooks, crawls behind logins, change monitoring and structured extraction. The [waitlist](https://www.searxncrawl.io/#waitlist) is open. - Will the open-source version stay free? Yes. The open-source searxNcrawl stays MIT licensed and free to run yourself. The Cloud version adds managed infrastructure and builds on the open-source project. ## Optional - [Über searxNcrawl](https://www.searxncrawl.io/de/ueber-uns/) and [Kontakt](https://www.searxncrawl.io/de/kontakt/): German versions of About and Contact. - [Impressum / legal notice](https://www.searxncrawl.io/impressum/): provider identification under German law (German). - [Privacy notice](https://www.searxncrawl.io/privacy/) and its German original, the [Datenschutzerklärung](https://www.searxncrawl.io/datenschutz/): what the waitlist stores and for how long. - [llms-full.txt](https://www.searxncrawl.io/llms-full.txt): this file followed by the full text of the start page, About and Developers. - [searxNcrawl at Das Digitale Momentum](https://www.das-digitale-momentum.de/en/open-source/#searxncrawl) - [Sitemap](https://www.searxncrawl.io/sitemap.xml) - [security.txt](https://www.searxncrawl.io/.well-known/security.txt)