searxNcrawl, a SearXNG MCP server with crawling

By Das Digitale Momentum, published November 3, 2026

searxNcrawl is an open-source MCP server that turns your SearXNG instance into a search tool for AI agents and adds crawl and crawl_site for clean Markdown. This page lists the tools and shows the setup for each MCP client.

What the SearXNG MCP server does

• search: queries your SearXNG instance and, opt-in, You.com in parallel and returns one merged, deduplicated list with the source of each result. Filters for language, time range, safe search and paging; SearXNG categories and engines. With include_content=true, each result also carries the Markdown of its page.
• crawl: one or many URLs to Markdown or JSON, with repeated blocks removed and links optionally stripped.
• crawl_site: follows links from a start URL within depth, page and time limits, tuned for documentation.

Why search and crawling belong in one server

SearXNG is a metasearch engine: it forwards a query to many search engines, merges their answers and tracks nobody. For an agent that is a good start, but a result list holds only titles and snippets of a few dozen words. To answer a question, the agent usually needs the page behind the link, and a page in HTML carries navigation, cookie banners and scripts that cost tokens without adding facts.

searxNcrawl closes that gap. The server that searches also renders pages in Chromium through Crawl4AI and Playwright and returns clean Markdown with repeated blocks removed. A typical agent flow is one search with include_content=true, or a search followed by crawl on the most relevant results; for documentation, crawl_site reads a whole section within the limits you set.

Crawling also works without any search provider. An agent that already knows the address, for example of a documentation page or an issue, calls crawl directly.

What a search returns

Through MCP, search returns structured results with the fields title, url, content, publishedDate and providers, the list of providers that found the result. With include_content=true, each result also carries markdown, the crawled page, or crawl_error when the page could not be read; the results arrive either way. The CLI prints the same results as Markdown, which is the quickest way to check a SearXNG instance:

SEARXNG_URL=http://localhost:8888 uvx --from searxncrawl search "model context protocol" --max-results 5

# Search: model context protocol
_Found 3 results_

## 1. <title of the first result>
<address of the first result>
**Sources:** searxng

<snippet of the first result>

The default fields keep each result small, which keeps the context of the agent short. SEARCH_RESULT_FIELDS selects other fields when a workflow needs them, for example only url and title for a quick overview.

Search providers: SearXNG and You.com

SearXNG is the default provider and needs no key. With your own You.com API key in YDC_API_KEY, searxNcrawl queries both providers in parallel and merges them into one list: duplicate addresses are joined conservatively, the results alternate between the providers, and results found by both come first. Each result names its providers, and if one provider fails the other still answers. You.com queries are processed in the US and return results for the first 10 pages only; the providers parameter picks a provider for a single call (changelog).

Results that only You.com found carry youcom in providers, so an agent can tell where a fact came from and weigh it accordingly.

Before you start

• A SearXNG instance with JSON output enabled. The guide to the SearXNG JSON API does it in three steps.
• uv, which runs searxNcrawl with uvx without a separate install.
• Room for Chromium: the first crawl downloads Playwright's Chromium, about 550 MB.

uv downloads the package on the first run and caches it, so later starts are quick. To give a whole team the same release, start it as uvx searxncrawl@0.33.1 in the MCP configuration.

Set up every agent in one step

The setup command finds Claude Code, Codex, Gemini CLI, Cursor, VS Code and Claude Desktop, asks for your SearXNG URL and an optional You.com API key, shows each change and backs up every file before it writes. uvx searxncrawl setup --remove undoes it.

uvx searxncrawl setup

The command edits the configuration files of the clients it finds and stores your SearXNG address there. Run it again after an update or to change the address or the key: it updates its own entries instead of adding duplicates.

Claude Code

claude mcp add searxncrawl --env SEARXNG_URL=http://localhost:8888 -- uvx searxncrawl

This registers searxNcrawl as an MCP server named searxncrawl and passes the SearXNG address as an environment variable. Add --env YDC_API_KEY=... the same way to enable You.com as a second provider.

Codex

codex mcp add searxncrawl --env SEARXNG_URL=http://localhost:8888 -- uvx searxncrawl

Like Claude Code, Codex passes the SearXNG address as an environment variable to the server it starts; add --env YDC_API_KEY=... to enable You.com as a second provider as well.

VS Code

code --add-mcp '{"name":"searxncrawl","command":"uvx","args":["searxncrawl"],"env":{"SEARXNG_URL":"http://localhost:8888"}}'

Cursor, Claude Desktop and other clients

Clients that read an mcpServers file take this block; replace the address with the one of your SearXNG instance.

{
  "mcpServers": {
    "crawler": {
      "command": "uvx",
      "args": ["searxncrawl"],
      "env": { "SEARXNG_URL": "http://your-searxng:8888" }
    }
  }
}

Claude Desktop can also install searxNcrawl from the bundle searxncrawl-<version>.mcpb on the GitHub releases page; the bundle asks for the SearXNG address, optional basic authentication and extra headers in a form.

LM Studio

LM Studio connects MCP servers since version 0.3.17 and, as its MCP documentation says, follows the mcp.json notation of Cursor, so the mcpServers block above works unchanged. Open the Program tab in the right sidebar, choose Install and then Edit mcp.json, and paste the block. LM Studio starts uvx itself, so uv has to be installed on the same machine.

OpenCode

OpenCode reads MCP servers from the mcp key of its opencode.json. A local server has the type local, its command as an array and its variables under environment (OpenCode MCP documentation):

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "searxncrawl": {
      "type": "local",
      "command": ["uvx", "searxncrawl"],
      "environment": { "SEARXNG_URL": "http://your-searxng:8888" },
      "enabled": true
    }
  }
}

OpenClaw

openclaw mcp add searxncrawl --command uvx --arg searxncrawl --env SEARXNG_URL=http://your-searxng:8888

OpenClaw keeps MCP servers in its own registry and passes them to the agent runs it manages; openclaw mcp add saves the command, its argument and the variable (OpenClaw MCP documentation). OpenClaw also has a SearXNG provider for its own web_search tool (SearXNG search in OpenClaw). That provider only searches; searxNcrawl adds crawl and crawl_site for the pages behind the results.

Crawl pages behind a login

Some documentation and dashboards are only visible after a login. Capture the session once with crawl-capture, from a login window or from a browser that is already signed in; it saves a Playwright storage state file. Pass that file as storage_state to crawl or crawl_site, and the crawler sees the pages as you do. Treat the file like a password, since it holds the cookies of your session.

Remote clients over HTTP

Remote and browser-based clients connect over HTTP; the server listens on port 8000 and serves the MCP endpoint at /mcp. For shared access, put an authenticated TLS proxy in front of it and set FASTMCP_HTTP_HOST_ORIGIN_PROTECTION=true to enable the Host and Origin checks.

SEARXNG_URL=http://localhost:8888 uvx searxncrawl --transport http

Choose STDIO when the agent runs on the same machine: the client starts searxNcrawl as a child process and nothing listens on the network. Choose HTTP when several clients share one server or the client runs elsewhere, and keep SEARXNG_URL and any keys on that server. Host and Origin allowlists alone neither enable the checks nor authenticate callers, so the proxy in front stays necessary.

Open WebUI

Open WebUI supports MCP natively since v0.6.31, over Streamable HTTP only (Open WebUI MCP documentation). Start searxNcrawl with --transport http as above, open Settings, Admin, Integrations and add a connection under External Tool Servers:

Type

MCP (Streamable HTTP), not OpenAPI

URL

http://localhost:8000/mcp, or http://host.docker.internal:8000/mcp when Open WebUI runs in Docker

Auth

None on your own machine; a token when a proxy in front of searxNcrawl asks for one

For servers that speak only STDIO, Open WebUI points to its proxy mcpo. searxNcrawl speaks HTTP itself, so it needs no proxy for Open WebUI.

Troubleshooting

• SearXNG answers 403: JSON output is off. The guide to the SearXNG JSON API turns it on.
• No search provider configured: since 0.32 there is no default address, so set SEARXNG_URL, YDC_API_KEY or both; setup warns when neither is set.
• 429 from a provider: searxNcrawl retries once when Retry-After asks for at most 5 seconds; otherwise the error says when to try again.
• You.com answers 401 or 403: 401 means a missing or wrong key, 403 a key without access to the search API.
• Results from some SearXNG engines are missing: the engine rate-limited or challenged your instance; the other engines still answer.
• The first crawl takes long: Chromium downloads once, about 550 MB. Where browsers are provisioned separately, set PLAYWRIGHT_AUTO_INSTALL=false.

Configuration

SEARXNG_URL

Address of your SearXNG instance, for example http://localhost:8888

SEARXNG_USERNAME, SEARXNG_PASSWORD

Basic authentication for SearXNG, both or neither

SEARXNG_EXTRA_HEADERS

Extra headers for SearXNG requests; values can be read from files

YDC_API_KEY

Your You.com API key, to add You.com as a second provider (opt-in)

PLAYWRIGHT_AUTO_INSTALL

false skips the automatic Chromium download

These options describe searxNcrawl 0.33.1; the MCP server documentation on GitHub lists every parameter.

Keep secrets out of shared configuration files. SEARXNG_EXTRA_HEADERS accepts references such as @~/.config/searxncrawl/cf_secret, which searxNcrawl reads from that file at start. Invalid headers switch SearXNG off without printing their values, and You.com stays available.

How we tested this page

We ran the CLI search above on 2026-11-04 with searxNcrawl 0.33.1 against SearXNG 2026.9.15 in Docker, set up as in the JSON API guide. searxNcrawl asked SearXNG for format=json with language, safe search and page set, and printed three results attributed to searxng. The configurations for LM Studio, OpenCode, OpenClaw and Open WebUI follow the documentation linked in their sections as we read it on 2026-11-04; we did not run those four clients.

We reviewed these facts on 2026-11-04 against the linked sources. Our editorial and corrections policy says how we keep them current and fix errors.

Frequently asked questions