PkgFoundry
Catalog PyPI
Back to packageRELEASE DETAILS

0.1.8

PyPI

agent-web-search-engine

AI Agent web search & intelligent filtering engine — bundled with SearXNG-core

Published17 Jul 2026
Direct dependencies31
Transitive dependenciesNot calculated

README

Version 0.1.8 documentation

Agent Web Search Engine

Python 核心引擎，提供搜索编排、网页爬取、内容提取和向量检索能力。

SearXNG-core 已打包在 wheel 中，安装后引擎自动管理 SearXNG 子进程，无需额外部署。

快速开始

# 克隆仓库（含 searxng-core submodule）
git clone --recurse-submodules <repo-url>
cd AgentWebSearchingTool/engine

# 打包 searxng-core 到包中（仅开发模式需要）
python scripts/prepare_build.py

# 创建虚拟环境并安装
python -m venv .venv
source .venv/bin/activate    # Windows: .venv\Scripts\activate
pip install -e ".[dev,retrieval]"

# 运行测试
pytest

# 启动服务（SearXNG 会自动在后台启动）
uvicorn src.router:app --reload --port 8000

pip 安装用户: pip install agent-web-search-engine 后，SearXNG 已内置在包中，引擎会自动启动它。

API 端点

方法路径说明
GET/health健康检查（含 SearXNG 状态）
POST/search执行搜索（自动使用内置 SearXNG）
POST/fetch抓取单页 + 内容提取
POST/scrape批量抓取多页
POST/filter向量检索筛选

模块架构

src/
├── router.py            # FastAPI 路由编排（含 SearXNG 生命周期管理）
├── searxng_runner.py    # SearXNG 子进程管理器（启动/停止/健康检查）
├── config.py            # YAML 配置加载
├── search/              # SearchProvider 接口 + SearXNG 适配器
├── fetch/               # HTTP + Playwright 混合爬取
├── extract/             # trafilatura 内容提取
└── retrieval/           # Embedding + Cross-encoder 两阶段检索

配置

编辑 config.yaml 或通过环境变量覆盖:

# SearXNG 配置（内置，通常无需修改）
SEARXNG_PORT=8888
SEARXNG_BIND_ADDRESS=127.0.0.1
SEARXNG_SECRET=my-secret-key      # 建议设置

# Engine 配置
SEARXNG_URL=http://127.0.0.1:8888
ENGINE_PORT=8000
DASHSCOPE_API_KEY=sk-xxx          # DashScope Embedding

依赖

用途安装方式
核心（含 SearXNG-core）pip install agent-web-search-engine
开发 + 测试pip install -e ".[dev]"
检索pip install -e ".[retrieval]" (dashscope, openai, torch, transformers)
浏览器爬取pip install -e ".[playwright]" + playwright install chromium

License

  • Engine 代码: MIT
  • Bundled SearXNG-core: AGPL-3.0-or-later

Downloads & popularity

Awaiting observations

No metric observations yet

Download counts and popularity will appear here as the collector receives them.

Declared dependencies 32

Native requirements
31 direct targetsNot calculated transitive count
Package / targetRequirementRelationship
fastapi
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
fastapi>=0.100runtime
uvicorn
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
uvicorn>=0.23runtime
httpx
Environment
{
  "extras": "http2",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
httpx[http2]>=0.25runtime
httpx-socks
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
httpx-socks>=0.10runtime
trafilatura
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
trafilatura>=1.6runtime
lxml
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
lxml>=5.0runtime
lxml_html_clean
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
lxml_html_clean>=0.1runtime
pyyaml
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
pyyaml>=6.0runtime
numpy
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
numpy>=1.24runtime
flask
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
flask>=3.0runtime
python-dateutil
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
python-dateutil>=2.9runtime
certifi
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
certifi>=2024.0runtime
markdown-it-py
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
markdown-it-py>=4.0runtime
msgspec
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
msgspec>=0.21runtime
isodate
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
isodate>=0.7runtime
typer
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
typer>=0.26runtime
typing-extensions
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
typing-extensions>=4.16runtime
async-timeout
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
async-timeout>=4.0.3runtime
valkey
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
valkey>=6.1runtime
sniffio
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
sniffio>=1.3runtime
Flask-Babel
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
Flask-Babel>=4.0runtime
pytest
Environment
{
  "extras": "",
  "marker": "extra == \"dev\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
pytest>=7.0; extra == "dev"runtime
pytest-asyncio
Environment
{
  "extras": "",
  "marker": "extra == \"dev\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
pytest-asyncio>=0.21; extra == "dev"runtime
pytest-cov
Environment
{
  "extras": "",
  "marker": "extra == \"dev\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
pytest-cov>=4.0; extra == "dev"runtime
ruff
Environment
{
  "extras": "",
  "marker": "extra == \"dev\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
ruff>=0.1; extra == "dev"runtime
httpx-socks
Environment
{
  "extras": "",
  "marker": "extra == \"dev\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
httpx-socks>=0.10; extra == "dev"runtime
playwright
Environment
{
  "extras": "",
  "marker": "extra == \"playwright\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
playwright>=1.40; extra == "playwright"runtime
dashscope
Environment
{
  "extras": "",
  "marker": "extra == \"retrieval\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
dashscope>=1.14; extra == "retrieval"runtime
openai
Environment
{
  "extras": "",
  "marker": "extra == \"retrieval\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
openai>=1.0; extra == "retrieval"runtime
sentence-transformers
Environment
{
  "extras": "",
  "marker": "extra == \"retrieval\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
sentence-transformers>=2.2; extra == "retrieval"runtime
transformers
Environment
{
  "extras": "",
  "marker": "extra == \"retrieval\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
transformers>=4.36; extra == "retrieval"runtime
torch
Environment
{
  "extras": "",
  "marker": "extra == \"retrieval\"",
  "requires_python": ">=3.11",
  "evaluation": "NOT_EVALUATED"
}
torch>=2.0; extra == "retrieval"runtime

Distribution files

2 collected
agent_web_search_engine-0.1.8-py3-none-any.whl

bdist_wheel · 3,665.21 KB · Python py3

Source file
agent_web_search_engine-0.1.8.tar.gz

sdist · 3,437.77 KB · Python source

Source file

Metadata history

Meaningful changes
Current collected state Current

Observed since 21 Oct, 08:56 UTC

No superseded metadata states collected. Unchanged observations do not create new revisions.

Record identity

package_version:d7df5f5fcb4fe5b112aa8f4fcf6253dbd3bc248e258fa1f13cef7f75b55788ab

Other collected versions

View all collected versions →