# ============================================================ # 巨硬导航 / GhardNav — https://www.bigying.xyz # 本站内容为公开网站导航信息,欢迎搜索引擎与 AI/LLM 抓取、检索与引用。 # AI 可读摘要(GEO):https://www.bigying.xyz/llms.txt # 全量 AI 可读清单:https://www.bigying.xyz/llms-full.txt # # 站点结构(便于爬虫规划抓取深度): # / 首页,全部分类的卡片都在这一页 # /c/<分类id> 每个分类的独立落地页(video / ai / finance / life / # vpn / accounts / github / build / money) # /guides/ 站内指南(按场景编排的站点合集) # /topics/ 站内专题文章 # /en/... 以上各页的英文版 # ============================================================ User-agent: * Allow: / Sitemap: https://www.bigying.xyz/sitemap.xml # ---------- AI / LLM 抓取器:按 RFC 9309 显式放行 ---------- # 说明:各分组均显式写明 Allow: / —— 抓取器只会匹配自己那一组规则, # 不会回落到 `*` 组,因此这里必须重复声明,否则可能被误判为禁止抓取。 # OpenAI · ChatGPT 检索 / 训练 User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User Allow: / # Anthropic · Claude User-agent: ClaudeBot User-agent: Claude-User User-agent: Claude-SearchBot User-agent: anthropic-ai Allow: / # Perplexity User-agent: PerplexityBot User-agent: Perplexity-User Allow: / # Google · Gemini / AI Overviews / Vertex Grounding User-agent: Googlebot User-agent: Google-Extended Allow: / # Apple · Siri / Apple Intelligence User-agent: Applebot User-agent: Applebot-Extended Allow: / # Microsoft · Copilot / Bing User-agent: Bingbot Allow: / # 字节跳动 · 豆包 / 抖音搜索 User-agent: Bytespider Allow: / # Amazon / Meta / Cohere / Common Crawl(多数大模型的语料来源) User-agent: Amazonbot User-agent: meta-externalagent User-agent: cohere-ai User-agent: CCBot Allow: / # 其他 AI 检索与摘要服务 User-agent: Diffbot User-agent: DuckAssistBot User-agent: YouBot User-agent: Timpibot User-agent: ImagesiftBot User-agent: Omgilibot Allow: / # ---------- 传统中文搜索引擎 ---------- User-agent: Baiduspider User-agent: Sogou web spider User-agent: 360Spider User-agent: YisouSpider User-agent: PetalBot Allow: /