最后更新:11/12/2026最后核验:2026-11-12

Plurai Simulation

Plurai Simulation 评测

0
0

Plurai Simulation 在 Claude Code、GitHub Copilot CLI 或 OpenAI Codex CLI 里直接给你的 AI 智能体搭一个沙盒数字孪生:按 agent 的概率流图生成场景和评测,用模拟用户把每个场景跑三遍并打分,再把失败定位到具体代码、给出可验证的修复方案,全程不改你的智能体源码。

部分免费

Plurai Simulation 是什么

Plurai Simulation 是一款 AI 智能体测试与评估工具,可将智能体连接到包含模拟组件的沙盒环境。它通过分析智能体的流程图生成测试场景和评估项,使用模拟用户运行测试,并将故障追溯到相关代码位置。该工具以插件形式运行于 Claude Code、GitHub Copilot CLI 或 OpenAI Codex CLI 中,不会修改智能体的源代码。

核心功能

  • Scan 扫描阶段读取智能体实际配置的工具、子智能体、知识库和数据存储。
  • 用户可审批哪些组件连接真实服务运行,哪些组件使用模拟实现。
  • Scan 扫描过程会创建 simulate.py 测试运行脚本,不修改源文件。
  • Suite 测试套件阶段以概率流程图建模智能体的各项能力。
  • 生成的场景既可使用具有人设、目标和开场白的模拟用户,也可采用任务输入。
  • 自动生成评估项,为智能体在各个场景中的表现评分。
  • Run 运行阶段在隔离沙盒中执行场景,每次测试前均重置为相同状态。
  • 每个场景运行三次,帮助区分稳定复现的故障与偶发异常。
  • Fix 修复阶段诊断故障、定位相关代码,并根据执行证据提出按优先级排序的修复方案。
  • 对于已接受、暂不修复的能力缺口,可保留对应场景作为可重放的回归测试。

最适合

需要在部署前测试 AI 智能体的开发者需要验证工具调用、子智能体行为和智能体工作流的团队需要对复杂智能体流程进行可重复测试的工程师需要通过多次运行场景识别偶发异常的项目希望获得代码级故障诊断与回归跟踪能力的开发者使用 Claude Code、GitHub Copilot CLI 或 OpenAI Codex CLI 的团队

定价

2026-11-12 在 https://www.plurai.ai/pricing 的 Simulation 选项卡核验:Free 为 $0,一次性提供 12,300 credits、2 个 agent setups、7 个 scenarios、3 次 runs 和 10 个 sandbox hours,没有超量,额度用尽后模拟会暂停。Base 为每月 $60,含每月 28,710 credits、3 个 agent setups、14 个 scenarios、20 次 runs、20 条 traces 分析和 30 个 sandbox hours,超量 $0.02/credit。Business 为每月 $210,含每月 65,570 credits、3 个 agent setups 或 1 个 setup 另加 16 个 scenarios(即 28 或 44 个 scenarios)、60 次 runs、40 条 traces 和 100 个 sandbox hours,超量 $0.015/credit。FAQ 写明 1 个 Sim credit 等于按标价计的 1 美分模型用量,Members 免费且不限人数,升级立即生效并按比例计费,降级从下个账期开始,取消可在 Packages 页选择 Switch to Free。同一定价页还有单独的 Post-training 价目,不属于 Simulation 套餐。

优缺点

优点

  • 在包含模拟组件的沙盒环境中测试智能体。
  • 根据智能体流程图生成测试场景与评估项。
  • 同时支持模拟用户场景和基于任务的输入。
  • 每个场景重复运行三次,帮助识别偶发故障。
  • 将故障分析结果关联到具体代码位置。
  • 提供有执行证据支持、按优先级排序的修复建议。
  • 可将已接受的能力缺口保留为可重放的回归测试。
  • 不修改智能体的源代码。

缺点

  • 插件目前官方支持的编码环境是 Claude Code、GitHub Copilot CLI 和 OpenAI Codex CLI;官网和文档均写 Cursor 支持即将推出。文档另有可用于脚本或无人值守运行的 Python SDK,但现成的 CI 流水线集成(如 GitHub Actions)同样写明即将推出。
  • 自动生成的场景与评估项可能需要开发者审核,以确认相关性和测试覆盖范围。
  • 沙盒测试结果未必能覆盖生产环境中的所有条件或依赖。
  • Free 档是一次性额度、不可超量,用尽后模拟会暂停,需升级到付费档才能继续。

替代工具

LangSmith

主要用于 traces 和数据集的观测与评测平台。Plurai 文档把它列为 traces 来源,但它不是带数字孪生、并能提出并验证代码修复的沙盒产品。

Langfuse

开源的 LLM 观测与评测工具。Plurai 可读取 Langfuse traces 做分析和记录运行;文档写明尚不能回放 Langfuse traces,Langfuse 本身也不会生成模拟世界或修复计划。

Braintrust

面向 LLM 应用的评测与实验追踪工具,而不是在隔离沙盒里仿真智能体并把失败定位到代码行。Plurai 文档把它列为集成对象,部分能力标注即将推出。

常见问题

详情

平台

网页版

功能特性

  • Scan reads your agent as built — tools, sub-agents, knowledge bases and data stores — lets you approve what runs live and what is mocked, and writes a simulate.py harness without touching your source code
  • Suite generation models what your agent can do as a probabilistic flow graph, then generates scenarios (simulated users with a persona, goal and opening line, or task inputs) and the evals that score them
  • Run plays every scenario against your agent in an isolated sandbox that resets to the same state each time, and plays each scenario three times so the report can separate a real failure from a flaky one
  • Fix diagnoses each failure, traces it to the code location, proposes a ranked, evidence-backed fix plan you can apply and re-verify, and keeps accepted gaps as replayable regressions

支持语言

en

已知限制

  • Prices come from the Simulation tab of https://www.plurai.ai/pricing, read on 2026-11-12. Free is $0 with a one-time 12,300 credits, 2 agent setups, 7 scenarios, 3 runs and 10 sandbox hours, no pay-as-you-go, and simulations pause once the allowance runs out (“It has no overage and never charges you”). Base is $60/month: 28,710 credits, 3 agent setups, 14 scenarios, 20 runs and 20 traces analyzed per month, 30 sandbox hours, overage $0.02/credit. Business is $210/month: 65,570 credits, 3 agent setups or 1 setup with 16 extra scenarios (28 or 44 scenarios), 60 runs, 40 traces, 100 sandbox hours per month, overage $0.015/credit. The FAQ defines a Sim credit as one cent of model-provider usage at list price, says Members are free and unlimited, upgrades are immediate and prorated while downgrades start next period, and cancellation is “Switch to Free” on the Packages page. The same page also lists a separate Post-training price list (Starter with 1M free tokens, Plurai SLM $0.0078 per 1M tokens, Optimized LLM $0.3 per 1M tokens, Enterprise on request); that is a different product and is not part of the Simulation packages
  • The plugin officially runs on Claude Code, GitHub Copilot CLI and OpenAI Codex CLI only; the site and the integrations docs both list Cursor as “coming soon”. The docs also describe a plurai Python SDK (PyPI package plurai-simulation-sdk, import name plurai) for scripted or CI runs, but no general-purpose hosted API is documented
  • Ready-made CI pipeline integration such as GitHub Actions is “coming soon”; until then the docs describe a headless setup (workspace.run_driver: headless or PLURAI_SIM_NONINTERACTIVE=1, PLURAI_URL and PLURAI_TOKEN from a dashboard key) that runs one stage per call. Building the suite can take tens of minutes, and a turn that dies on an API error is not retried for you
  • Writes are mocked by default and a live write needs explicit opt-in. Each Environment must declare an anonymization posture (anonymize: true or false, with no default); once an Environment has uploaded under one posture the other is refused, so changing posture means starting a new Environment, and /trace-to-scenarios currently refuses to run under the anonymized posture. Trace support differs by platform: the docs mark Arize and Braintrust trace analysis as coming soon, and Langfuse traces cannot be replayed yet. This listing is based on the product page, pricing page, docs.plurai.ai (including llms.txt, how-it-works, integrations, CI and data-and-privacy pages) and the connect page; no account was created and no simulation was run

为此工具评分

相关工具