feat: add seedance2 ai video compatibility
This commit is contained in:
kris
@@ -0,0 +1,262 @@
|
||||
# Seedance 2.0 AI Video Execution Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** 在不打断现有 AI 视频链的前提下，新增 `Seedance 2.0` 兼容入口，并把前端、主 Agent、执行结果和回归链统一到可扩展的视频引擎模型。
|
||||
|
||||
**Architecture:** 保留现有 `/v2/pipelines/ai-video` 业务入口不变，把视频生成从“固定 huobao/doubao 默认值”升级为“provider-aware execution”。前端新增视频引擎选择与 Seedance 定向字段，后端统一把 provider/version/meta 写进 job artifacts 和结果，主 Agent 透传 provider 进入执行链并在结果卡中可追溯。
|
||||
|
||||
**Tech Stack:** FastAPI, SQLite-backed job records, existing `HuobaoDramaClient` integration layer, vanilla JS SPA, node test runner, unittest
|
||||
|
||||
---
|
||||
|
||||
### Task 1: 后端视频引擎兼容层
|
||||
|
||||
**Files:**
|
||||
- Modify: `collector-service/app/core_main.py`
|
||||
- Modify: `collector-service/app/integrations.py`
|
||||
- Test: `tests/test_production_baseline.py`
|
||||
|
||||
- [ ] **Step 1: 写失败测试，锁定 `video_provider` 会进入 AI 视频 job artifacts 和执行结果**
|
||||
|
||||
```python
|
||||
def test_ai_video_request_persists_video_provider(client, approved_headers):
|
||||
response = client.post(
|
||||
"/v2/pipelines/ai-video",
|
||||
json={
|
||||
"project_id": "project_demo",
|
||||
"assistant_id": "",
|
||||
"knowledge_base_id": "kb_demo",
|
||||
"title": "Seedance test",
|
||||
"brief": "创业者口播视频",
|
||||
"style": "realistic",
|
||||
"shots": 4,
|
||||
"duration": 5,
|
||||
"video_provider": "seedance2",
|
||||
"video_model": "seedance-2.0-pro",
|
||||
},
|
||||
headers=approved_headers,
|
||||
)
|
||||
assert response.status_code == 200
|
||||
payload = response.json()
|
||||
assert payload["artifacts"]["video_provider"] == "seedance2"
|
||||
assert payload["artifacts"]["video_model"] == "seedance-2.0-pro"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: 运行测试确认先失败或至少缺少 `seedance2` 兼容断言**
|
||||
|
||||
Run: `python3 -m unittest tests.test_production_baseline -v`
|
||||
Expected: 现有测试不覆盖 `seedance2`，新增断言失败或行为缺失。
|
||||
|
||||
- [ ] **Step 3: 实现 provider 兼容层**
|
||||
|
||||
```python
|
||||
def normalize_ai_video_provider(provider: str, model: str) -> tuple[str, str]:
|
||||
normalized = str(provider or "").strip().lower()
|
||||
if normalized in {"seedance", "seedance2", "seedance-2.0", "seedance_2_0"}:
|
||||
return "seedance2", model or "seedance-2.0-pro"
|
||||
if normalized in {"doubao", "jimeng", "huobao"}:
|
||||
return "doubao", model
|
||||
return normalized or "doubao", model
|
||||
```
|
||||
|
||||
```python
|
||||
video_provider, video_model = normalize_ai_video_provider(request.video_provider, request.video_model)
|
||||
```
|
||||
|
||||
```python
|
||||
"video_provider": video_provider,
|
||||
"video_model": video_model,
|
||||
"video_provider_label": "Seedance 2.0" if video_provider == "seedance2" else video_provider,
|
||||
```
|
||||
|
||||
- [ ] **Step 4: 在内部渲染步骤里把 provider 元信息回写进结果**
|
||||
|
||||
```python
|
||||
"video_provider": video_provider,
|
||||
"video_model": video_model,
|
||||
"video_provider_label": "Seedance 2.0" if video_provider == "seedance2" else video_provider,
|
||||
```
|
||||
|
||||
- [ ] **Step 5: 再跑后端测试确认通过**
|
||||
|
||||
Run: `python3 -m unittest tests.test_production_baseline -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 6: 提交这一批**
|
||||
|
||||
```bash
|
||||
git add collector-service/app/core_main.py collector-service/app/integrations.py tests/test_production_baseline.py
|
||||
git commit -m "feat: add seedance2 ai video provider support"
|
||||
```
|
||||
|
||||
### Task 2: 前端 AI 视频任务表单与结果展示
|
||||
|
||||
**Files:**
|
||||
- Modify: `web/storyforge-web-v4/assets/app.js`
|
||||
- Test: `web/storyforge-web-v4/tests/workbench-pages.test.mjs`
|
||||
|
||||
- [ ] **Step 1: 写失败测试，锁定 AI 视频表单出现视频引擎选择和 Seedance 定向字段**
|
||||
|
||||
```js
|
||||
test("ai video action exposes seedance 2.0 provider controls", () => {
|
||||
const aiVideo = extractBetween(APP, "function openCreateAiVideoAction(defaults = {})", "function openCreateRealCutAction(defaults = {})");
|
||||
assert.match(aiVideo, /videoProvider/);
|
||||
assert.match(aiVideo, /Seedance 2.0/);
|
||||
assert.match(aiVideo, /cameraMotion/);
|
||||
assert.match(aiVideo, /visualStyle/);
|
||||
});
|
||||
```
|
||||
|
||||
- [ ] **Step 2: 跑测试确认先失败**
|
||||
|
||||
Run: `node --test web/storyforge-web-v4/tests/workbench-pages.test.mjs`
|
||||
Expected: FAIL because current form has no Seedance fields.
|
||||
|
||||
- [ ] **Step 3: 实现前端表单扩展**
|
||||
|
||||
```js
|
||||
{ name: "videoProvider", label: "视频引擎", type: "select", value: defaults.videoProvider || "doubao", options: [
|
||||
{ value: "doubao", label: "当前默认" },
|
||||
{ value: "seedance2", label: "Seedance 2.0" }
|
||||
] }
|
||||
```
|
||||
|
||||
```js
|
||||
{ name: "cameraMotion", label: "镜头运动", value: defaults.cameraMotion || "", placeholder: "例如：慢推近、环绕、手持跟拍" },
|
||||
{ name: "visualStyle", label: "画面风格", value: defaults.visualStyle || "", placeholder: "例如：电影感、高反差、创业访谈质感" },
|
||||
```
|
||||
|
||||
- [ ] **Step 4: 提交时把 provider 和 Seedance 定向字段发给后端**
|
||||
|
||||
```js
|
||||
video_provider: values.videoProvider || "doubao",
|
||||
video_model: values.videoProvider === "seedance2" ? "seedance-2.0-pro" : "",
|
||||
camera_motion: values.cameraMotion || "",
|
||||
visual_style: values.visualStyle || "",
|
||||
```
|
||||
|
||||
- [ ] **Step 5: 在 job 详情/结果卡里显示视频引擎**
|
||||
|
||||
```js
|
||||
<span class="tag blue">${escapeHtml(job.artifacts?.video_provider_label || job.artifacts?.video_provider || "默认引擎")}</span>
|
||||
```
|
||||
|
||||
- [ ] **Step 6: 跑前端测试确认通过**
|
||||
|
||||
Run: `node --test web/storyforge-web-v4/tests/workbench-pages.test.mjs`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 7: 提交这一批**
|
||||
|
||||
```bash
|
||||
git add web/storyforge-web-v4/assets/app.js web/storyforge-web-v4/tests/workbench-pages.test.mjs
|
||||
git commit -m "feat: add seedance2 controls to ai video flow"
|
||||
```
|
||||
|
||||
### Task 3: 主 Agent 透传与执行追溯
|
||||
|
||||
**Files:**
|
||||
- Modify: `collector-service/app/oneliner_features.py`
|
||||
- Test: `tests/test_main_agent_governance.py`
|
||||
|
||||
- [ ] **Step 1: 写失败测试，锁定主 Agent 创建 AI 视频任务时会透传 provider**
|
||||
|
||||
```python
|
||||
def test_oneliner_ai_video_run_preserves_video_provider(client, approved_headers):
|
||||
response = client.post(
|
||||
"/v2/oneliner/actions/execute",
|
||||
json={
|
||||
"action_key": "create-ai-video",
|
||||
"project_id": "project_demo",
|
||||
"platform": "douyin",
|
||||
"payload": {
|
||||
"video_provider": "seedance2",
|
||||
"video_model": "seedance-2.0-pro",
|
||||
},
|
||||
},
|
||||
headers=approved_headers,
|
||||
)
|
||||
assert response.status_code == 200
|
||||
payload = response.json()
|
||||
assert payload["payload"]["job"]["artifacts"]["video_provider"] == "seedance2"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: 跑测试确认先失败**
|
||||
|
||||
Run: `python3 -m unittest tests.test_main_agent_governance -v`
|
||||
Expected: FAIL because provider not yet propagated.
|
||||
|
||||
- [ ] **Step 3: 实现主 Agent 透传**
|
||||
|
||||
```python
|
||||
video_provider=str(requested_payload.get("video_provider") or requested_payload.get("videoProvider") or "doubao"),
|
||||
video_model=str(requested_payload.get("video_model") or requested_payload.get("videoModel") or ""),
|
||||
```
|
||||
|
||||
```python
|
||||
"video_provider": job.get("artifacts", {}).get("video_provider", ""),
|
||||
"video_provider_label": job.get("artifacts", {}).get("video_provider_label", ""),
|
||||
```
|
||||
|
||||
- [ ] **Step 4: 结果卡补充视频引擎标签**
|
||||
|
||||
```python
|
||||
result_sections.append({
|
||||
"title": "视频引擎",
|
||||
"items": [{"label": "本轮视频引擎", "value": provider_label}],
|
||||
})
|
||||
```
|
||||
|
||||
- [ ] **Step 5: 跑后端治理测试确认通过**
|
||||
|
||||
Run: `python3 -m unittest tests.test_main_agent_governance -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 6: 提交这一批**
|
||||
|
||||
```bash
|
||||
git add collector-service/app/oneliner_features.py tests/test_main_agent_governance.py
|
||||
git commit -m "feat: propagate seedance2 through main agent"
|
||||
```
|
||||
|
||||
### Task 4: 整体回归、部署与版本记录
|
||||
|
||||
**Files:**
|
||||
- Modify: `CHANGELOG.md`
|
||||
|
||||
- [ ] **Step 1: 更新版本记录**
|
||||
|
||||
```markdown
|
||||
### Seedance 2.0 生视频兼容接入
|
||||
|
||||
- AI 视频链新增 `Seedance 2.0` 视频引擎入口，并统一把 provider/version 回写到任务结果与主 Agent 执行卡。
|
||||
- 前端 AI 视频创建表单新增视频引擎选择与 Seedance 定向参数。
|
||||
- 主 Agent 创建 AI 视频任务时会透传视频引擎，结果页可追溯本轮使用的引擎。
|
||||
```
|
||||
|
||||
- [ ] **Step 2: 跑全量回归**
|
||||
|
||||
Run: `python3 -m unittest tests.test_main_agent_governance tests.test_platform_contracts tests.test_production_baseline -v`
|
||||
Expected: PASS
|
||||
|
||||
Run: `node --test web/storyforge-web-v4/tests/dashboard-home.test.mjs web/storyforge-web-v4/tests/workbench-pages.test.mjs`
|
||||
Expected: PASS
|
||||
|
||||
Run: `bash scripts/check_repo_baseline.sh`
|
||||
Expected: `baseline checks passed`
|
||||
|
||||
- [ ] **Step 3: 发布到 NAS 并跑 smoke**
|
||||
|
||||
Run: `FNOS_PASSWORD='Admin_yqs_asd20260101.' bash /Users/kris/code/StoryForge-gitea/scripts/deploy_fnos_storyforge_web.sh`
|
||||
Expected: `fnOS StoryForge Web V4 ready: http://192.168.31.188:19192/`
|
||||
|
||||
Run: `bash /Users/kris/code/StoryForge-gitea/scripts/smoke_fnos_storyforge_lan.sh`
|
||||
Expected: `fnOS lan smoke passed`
|
||||
|
||||
- [ ] **Step 4: 提交汇总版本**
|
||||
|
||||
```bash
|
||||
git add CHANGELOG.md
|
||||
git commit -m "chore: record seedance2 ai video rollout"
|
||||
```
|
||||
Reference in New Issue
Block a user