# Prompt Set — smoke battery v1 rerun（2026-08-07 补测）

> 预注册（pre-registered）：本 prompt 集在**任何 API 调用之前**定稿（git 提交前即写死），两模型使用完全相同的 prompt。
> 来源声明：2026-08-03 首测的逐字 prompt 从未存档（该事实已在本页时间线与 raw-logs 归档中如实披露）；本补测 prompt 依据 `code/evaluation-toolkit/benchmarks/task-battery.json`（git 4d467ac）的任务定义（prompt_style / check / ground_truth / notes）**重建**，任务语义一致，但**不声称**与 2026-08-03 逐字相同。
> 采样参数（两模型一致）：temperature=default · top_p=default · max_tokens 见各任务 · thinking=disabled（`{"type": "disabled"}`）· N=1

---

## 1. ipv4-validator（coding）

```text
Write a Python function is_valid_ipv4(s: str) -> bool that returns True if s is a valid IPv4 address and False otherwise. An IPv4 address consists of exactly four decimal octets separated by dots; each octet must be between 0 and 255, must contain no leading zeros (except the octet '0' itself), and must contain only digits. Return only the function definition, no explanation, no test code.
```

- max_tokens: 512
- 判定：提取代码执行，9 个预注册边界用例（`192.168.1.1`→T, `0.0.0.0`→T, `255.255.255.255`→T, `1.2.3.04`→F(前导零), `01.2.3.4`→F(前导零), `256.1.1.1`→F(超界), `1.2.3`→F(3段), `1.2.3.4.5`→F(5段), `1.2.3.abc`→F(非数字)）

## 2. bugfix-second-largest（coding）

```text
The function below is supposed to return the second-largest DISTINCT element of a list of integers, or None if fewer than two distinct elements exist. It has a bug. Fix it.

```python
def second_largest(xs):
    # BUG: fails when the largest value appears more than once
    xs = sorted(xs)
    return xs[-2]
```

Return only the corrected function definition, no explanation, no test code.
```

- max_tokens: 512
- 判定：提取修复后代码执行，6 个预注册用例（`[3,1,2]`→2, `[5,5,4]`→4, `[10,10,10,9]`→9, `[-1,-2,-3]`→-2, `[7,7,7,7,7,6,8]`→7, `[100,1,100,2]`→2）

## 3. math-quadratic-roots（math）

```text
Solve for x: x^2 - x - 12 = 0. Answer with just the number(s), no explanation.
```

- max_tokens: 128
- 判定：输出同时包含根 `4` 与 `-3`（b+c = -1+(-12) = -13，与任务定义一致）

## 4. math-consecutive-integers（math）

```text
Two consecutive integers sum to 575. What are they? Answer with just the number(s), no explanation.
```

- max_tokens: 128
- 判定：输出同时包含 `287` 与 `288`（23×25 = 575 的来源）

## 5. logic-syllogism（math）

```text
Premises: All humans are mortal. Socrates is a human. Conclusion: Socrates is mortal. Does the conclusion follow from the premises? Answer yes or no.
```

- max_tokens: 64
- 判定：输出含 `yes`

## 6. knowledge-us-president（multilingual）

```text
Who was the 36th president of the United States? Answer with just the name.
```

- max_tokens: 64
- 判定：输出含 `Johnson`（ground truth: Lyndon B. Johnson）

## 7. knowledge-burkina-faso-capital（multilingual）

```text
What is the capital of Burkina Faso? Answer with just the city name.
```

- max_tokens: 64
- 判定：输出含 `Ouagadougou`

## 8. writing-product-copy（multilingual）

```text
Write exactly 3 sentences of product copy for CloudShelf — a self-hosted document store for small engineering teams. Each sentence must be at most 25 words. Mention the product name CloudShelf. No headings, no bullet points, no extra text.
```

- max_tokens: 256
- 判定（rubric 预注册）：恰好 3 句、每句 ≤25 词、提及产品名

## 9. extraction-dates-names-json（coding）

```text
Extract all dates and all person names from the passage below into a JSON object with two keys: "dates" (array of date strings in YYYY-MM-DD format) and "names" (array of person names). Return only the JSON object, no explanation.

Passage:
On 2026-08-03, the Benchmarks Desk at China AI Bench published its first smoke battery results. The report was compiled by Eli Chen, the desk's lead editor, with assistance from Mei Lin, who reviewed the sampling parameters. The follow-up verification run was completed on 2026-07-31, three days before publication, and the final revision was approved by the editorial board on August 5, 2026.
```

- max_tokens: 256
- 判定：JSON 解析后 `dates` ⊇ {2026-08-03, 2026-07-31, 2026-08-05}，`names` ⊇ {Eli Chen, Mei Lin}

## 10. summary-3-item-json（coding）

```text
Summarize the passage below as a JSON array of exactly 3 strings, each string being one concise point. Return only the JSON array, no explanation.

Passage:
On 2026-08-03, the Benchmarks Desk at China AI Bench published its first smoke battery results. The report was compiled by Eli Chen, the desk's lead editor, with assistance from Mei Lin, who reviewed the sampling parameters. The follow-up verification run was completed on 2026-07-31, three days before publication, and the final revision was approved by the editorial board on August 5, 2026.
```

- max_tokens: 256
- 判定：JSON 解析为长度 3 的非空字符串数组

## 11. needle-426-token（longContext）

```text
Read the deployment report below and answer: what is the serial number of the primary gateway node gw-01? Answer with just the serial number, no explanation.

Report:
CloudShelf deployment report — production cluster EU-west
Prepared by the platform engineering group. Revision 4.2. Date of issue: 2026-07-28.

1. Summary
The EU-west production cluster was upgraded from release 3.9 to release 4.0 during the maintenance window of July 25, 2026. The upgrade completed in 41 minutes with no data loss. All fourteen application nodes and both gateway nodes are now running the new image tag. Rollback procedures were not exercised because no critical regression was observed in the first twelve hours of post-upgrade monitoring.

2. Node inventory
The cluster comprises fourteen application nodes (app-01 through app-14), two gateway nodes (gw-01, gw-02), one primary database instance (db-primary) and one replica (db-replica). Each application node runs eight worker processes with a shared memory pool of 4 GiB. The gateway nodes terminate TLS and route traffic on ports 443 and 8443. The database runs PostgreSQL 17 with a 512 GiB buffer cache.

3. Configuration changes
Release 4.0 introduced three notable changes. First, the request cache now respects the Vary header, which reduced upstream calls by 18 percent under the July load test. Second, connection pooling was moved from the application layer into the database driver, cutting average connection establishment latency from 9.2 ms to 3.4 ms. Third, the metrics endpoint now exposes histogram buckets for p50, p95 and p99 so that dashboards no longer need to aggregate raw counters.

4. Security and compliance
All certificates were rotated on July 27, 2026. The new certificate bundle expires on October 25, 2026. Access to the admin API is now restricted to the operations subnet, and every administrative action is written to the audit log. The quarterly penetration test report is pending publication; no findings from the previous cycle remain open.

5. Incident notes
One incident was logged on July 26, 2026: a single application node (app-09) experienced elevated memory pressure for 22 minutes during a deployment of the search index. The node remained available, and the pressure subsided once the index warm-up completed. The primary gateway node gw-01 carries the serial number XK-9472 and its firmware was updated to version 2.1.8 as part of this maintenance window. The standby gateway gw-02 carries serial number XK-9473 and was not modified.

6. Next steps
The next maintenance window is scheduled for August 22, 2026. Planned work includes the introduction of read replicas for the search index and a hardware refresh for the two oldest application nodes. A capacity review for the 2026 holiday traffic peak is scheduled for September 3, 2026.
```

- max_tokens: 64
- 判定：输出含 `XK-9472`（报告实际 token 数以 API usage.prompt_tokens 为准，见各调用记录）

---

## 补测判定汇总（2026-08-07，真实 API 调用）

| 任务 | V4 Pro | V4 Flash-0731 |
|---|---|---|
| ipv4-validator | PASS | PASS |
| bugfix-second-largest | PASS | PASS |
| math-quadratic-roots | PASS | **FAIL**（答 "9, -4"，缺 -3） |
| math-consecutive-integers | PASS | PASS（287/288） |
| logic-syllogism | PASS | PASS |
| knowledge-us-president | PASS | PASS |
| knowledge-burkina-faso-capital | PASS | PASS |
| writing-product-copy | PASS | PASS |
| extraction-dates-names-json | PASS | PASS |
| summary-3-item-json | PASS | PASS |
| needle-426-token | PASS | PASS |
| **Score** | **11/11** | **10/11** |

总成本：$0.001228（22 次调用，按官方定价 $0.435/$0.87 Pro、$0.14/$0.28 Flash 逐调用实测 token 计）。
补测结果与原测（2026-08-03）分数一致（11/11 与 10/11），但 Flash 的失败项不同（原测 23×25→341，补测 quadratic→"9, -4"）——如实记录，不合并、不抹平。