幂等批量导入
批量导入请求可能成功，但其响应却丢失了：可能是调用超时、连接中断，或者您的编排器在保存返回的jobId 之前已停止运行。您无法区分“未完成”和“已完成但响应丢失”这两种情况。重试会启动第二个任务，导致每行数据被导入两次；而不重试则可能导致导入失败。
幂等键消除了这种两难选择。为请求标记一个幂等键，携带相同键的重试将返回原始的jobId ，而非创建第二个任务。
支持幂等批量导入需使用 Milvus 3.0.2 或更高版本，或 2.6 系列中的 2.6.24 或更高版本。较旧的服务器会忽略该标头，无论是否重试，每个请求都会启动一个新任务。
发送键
该键作为Idempotency-Key HTTP 头传递，而非包含在请求正文中。
curl -X POST "http://localhost:19530/v2/vectordb/jobs/import/create" \
-H "Authorization: Bearer root:Milvus" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: nightly-2026-09-07-batch-3" \
-d '{"collectionName": "events", "files": [["events/2026-09-07/part-0.parquet"]]}'
resp = bulk_import( url="http://localhost:19530", collection_name="events", files=[[“events/2026-09-07/part-0.parquet”]], headers={IDEMPOTENCY_KEY_HEADER: "nightly-2026-09-07-batch-3"}, ) resp.json()[“data”][“jobId”] # 每次重试的 ID 均保持不变
`IDEMPOTENCY_KEY_HEADER` is `"Idempotency-Key"`; the raw string works too. The `headers` parameter and `IDEMPOTENCY_KEY_HEADER` require pymilvus v3.0.3 or later on the 3.x line, or v2.6.18 or later on the 2.6 line. On an older pymilvus, the `import` line above fails with an `ImportError` at the `IDEMPOTENCY_KEY_HEADER` lookup, and passing `headers` to `bulk_import` has undefined behavior — the call may error, or the import may be submitted without the key.
Poll `/v2/vectordb/jobs/import/describe` with the returned `jobId` as usual. A retry that arrives while the original is still being registered waits for it, so a retry never observes a half-registered job.
## Choosing a key
- **One key per logical request.** Name the unit of work, such as `orders-2026-09-07-batch-3`, or generate a UUID and store it with the work item before you send the request. Include an attempt number if you may need to re-run the same work.
- **Retry with the same key.** Minting a new key on a retry is what duplicates data. The exceptions are listed under [When to send a new key](#when-to-send-a-new-key).
- **Never reuse a key for different files.** Milvus does not compare the two requests. You get the original `jobId` back, and the new files are not imported.
- **Keep it short and printable.** Printable ASCII, at most 256 bytes by default (`streaming.idempotency.maxKeyLength`). Anything else is rejected with error `1100`.
- **Do not put secrets in it.** The key is stored with your data.
## How long a key is remembered
The key is scoped to the collection by its internal ID, so renaming the collection does not break a retry, as long as the retry names the collection by its new name. Dropping and recreating the collection under the same name does break it, and a fresh job is the right answer there.
Milvus remembers an import key for about a day. Treat that as nominal rather than guaranteed: heavy DDL or import traffic shortens it, and a coordinator restart lengthens it. A retry after the key is forgotten starts a fresh job.
## When to send a new key
Two cases, and only two:
- **The original job failed or was aborted.** Describe returns `state: Failed` with a reason. Retrying the key returns that same `jobId` for as long as the key is remembered. Fix the cause, then send a new key.
- **The original job was cleaned up.** Milvus keeps finished jobs for `dataCoord.import.taskRetention` (48 hours by default). If that expires while the key is still remembered, the retry returns the old `jobId`, and describe answers with the error `import job does not exist` instead of a job state. That job may have succeeded. Confirm the data is missing before you send a new key.
A retry that is rejected, for example because the import job limit is full, is not one of these cases. Retry it later with the same key.
## Configuration
Four parameters bound the behavior on this page. None of them needs changing for the key to work.
| Parameter | Default | What it does |
|---|---|---|
| `streaming.walBroadcaster.tombstone.maxLifetime` | `24h` | How long an import key is remembered. Not present in the default `milvus.yaml`; add it explicitly to change it. The same store also rejects duplicate DDL submissions, so do not lower it just to shorten the import window. |
| `streaming.walBroadcaster.tombstone.maxCount` | `8192` | How many keys and DDL records the same store holds before the oldest are dropped, whatever their age. This is what lets heavy DDL or import traffic forget a key before `maxLifetime`. Not present in the default `milvus.yaml`. |
| `dataCoord.import.taskRetention` | `172800` (48 hours) since 2.6.24 and 3.0.2; `10800` before | How long a finished import job stays queryable. Keep it at least twice `maxLifetime`. |
| `streaming.idempotency.maxKeyLength` | `256` | The maximum key length in bytes. A longer key is rejected with error `1100`. `0` disables the bound. Not present in the default `milvus.yaml`. |
The `milvus.yaml` shipped before 2.6.24 and 3.0.2 sets `taskRetention: 10800` explicitly, and an upgrade does not override a value in your file. If you upgraded with that file, raise `taskRetention` to at least twice `maxLifetime`. The new value applies only to jobs that finish after the change.
