# Jobs, waiting and retries

Source: https://developers.novilabs.com/forecasts/guides/jobs-and-retries/

> How long-running work runs as jobs, how to wait without polling, and how to retry safely with idempotency keys and ETags.

A `POST` returns the job and
its `Location` at once; a waiting `GET` reads it.

## The four jobs

| Job | Start, then read | Statuses | Running at once | Kept |
| --- | --- | --- | --- | --- |
| Plan | `POST /plans`, `GET /plans/{id}` | `running` (queued too), `done`, `error` | Your unfinished forecasts, up to `limits.plan_unfinished_forecasts` | Unsaved 90 days; saved until deleted |
| Export | `POST /exports`, `GET /exports/{id}` | `queued`, `running`, `done`, `error`, `cancelled`, `expired` | 20 per model set and 3 per plan by default | Its file for `limits.ttl_hours` |
| Async forecast | `POST /forecasts` with `async: true`, `GET /forecasts/{id}` | `running`, `done`, `error` | 50 per company | An hour after it finishes |
| Plan economics | `POST /plans/{id}/economics`, `GET /plans/{id}/economics/{job_id}` | `running`, `done`, `error` | 4 per company | An hour after it finishes |

Past a cap, retry after one finishes, or after `Retry-After` when given ([codes](https://developers.novilabs.com/forecasts/guides/errors-and-limits/#limits)).
Every job carries `seconds`, its run time. Plans report `progress` as a share done; exports the step, share, rows and
bytes.

## Wait instead of polling

Add `wait` (1 to 25 seconds) to the job's `GET`. It answers when the status changes or the time passes; a finished
job answers at once.

```sh
curl -sS --fail-with-body \
  -H "Authorization: Bearer $NOVI_API_TOKEN" \
  "$NOVI_API_BASE/plans/$PLAN_ID?wait=25"
```

Set your client's timeout above the wait, for example 60 seconds.

## Links and Location

`Location`, `links.self` and an export's `file.url` are paths, such as `/forecasts/api/v1/exports/<id>/file`. Resolve
them against `https://<your-company>.novilabs.io`.

## Retry safely with Idempotency-Key

These calls take an `Idempotency-Key` header: `POST /plans`, `POST /exports`, `POST /plans/{id}/economics`,
`POST /forecast-sources`, `POST /area-sets`, `POST /saved/{collection}`, and `POST /forecasts` with `async: true`
(otherwise the key is ignored).

- Send your own key, 1 to 128 letters, digits and `.`, `_`, `:` or `-`, such as a UUID, on every retry of that
  request.
- The same key with the same body within 24 hours (for an async forecast or plan economics job, while the job is
  kept) returns what the first request made, with `Idempotent-Replayed: true`, and starts nothing.
- The same key with a different body is a conflict error with `details.field` `Idempotency-Key`.
- Only a request that succeeds records its key, so retry a server error or a lost response with the same key.
- Keys are per operation.

> **Tip:**
> An identical `POST /exports` for the same plan also returns the existing export, with `reused: true`, key or not.

## ETags and compression

Send a read's `ETag` back as `If-None-Match`: when nothing changed, the answer is Not Modified, with no body.

| Read | Behavior |
| --- | --- |
| Dataset reads, economics decks, `GET /changelog` | `ETag` |
| A done or failed plan, `GET /plans/{id}` | `ETag` |
| A curve page, `GET /plans/{id}/curves` | `ETag` on each page |
| An export file | Strong `ETag`, `Accept-Ranges: bytes`; `Range` resumes and `If-Range` guards the resume |

Responses over 1 KB are gzipped when you send `Accept-Encoding: gzip`.

## Paging

Lists page with `limit` and an opaque `cursor`. Pass `next_cursor` back until it is `null`, with the same filters
and sort each time. On `GET /plans`, plans created or deleted between pages are neither skipped nor repeated.

## Request ids

Every response carries `X-Request-Id`. Send your own to match your logs, and quote it when you ask Novi about a call.
