# Query history settings

> Control what Orbital records about each query - results, remote calls, traces and errors - per server, per workspace, or per query.

Source: https://orbitalhq.com/docs/workspace/query-history-settings

As Orbital runs queries, it records what happened: the results returned, the calls made to
remote services and what they sent back, the trace of how the query was planned, and the details
of any failures. This is what powers the [query history and lineage views](/docs/querying/observability).

Recording everything isn't always what you want. Response bodies can contain data that shouldn't
be kept, some workspaces run high-volume queries where storing every row is wasteful, and
production servers often need to enforce a policy that individual teams can't change.

Query history settings let you decide what gets recorded, at three levels.

## The three levels

| Level | Where it's set | Who sets it |
|---|---|---|
| Server | `orbital.history.*` in the server configuration | The operator running Orbital |
| Workspace | The **Workspace settings** page in the UI, or the `configuration` block in `workspace.conf` | Anyone with the *Edit schema* privilege in the workspace |
| Query | A `@QueryTelemetry` annotation on the query itself | The author of the query |

Each level can override the one above it, with two exceptions that keep the server in control:

 * **The server can lock a setting.** A locked setting can't be changed by a workspace or a query.
 * **Queries only get a say if the workspace lets them.** `@QueryTelemetry` annotations are ignored unless the workspace turns on *Allow per-query overrides*.

Beyond that, the rules are simple. A workspace value beats the server default. A query value beats the workspace value.
Anything a lower level doesn't set is inherited from the level above.

## The settings

The same six settings are available at every level.

| Setting | What it controls | Default |
|---|---|---|
| `persistResults` | Whether the rows returned by a query are stored. Needed to view results in the query history, and to download them later. | `true` |
| `persistErrors` | Whether details of failed queries are stored, so failures can be investigated from the query history. | `true` |
| `persistRemoteCallMetadata` | Whether the URL, timing and status of each call to a remote service are recorded. Needed for the call sequence view. | `true` |
| `persistRemoteCallResponses` | Whether the response bodies from remote services are stored. Needed for value-level lineage, and for downloading a query as a regression pack. | `true` |
| `persistTraceEvents` | Whether the detailed trace of how a query was planned and executed is stored. | `true` |
| `maxPayloadSizeInBytes` | The largest remote response body (in bytes) that will be stored. Larger responses are still recorded, but their body is not kept. | `2048` |

`maxPayloadSizeInBytes` is the one setting that only moves in one direction: a workspace or a query
can lower it, but can never raise it above the server's value.

## Configuring a workspace in the UI

Open **Workspace settings** from the bottom of the side navigation, then the **Query history** tab.
Each setting shows three choices:

 * **Default (On)** or **Default (Off)** - inherit whatever the server has configured. The label shows you what that currently is.
 * **On** - turn the setting on for this workspace, regardless of the server default.
 * **Off** - turn the setting off for this workspace, regardless of the server default.

Underneath each control, **In effect** tells you what the setting resolves to right now, taking
your choice and the server default into account.

The maximum payload size is a number. Leave it blank to inherit the server's limit, or enter a
smaller value. The page won't let you enter a value above the server's limit.

Finally, **Allow per-query overrides** decides whether `@QueryTelemetry` annotations on queries in this
workspace are honoured. It's off by default.

Viewing the page requires the *Browse schema* privilege. Saving changes requires *Edit schema* - without it,
the page is read-only and says so.

### Settings locked by the server

If the server operator has locked a setting, the page shows it with a lock icon and its server value,
and you can't change it. A banner at the top explains which settings are locked. To change a locked
setting, ask whoever operates the server - see [locking settings](#locking-settings-on-the-server) below.

## Configuring a workspace in `workspace.conf`

The UI writes to the `configuration` block of your `workspace.conf`, and you can edit that block directly
instead. Only include the settings you want to override - anything you leave out is inherited from the server.

```hocon
configuration {
   queryPersistence {
      persistResults: false
      persistRemoteCallResponses: false
      maxPayloadSizeInBytes: 1024
      allowQueryOverride: true
   }
}
```

`allowQueryOverride` is the only setting that lives at the workspace level and nowhere else. A workspace can
turn it on without overriding anything else, in which case queries inherit the server defaults but are
allowed to change them.

**Note: Locked settings are rejected**
If you set a value for a setting the server has locked, saving through the UI or the API is rejected
  with an error naming the locked setting. A value written directly into `workspace.conf` for a locked
  setting is ignored.

## Overriding settings on a query

A query can carry a `@QueryTelemetry` annotation to change what's recorded for that query alone.
Any setting you leave out is inherited from the workspace.

```taxi
import com.orbitalhq.telemetry.QueryTelemetry

@QueryTelemetry(persistResults = false, persistRemoteCallResponses = false)
find { Order[] }
```

This is useful for a query that returns bulk data you don't need to keep, or one that passes through
payloads that shouldn't be stored.

### Annotations are off by default

`@QueryTelemetry` has **no effect** until the workspace turns on *Allow per-query overrides*
(`allowQueryOverride: true` in `workspace.conf`). Until then the annotation is still valid and the query
still runs, but the settings on it are ignored and the workspace's settings apply.

This is deliberate: a workspace owner decides whether query authors get to change what's recorded.

### Overrides work in both directions

Once the workspace has opted in, an annotation can turn a setting on that the workspace turned off, or
off that the workspace turned on. For example, with `allowQueryOverride: true`:

| Workspace | Annotation | Result |
|---|---|---|
| `persistResults: false` | `persistResults = true` | Results are stored for this query |
| `persistResults: true` | `persistResults = false` | Results are not stored for this query |
| Not set (server says `true`) | `persistResults = false` | Results are not stored for this query |
| Not set (server says `true`) | No annotation | Results are stored |

Two things an annotation can never do:

 * Change a setting the server has locked.
 * Raise `maxPayloadSizeInBytes` above the server's value. It can lower it further than the workspace did, but not raise it.

## Server configuration

Server defaults are set with the `orbital.history` settings, in the same way as
[any other Orbital configuration](/docs/deploying/configuring-orbital). These were previously named
`vyne.analytics.*`; the old names still work.

```yaml
orbital:
  history:
    persist-results: true
    persist-errors: true
    persist-remote-call-metadata: true
    persist-remote-call-responses: true
    persist-trace-events: true
    max-payload-size-in-bytes: 2048
```

### Locking settings on the server

For deployments where certain data must never be stored regardless of how a workspace is configured,
list the settings to lock in `orbital.history.locked-fields`. A locked setting always uses the server's
value: workspaces can't override it, and neither can queries, even in a workspace with per-query
overrides turned on.

```yaml
orbital:
  history:
    persist-remote-call-responses: false
    locked-fields:
      - persistRemoteCallResponses
```

The names in `locked-fields` are the setting names from the table above, exactly as written. Locked
settings appear read-only on the Workspace settings page.

## API

The Workspace settings page uses these endpoints, and you can call them directly.

```
GET /api/{orgId}/{workspaceId}/configuration/query-persistence
PUT /api/{orgId}/{workspaceId}/configuration/query-persistence
```

`GET` returns everything needed to understand the workspace's current state: the overrides stored for the
workspace (`null` means inherited), the server defaults, the locked settings, and the values that are in effect.

```json
{
  "workspace": {
    "maxPayloadSizeInBytes": null,
    "persistRemoteCallResponses": false,
    "persistRemoteCallMetadata": null,
    "persistTraceEvents": null,
    "persistResults": null,
    "persistErrors": null,
    "allowQueryOverride": true
  },
  "serverDefaults": {
    "maxPayloadSizeInBytes": 2048,
    "persistRemoteCallResponses": true,
    "persistRemoteCallMetadata": true,
    "persistTraceEvents": true,
    "persistResults": true,
    "persistErrors": true,
    "allowQueryOverride": false
  },
  "lockedFields": ["persistTraceEvents"],
  "effective": {
    "maxPayloadSizeInBytes": 2048,
    "persistRemoteCallResponses": false,
    "persistRemoteCallMetadata": true,
    "persistTraceEvents": true,
    "persistResults": true,
    "persistErrors": true,
    "allowQueryOverride": true
  }
}
```

`PUT` takes just the `workspace` object - the overrides to store - and returns the same response as `GET`.
Send `null` for a setting to go back to inheriting it. A request that sets a locked setting, or a
`maxPayloadSizeInBytes` above the server's value, is rejected with a `400`.

`GET` requires the *Browse schema* privilege; `PUT` requires *Edit schema*.

## Current limitations

 * When Orbital is running with a separate analytics server (`orbital.history.mode: Remote`), the server defaults apply to every query. Workspace and query overrides are only honoured when history is written in-process, which is the default.
 * `maxPayloadSizeInBytes` is configurable and capped at every level, but stored response bodies are not yet truncated to it.
