# Validating data

> Declare validation rules against your types, and control what happens when data breaks them

Source: https://orbitalhq.com/docs/data-validation/data-validation

Orbital can validate the data flowing through your queries — both the arguments callers pass in, and the
data your services return — against rules declared in your taxonomy.

Because rules are declared against **types**, you define them once, and they're enforced everywhere that type
appears, no matter which service the data came from.

There are three things to know:

 * **Declare** rules using annotations such as `@NotEmpty` or `@MaxLength`
 * **Activate** them by putting a `@Valid` scope over the data you want checked
 * **Decide** what happens when data fails, using `@OnValidationFailure`

The validation annotations are built into Orbital, in the `com.orbitalhq.validation` namespace. You don't
need to add anything to your project to use them.

## A quick example

Here's a type with a rule, and a model that activates it:

```taxi
import com.orbitalhq.validation.NotEmpty
import com.orbitalhq.validation.Valid

@NotEmpty
type Name inherits String

@Valid
model Person {
   name : Name
}
```

If a service returns this `Person`:

```json
{ "name": "" }
```

...then Orbital produces a single violation — rule `com.orbitalhq.validation.NotEmpty`, at path `name`, with
severity `Error` — and by default rejects the query with an HTTP 400.

Remove the `@Valid`, and the same data flows through untouched. Rules do nothing until something activates them.

**Note: Imports in these examples**
Everything validation-related lives in `com.orbitalhq.validation`. The examples below assume
  you've imported the annotations they use (as above), but you can always write the fully-qualified name
  inline instead — `@com.orbitalhq.validation.NotEmpty` — which is handy in a one-off query. The
  same applies to the `Severity` and `ValidationAction` enums.

## Declaring rules

Rules are annotations. You can put them on a type, on a field, or on a query parameter.

**On a type** — every field of that type inherits the rule:

```taxi
@NotEmpty
type Name inherits String
```

**On a field** — the rule applies to that field only:

```taxi
@Valid
model Person {
   @NotEmpty name : Name
}
```

**On a query parameter** — see [validating query arguments](#query-arguments) below.

### Rules are inherited

A rule declared on a type applies to every type that inherits from it. `FirstName` picks up the `@NotEmpty`
declared on `Name`:

```taxi
@NotEmpty
type Name inherits String

type FirstName inherits Name

@Valid
model Person {
   firstName : FirstName
}
```

An empty `firstName` reports a `NotEmpty` violation at path `firstName`.

### Multiple rules on the same field

Rules accumulate — Orbital evaluates all of them and reports everything that failed, rather than stopping at
the first failure:

```taxi
@Valid
model Doc {
   @MinLength(value = 5)
   @MaxLength(value = 3)
   code : String
}
```

A `code` of `abcd` reports two violations, one for each rule.

### Customising the message and severity

Every rule accepts a `message` and a `severity`, whatever else it takes:

```taxi
@Valid
model Doc {
   @MaxLength(value = 3, severity = Severity.Warning, message = "too long")
   code : String
}
```

Severity is one of `Info`, `Warning` or `Error`, and defaults to `Error`. Severity matters because it
determines whether a violation is serious enough to actually fail the query — see
[thresholds](#choosing-a-threshold).

See the [validation rule reference](/docs/data-validation/validation-rules) for the full set of rules and
their parameters.

## Activating validation with @Valid

Declared rules are inert until a `@Valid` scope covers them. This is deliberate: it lets you describe your
data's constraints across your whole taxonomy, and then choose exactly where those constraints are enforced.

`@Valid` does two things for everything inside its scope:

 * runs every declared rule, and
 * enforces the type system's own nullability — a **non-nullable field holding null is an `Error` violation**

### The same schema, with and without

Without `@Valid`, nothing is checked:

```taxi
@NotEmpty
type Name inherits String

// No @Valid - rules are declared, but never run
model Person {
   name : Name
}
```

A `Person` of `{ "name": "" }` produces no violations. Neither does `{ "name": null }`, even though `name`
is non-nullable.

With `@Valid`, both are reported:

```taxi
@NotEmpty
type Name inherits String

@Valid
model Person {
   name : Name
   nickname : Name?
}
```

| Data | Violations |
|------|------------|
| `{ "name": "", "nickname": null }` | `NotEmpty` at `name` |
| `{ "name": null, "nickname": null }` | `NotNull` at `name` |
| `{ "name": "Jimmy", "nickname": null }` | none — `nickname` is nullable |

**Note: Nullability comes for free**
You never declare a "required" rule. A field is required if your taxonomy says it's non-nullable, and
  `@Valid` is what turns that declaration into an enforced constraint.

### @Valid recurses

A `@Valid` scope covers the whole subtree beneath it — nested models, collections, and collections of models,
to any depth:

```taxi
@NotEmpty
type Name inherits String

model Person { name : Name }
model Actor { actor : Person }
model Cast { actors : Actor[] }

@Valid
model Film { cast : Cast }
```

`Film` is the only model marked `@Valid`, but the `NotEmpty` on `Name` is enforced all the way down.

Recursive models are validated at every level too. Here, all four names are checked:

```taxi
@MaxLength(value = 3)
type ConstrainedName inherits String

@Valid
model Node {
   name : ConstrainedName
   child : Node?
}
```

### Scoping @Valid to part of a model

Put `@Valid` on a field, and only that field's subtree is validated:

```taxi
@NotEmpty
type Name inherits String

model Inner { name : Name }

model Outer {
   @Valid validated : Inner
   unvalidated : Inner
}
```

Given `{ "validated": { "name": "" }, "unvalidated": { "name": "" } }`, only `validated.name` is reported.

The reverse works too — a `@Valid` model nested inside a model that isn't `@Valid` still activates its own
subtree, and `@Valid` is inherited when one model inherits another. Marking the same thing `@Valid` twice
(on a field, and on that field's type) doesn't double up the violations.

### The one exception: query parameters

A rule annotation placed **directly on a query parameter** activates itself. You don't need `@Valid`:

```taxi
query Greet(
   @com.orbitalhq.validation.NotEmpty name : String
) {
   find {
      message : String = "Hello " + name
   }
}
```

If the parameter's type is a model, you can use `@Valid` on the parameter (or on the model) to cascade
validation through the whole object graph — see [query arguments](#query-arguments).

## Violation paths

Every violation carries the path to the value that failed, so callers can pinpoint exactly what to fix.
Paths use dotted attribute names, with `[n]` for collection indices.

Using the `Film` schema above:

```json
{
  "cast": {
    "actors": [
      { "actor": { "name": "Ridley" } },
      { "actor": { "name": "" } }
    ]
  }
}
```

produces one violation, at path:

```
cast.actors[1].actor.name
```

Paths nest through as many levels of collection as you have. A model like this:

```taxi
@NotEmpty
type Tag inherits String

model Inner { tags : Tag[] }
model Wrapper { inners : Inner[] }

@Valid
model Root { wrapper : Wrapper }
```

reports an empty tag at `wrapper.inners[1].tags[1]`.

A few specifics worth knowing:

 * **Rules on a collection itself** (such as `@MinItems`) report at the collection's own path — `tags`, not `tags[0]`
 * **A rule on a top-level scalar parameter** reports the empty path, `""`, since there's no attribute to name
 * **A collection validated as a whole** (rather than element-by-element) indexes from the root: `[2].name`

**Note: Operations that return a collection**
When an operation's return type is a collection (say `operation getItems():LineItem[]`), Orbital
  validates each element as it arrives, so paths are relative to the element — `name`, not
  `[1].name`. A collection that's a _field_ of a returned model (like
  `items : LineItem[]`) is indexed as you'd expect: `items[1].name`.

## Controlling what happens on failure

Producing violations and acting on them are separate steps. What Orbital does with a set of violations is
controlled per-query, with `@OnValidationFailure`:

```taxi
import com.orbitalhq.validation.OnValidationFailure
import com.orbitalhq.validation.ValidationAction
import com.orbitalhq.validation.Severity

@OnValidationFailure(action = ValidationAction.Warn, threshold = Severity.Error)
query GetOrder {
   find { Order }
}
```

| Parameter | Values | Description |
|-----------|--------|-------------|
| `action` | `Reject`, `Warn`, `Drop` | What to do when a violation meets the threshold |
| `threshold` | `Info`, `Warning`, `Error` | The minimum severity that triggers the action |

### The actions

**`Reject`** fails the request with an HTTP 400 and a structured report of the violations.

**`Warn`** lets the data through unchanged. The violations are still reported — they're logged, and passed to
any violation sinks you've configured — but the caller gets their results.

**`Drop`** skips the offending record and keeps going. This is for streams, where failing the whole stream
because of one bad message is usually worse than skipping it. Dropping is never silent — every dropped record
is reported and logged, exactly as it is under `Warn`.

### Defaults

If a query has no `@OnValidationFailure`, Orbital picks a sensible default based on the kind of query:

| Query | Default |
|-------|---------|
| Request/response (`find`) | `Reject` at `Error` |
| Streaming (`stream`) | `Drop` at `Error` |

Query arguments are the exception: they always `Reject`. Dropping a record makes no sense when the record is
the caller's own input, so a `Drop` on the arguments boundary is treated as `Reject`.

### Choosing a threshold

`threshold` is the minimum severity that triggers the action. Anything below it is reported but never fatal.

With the default `threshold = Error`:

 * an `Error` violation rejects the query
 * a `Warning` violation is logged, and the query proceeds
 * an `Info` violation is logged, and the query proceeds

Lower the threshold to `Warning` and warnings become fatal too. **All violations are reported regardless of
threshold** — the threshold only decides which ones are severe enough to act on.

This pairs well with per-rule severity overrides: mark the rules you're not yet confident about as
`severity = Severity.Warning`, watch them in your logs, then promote them to `Error`
once you trust them.

### The rejection response

A rejected query returns HTTP 400 with a `ValidationFailureReport`:

```json
{
  "message": "...",
  "violations": [
    {
      "path": "items[1].name",
      "rule": "com.orbitalhq.validation.MaxLength",
      "severity": "Error",
      "message": "Value 'toolong' is longer than the maximum length of 3 (was 7)"
    }
  ]
}
```

Here's a complete worked example. This query is published as an HTTP endpoint, and returns an `Order` whose
line items are validated:

```taxi
import taxi.http.HttpOperation
import com.orbitalhq.validation.Valid
import com.orbitalhq.validation.MaxLength

@Valid
model LineItem {
   sku : Sku inherits String
   @MaxLength(value = 3) name : ItemName inherits String
}

model Order {
   orderId : OrderId inherits String
   items : LineItem[]
}

service OrderService {
   operation getOrder() : Order
}

@HttpOperation(url = "/api/q/order", method = "GET")
query GetOrder {
   find { Order }
}
```

`Order` isn't itself `@Valid` — its `LineItem` elements are, so validation cascades into the `items`
collection and the violation path carries the index.

When `OrderService` returns:

```json
{
  "orderId": "o1",
  "items": [
    { "sku": "a", "name": "ok" },
    { "sku": "b", "name": "toolong" }
  ]
}
```

...a `GET /api/q/order` responds with `400 Bad Request`:

```json
{
  "message": "Validation failed for the result of operation getOrder: 1 violation(s) at items[1].name",
  "violations": [
    {
      "path": "items[1].name",
      "rule": "com.orbitalhq.validation.MaxLength",
      "severity": "Error",
      "message": "Value 'toolong' is longer than the maximum length of 3 (was 7)"
    }
  ]
}
```

## Where validation runs

Orbital validates at two points in a query.

### Query arguments

Arguments are validated **before any operation is invoked** — a query rejected on its arguments never calls
your upstream services.

A rule on the parameter itself self-activates:

```taxi
import taxi.http.HttpOperation
import taxi.http.QueryVariable

@HttpOperation(url = "/api/q/greet", method = "GET")
query Greet(
   @com.orbitalhq.validation.NotEmpty
   @QueryVariable("name") name : String
) {
   find {
      message : String = "Hello " + name
   }
}
```

`GET /api/q/greet?name=Jimmy` returns the greeting. `GET /api/q/greet?name=` returns `400`:

```json
{
  "message": "Validation failed for query 'Greet' on parameter(s): name",
  "violations": [
    {
      "path": "",
      "rule": "com.orbitalhq.validation.NotEmpty",
      "severity": "Error",
      "message": "Value must not be empty"
    }
  ]
}
```

Add `@OnValidationFailure(action = ValidationAction.Warn)` to that query, and the same request succeeds with
`"message": "Hello "` — the violation is logged instead.

When a parameter's type is a model, `@Valid` cascades through the whole object graph:

```taxi
@com.orbitalhq.validation.Valid
model Person {
   personId : PersonId inherits String
   contact : ContactDetails
}

model ContactDetails {
   @com.orbitalhq.validation.NotEmpty email : EmailAddress inherits String
}

query FindPersonId( person : Person ) {
   find { PersonId }
}
```

Passing a `person` with an empty `contact.email` rejects the query, reporting the violation at path
`contact.email`, before any service is called.

### Data returned by services

Every record returned by a service during query execution is validated as it's received.

For a request/response operation, the whole result is validated and — by default — a violation rejects the
query. For a streaming operation, each message is validated as it arrives, and by default violating messages
are dropped while valid ones flow on:

```taxi
import com.orbitalhq.validation.Valid
import com.orbitalhq.validation.NotEmpty

@Valid
model Notification {
   @NotEmpty message : NotificationMessage inherits String
}

service Notifications {
   operation watchNotifications() : Stream<Notification>
}
```

Given a stream of `"hello"`, `""` and `"world"`, a `stream { Notification }` query delivers `hello` and
`world`, and logs the dropped record.

### Validation and data policies

Validation runs on the data **as it was received from the upstream service**, before any
[data policies](/docs/data-policies/data-policies) mask, obfuscate or null out values.

That ordering is deliberate:

 * a value that a policy has masked must not then fail a rule that the raw value satisfied, and
 * a violation must describe the data as it actually arrived, not a redacted view of it

The practical consequence: write your rules against the real shape of your upstream data, not against what a
given caller is permitted to see.

## When a rule cannot be evaluated

A misconfigured rule degrades — it never brings down a request.

If a `@Pattern` regex won't compile, that rule is disabled at that declaration site. Instead of failing, it
reports `Info`-severity advice telling you the rule didn't run:

```taxi
@Valid
model Doc {
   @Pattern(regex = "[A-Z")
   @MaxLength(value = 3)
   code : String
}
```

Validating `{ "code": "abcd" }` produces two violations:

| Path | Rule | Severity | Message |
|------|------|----------|---------|
| `code` | `com.orbitalhq.validation.Pattern` | `Info` | Validation rule Pattern was not evaluated: the configured regex `[A-Z` is not a valid regular expression |
| `code` | `com.orbitalhq.validation.MaxLength` | `Error` | Value 'abcd' is longer than the maximum length of 3 (was 4) |

Note what this buys you:

 * **The broken rule is visible, not silent.** A rule that quietly passed everything would be worse than one that fails.
 * **Sibling rules keep working.** `@MaxLength` on the same field is unaffected.
 * **`Info` doesn't reject.** With the default `Error` threshold, the advice is reported but never causes a `400` on its own.

The same applies to a rule annotation that Orbital has no implementation for — it's skipped with a warning
rather than failing the query.

**Warning: Offending values are in responses, never in logs**
Rule messages quote the value that failed (`Value 'toolong' is longer than...`), so they're
  rendered into the HTTP failure report that the caller receives — but they are deliberately kept out of
  Orbital's server logs. Log lines carry only the path, rule name and severity, so validating sensitive
  data doesn't write it to disk.

## Coming soon

Support for writing your own validation rules in Kotlin is planned for a future release.