# Overview

> An overview of how to connect data sources to Orbital

Source: https://orbitalhq.com/docs/workspace/overview

An Orbital Workspace is a collection of schemas, API specs and Taxi projects that describe
data sources (APIs, Databases, Message queues and Serverless functions) and provide a description
of the data and capabilities they provide.

These descriptions are used to generate integration on-the-fly, and power your data catalog.

Data sources are described using some form of API spec language (Protobuf, OpenAPI, Avro, Taxi), along with
additional metadata that defines how concepts are related semantically.

There's lots of different ways to tell Orbital about your data sources - we believe that Orbital should fit with
the way you work today.

## Components

Generally there's at least the following components:

 * A taxonomy project (built using Taxi) that defines the terms embedded in your API contracts.  This is most commonly stored in Git, or locally on your machine (when you're first getting started).
 * A series of API specs, enriched with metadata using terms from your Taxonomy project.

### Your core taxonomy
The core taxonomy is a collection of terms that describe your data attributes.  Think of them
as simple tags, which describe your data - such as `FirstName`, `LastName`, etc.

These tags provide
In practice, these tags are actually semantic types, defined using [Taxi](https://taxilang.org).

```taxi
type FirstName inherits String
type LastName inherits String
type CustomerId inherits Int
```

By using Taxi, you get the benefit of a full type system, rich build tooling to verify correctness, and an
open source ecosystem to build your own plugins.

A common pattern for building a core taxonomy involves the following steps:

 * Create it as a standalone project, independent of any specific API or service.
 * Store it in a Git repository.
 * Orbital [monitors the git repository](/docs/workspace/connecting-a-git-repo) and automatically deploys changes.

### Services and data Sources
Services and data sources are described using existing API specs (OpenAPI / Protobuf / Json Schema, etc), enriched
with semantic tags from [your core taxonomy](#your-core-taxonomy).


```yml ShoppingCart OpenAPI Spec
# An extract of the ShoppingCartApi OpenAPI spec:
components:
schemas:
ShoppingCart:
properties:
custId: # Field name
x-taxi-type:
> name: CustomerId # <-- Semantic type
type: string
```

```protobuf Example.proto
import "org/taxilang/dataType.proto";

message Customer {
optional string customer_name = 1 [(taxi.dataType)="CustomerName"];
>     optional int32 customer_id = 2 [(taxi.dataType)="CustomerId"];
}
```


For more information, follow our guides on using [OpenApi](/docs/describing-data-sources/open-api) or [Protobuf](/docs/describing-data-sources/protobuf) to describe your data sources.

#### Using Taxi
Another option is authoring your API specs directly in [Taxi](/docs/describing-data-sources/taxi):

```taxi
model Customer {
  id : CustomerId inherits Int
  firstName : CustomerFirstName inherits Sring
}

service CustomerService {
  @HttpOperation(method = "GET", url = "/shoppingCart/{CustomerId}")
  operation getShoppingCartByCustomerId(CustomerId):ShoppingCart
}
```

Alternatively, you can just generate schema definitions directly from code, or author your definitions in Taxi.

Once you choose how to describe your services, you simply need to [publish](#publishing-and-updating) your specs to Orbital

## Publishing and updating your specs
How and when you choose to publish / update these sources is up to you - teams tend to have different preferences around this.

You're also free to mix and match - to pick the publication method that best works for your tech and team working style.

The following publication methods are available:

 * Pushing API specs to Orbital (eg., when your microservices start)
 * Pushing updates to Orbital manually (eg., within a CI/CD pipeline)
 * Having Orbital poll sources for changes

### Pushing API specs directly from your apps
Applications can generate Taxi, direct from code, and publish to Orbital.

 * Suits microservices and teams who prefer generating their API specs from code
 * Currently only Java / Kotlin with Spring Boot is supported, but other SDKs are planned - [reach out](https://github.com/orbitalapi/orbital/discussions) to tell us about your usecase.

Learn more about this approach [here](/docs/connecting-data-sources/schema-publication-methods)

### Pulling API specs from Orbital

Orbital can be configured to fetch API specs (projects) from either Git repositories
or the file system where Orbital is running (intended for local development).

In addition to storing your [core taxonomy](#your-core-taxonomy) in Git, you can fetch other API specs (such as Protobuf or OpenApi) from
a git repository.

Orbital will poll your git repo, and automatically update as changes are merged into your target branch.

For more information, read about [pulling API specs from git](/docs/workspace/connecting-a-git-repo) or [reading API specs from local disk](/docs/workspace/connecting-a-disk-repo)

## Workspace.conf file
The `workspace.conf` file is a HOCON file that describes all the locations to read projects from.  (Data sources that
are [pushing their API specs](#pushing-api-specs-directly-from-your-apps) are not included.)

Projects can be pulled from multiple different formats and approaches.  The configuration for these
repositories is defined in a [HOCON](https://github.com/lightbend/config#examples-of-hocon) format file.

### Passing a workspace.conf file
By default, the configuration file is called `workspace.conf`.  However, the location of the file can be changed by setting `--vyne.workspace.config-file=/path/to/workspace.conf`
on the command line, or through any of the supported configuration overriding mechanisms.

### Reading workspace.conf from git
For production deployments, it's often preferable to read config directly from git. This is useful both for Infrastructure-as-code, as well as for
deploying to services where there's ephemeral storage (like AWS ECS)

You can configure Orbital to read a `workspace.conf` file from a git repository, by passing the following command line settings:

| Setting                     | Description                                                                                                                                                                                                                                                                                                                |
|-----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `vyne.workspace.git.url`    | The url of the Git repo to clone. If pulling from Github, Gitlab or Azure DevOps, use a personal access token in the url (eg: `https://username:personalAccessToken@github.com/username/yourRepoName.git` or `https://[username]:[personalAccessToken]@dev.azure.com/[yourOrgName]/[yourProjectName]/_git/[yourRepoName]`) |
| `vyne.workspace.git.branch` | The name of the branch to check out                                                                                                                                                                                                                                                                                        |
| `vyne.workspace.git.path`   | *Optional* The path within the repo to read the config file. Defaults to `workspace.conf`                                                                                                                                                                                                                                  | 

### Using a single-project workspace
For demos / test config, it's sometimes useful to start Orbital with a single project configured.

You can by-pass the workspace config, and point Orbital directly to a single local file-system project.

To do this, start Orbital with `--vyne.workspace.project-file=/path/to/taxi.conf`

### Configuration conventions
Durations are defined using [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601#Durations) formats.  For example:
 * 1 Day = `P1D`
 * 3 Seconds = `PT3S`

### Declaring projects in your workspace.conf
Projects are read either directly from disk on your Orbital server, or pulled from a git repository.

Workspaces can have a combination of local disk-based and git based projects (though for production it's
more common for projects to be read from a git repository)

Each of these approaches has their own dedicated documentation:
 * [Adding git projects to workspace.conf](/docs/workspace/connecting-a-git-repo#adding-a-git-repository-through-config)
 * [Adding file projects to workspace.conf](/docs/workspace/connecting-a-disk-repo#adding-a-file-repository-through-config)

### Kitchen sink configuration example

```json5
file {
   changeDetectionMethod=WATCH
   incrementVersionOnChange=false
   projects=[
      {
        isEditable=true
        path="/opt/var/orbital/schemas/taxi"
      }
   ]
   pollFrequency=PT5S
   recompilationFrequencyMillis=PT3S
}
git {
   checkoutRoot="/my/git/root"
   pollFrequency=PT30S
   repositories=[
      {
         branch=master
         name=my-git-project
         uri="https://github.com/something.git"
      }
   ]
}
```