Orbital

Managing data sources

Configuring connections

Creating connections to databases, message brokers, and more

Overview

The connections.conf file defines connections for systems that Orbital understands how to connect to.

A connection is required for most - but not - all data sources (eg., Databases, AWS services, etc). HTTP connections (OpenAPI / SOAP, etc) are generally self-descriptive enough not to require additional metadata.

Declaring in your taxi project

Connections are defined in a connections.conf, which lives inside a taxi project within your workspace.

Inside your taxi.conf file in your project, declare where you’re keeping your config for Orbital.

By convention, this lives at orbital/config/connections.conf, but this is configurable.

taxi.conf
name: com.myproject/demoversion: 0.1.0sourceRoot: src/ additionalSources: {     "@orbital/config" : "orbital/config/*.conf"    }

Your connections config file is a HOCON file, specifying how to connect to various resources.

For example:

connections.conf
jdbc {  orders {    connectionName = orders    connectionParameters {      database = ordersdb      host = localhost      username = martypitt      password = ${superSecretPassword}      port = 5432
    }    jdbcDriver = POSTGRES  }}

Using environment variables

Environment variables can be used in configuring connections. For example, you may wish to:

  • Provide secrets via environment variables
  • Parameterize topic names or table names between environments

In configuration files

It may not always be desirable to specify sensitive connection information directly in the config file - especially if these are being checked into source control.

Environment variables can be used anywhere in the config file, following the HOCON standards.

For example:

connections.conf
jdbc {   another-connection {      connectionName = another-connection      jdbcDriver = POSTGRES       connectionParameters {         // .. other params omitted for bevity ..         password = ${postgres_password} // Reads the environment variable "postgres_password"      }   }}

Correctly handling substitutions in Urls

Substitution of variable inside a Url using Hocon can be tricky.

In short, here’s how substitutions need to be defined:

query-server {   // The Url has specifal characters (:), so needs to be inside of quotes.   // However, variable substitution doesn't work inside of quotes,   // so the variable must be outside of quotes.   url="http://"${MY_VARIABLE}":9305"}

For more information, see this issue in the Hocon library

In annotations

You can use environment variables in any annotations, to allow you to parameterize things like topic names, or table names between environments.

Any annotation used in a Taxi schema may use environment variables. Environment variables are defined ${LikeThis} within the string.

For example:

@KafkaService( connectionName = "market-prices" )service MyKafkaService {      @KafkaOperation( topic = "${STOCK_PRICES_TOPIC}", offset = "earliest" )   stream streamStockPrices():Stream<StockPrice>}

Next steps

Next, you can edit your connections.conf file to add connections for your services.