0.30 is Out - SQL Server, new authentication methods, and JSON inside XML inside a... duck?

​

Orbital 0.30.0 - Full House Edition

We’re excited to announce the release of Orbital 0.30.0 - our “Full House” edition, because we got so distracted by Valentine’s Day last month that we didn’t manage to ship a release, making this one double-packed.

If you’re not familiar with Orbital, it’s a platform for automating integration between APIs, Databases, Message Brokers, Serverless Functions, and more. We have great stuff like API-spec driven data sources, self-repairing integration, automated lineage and more.

So, let’s take a look at what we’ve been working on for the past two months. It’s a full house jam packed with features and squashed bugs.

​

Highlights

​

MS SQL Server support

We’ve extended our family of supported databases to include MS SQL Server for comprehensive read and write operations.

So, it’s “DATABASES! DATABASES! DATABASES!”, sweaty armpits, and happy Microsoft shops.

Our docs explain all the details on how to connect to MSSQL, to get your Northwind blowing. (Get it? That’s a MSSQL sample database joke. If you know, you know.)

​

SAML authentication support

Someone paid us a lot of money to build SAML. We didn’t wanna do it. But we did, and now it’s awesome. Learn more about how to configure SAML authentication in Orbital.

​

mTLS Support

Mutual trust is a big theme in the Tanner household, and it’s something we can really get behind.

mTLS is now supported when calling HTTP(s) APIs. Read more about how to set up mTLS

​

Mixed messages

Orbital will now read an XML message with a JSON message in the middle, or JSON with XML in it, or a CSV file with a JSON blog in that, or really anything inside of another thing, which is inside yet another thing. It’s the integration version of a Turducken.

For example, here’s some XML inside a JSON message:

{
    "messageId" : "123",
    "xmlRecord" : "<person><name>Jimmy</name></person>"
}

Here’s the associated definition:

import com.orbitalhq.formats.Xml
model MyMessage {
    messageId : MessageId inherits String
    xmlRecord : Person
}

@Xml
model Person {
    name : PersonName inherits String
}

​

Using Jwt claims within a query

You can now pass the claims from a JWT token into Orbital so the claims can be used during query execution, and passed to upstream services.

Let’s look at a simple example where your JWT token has a key piece of information - the customers accountId. If you want to use these to connect with external REST services to run specific Orbital queries, the first thing to do is create a model that builds on Orbital’s basic JwtClaim model:

// Custom JwtClaim model with contactId and accountId
model CompanyXJwtClaim inherits JwtClaim {
  accountId: AccountId // a claim present with the key accountId
  // other claims omitted
}

Adding this model to your system makes sure that accountId is available for Orbital when it runs queries.

For example, a service may require the users accountId from the JWT be passed in the URL:

// A service needs to use the current users accountId in the URL: 
service AccountService {
   @HttpOperation(method = "GET",url = "/account/{accountId}")
   operation getByAccountId(@PathVariable("accountId") accountId: AccountId): Account
}

To make the users accountId available, we simply expose the JWT to the query:

query MyQuery ( jwt:CompanyXJwtClaim ) {
   given { jwt }
   find { Account }
}

That’s it! The presented JWT is read and the accountId is used to invoice the getByAccountId operation.

​

Errors on streaming messages

Previously, it was hard to see when things went wrong inside streaming queries, requiring tailing logs to debug issues.

Now, errors in streams are shown in a dedicated UI.

Errors in streaming queries are now shown off for the beautiful learning opportunities they are.

​

Usability

​

Filtering in the results table

Results returned from the server can be filtered client side in the query view:

Filtering tables in query results

​

Language features

​

Type checker

We’ve enabled Taxi’s type checker by default, to help surface errors in queries earlier.

For example, everyone in our office has made this mistake before:

find { Film[] } as {
   cast: (Actor[]) -> { // because this is projecting an array...
     name : ActorName
   } // <--- ...this should be an array
} // <-- so should this

Now, the compiler tells us what a mistake it was, so we can take a good long look at ourselves.

​

Scoped Variables

When projecting, you sometimes need to hoist multiple variables into the scope. This is to make them available at the top level of a projection, without declaring them directly in the output.

For example, in the below example, we want to operate on a single item from an array of values, as well as the parent object.

However, we need both the objects in scope:

find { Film } as (Film, first(Actor[])) -> {  // two values declared in scope
    title : Title // title comes from Film
    starring : ActorName // ActorName comes from Actor
}

Our query syntax now supports adding these additional variables into the query scope

​

Compiler warnings when something should be declared closed

Marking a model as closed in TaxiQL indicates that the query engine should not attempt to build an instance if you ask for it.

Generally, this is the expected behaviour for things that a server returns.

For example:

// I want the Film object as returned from the server, not constructed
model Film {
   title : Title inherits String
}

service Films {
   operation findAllFilms():Film[]
}

If writing a query like:

find { Film[] }

In the above example, you don’t want Orbital to try to construct a Film instance, you want one returned.

If you forget to mark Film as closed then the compiler will now warn you.

// That's better:
closed model Film {
   title : Title inherits String
}

​

Fixes

Here’s a big long list of things that we showed to people who asked why we hadn’t been to the pub this month.

To play us out, while you read through our bug list, here’s the Full House theme song.

• given clause with cast statement referencing parameter throws exception
• Server sends all responses as arrays
• Server sends arrays when response isn’t an array
• Response arrays of single item are incorrectly unwrapped
• Compiler not erroring when projecting array to non-array
• Can’t use params in call to table operation
• Live query view in history is broken
• Removing a project results in an orphaned view for a project that no longer exists
• Can’t onboard Kafka connections via UI
• Viewing Operations in Services view is borked
• Incorrect responsive layout in TypeViewer component
• Fix streaming results progress bar and record count
• Query History should not list active queries in the history panel, only in the “active” panel
• Copy as Curl breaks with single / double quote issues
• “undefined” entries showing in catalog
• Error not thrown when inner-projection mismatch on array properties
• Schema view shows all types in uppercase
• Duplicates listed in the intellisense
• Active queries in history is missing icons
• Docs missing from catalog page
• Tables in connection details need a minimum width
• Make checkboxes in app-model-attribute-tree-list readonly
• Disable project selector when adding connection
• Inlay type hints not shown in raw results
• Can’t authenticate to Confluent Cloud Kafka
• Format JSON responses in Response not working
• Ensure UI is in correct state when query is run
• Query History takes too long to load
• Projections with fields that are subtypes assign incorrectly
• Fields on anonymous types are not considered for population of graph queries
• Projection of a scoped top-level array fails
• Lineage for cached items can’t trace further back than the cache query
• Query Plan diagram is broken when cache is invoeld
• Services diagram renders shared attributes incorrectly
• StateStore is applied without the annotation
• Streaming queries are rewritten to join streams incorrectly
• Better UX when nothing is registered
• Starting a query with Apple+Enter / Ctrl+Enter shows incorrect time in ticker
• Http 204 responses are not persisted
• Language Server incorrectly reports some types as not defined
• connections.conf not resolving env variables from env.conf
• After saving a query with a streaming union type, can’t rerun another adhoc query with the same streaming union type
• SQL queries fail if the table name is different from the model name
• Profiler view doesn’t show all interactions