Gateway-based imports
Elimity Insights v3.42.0 introduced a new feature for importing data into custom sources. More specifically, it aims to avoid the out-of-band configuration management and scheduling necessary for agent-based imports. The following table compares several aspects of both agent- and gateway-based imports:
Traffic direction
Agent pushes to Elimity Insights' API
Elimity Insights pulls from gateway's API
Authentication with Elimity Insights
Connector admin generates API credentials in Elimity Insights and places them in out-of-band configuration for the agent
Connector admin configures gateway URL in Elimity Insights, HTTPS by itself already provides the necessary authentication
Authentication with underlying data source
Connector admin places credentials in out-of-band configuration for the agent
Connector admin provides credentials as gateway configuration values in Elimity Insights
Authentication with gateway
Not applicable
Optional and only necessary if underlying data source requires additional protection; refer to the dedicated section below for additional information
Scheduling
Connector admin configures schedule out-of-band
Connector admin configures CRON schedule in Elimity Insights
Deployment
Job in Azure Container Apps, job in Google Cloud Run, ...
App in Azure Container Apps, service in Google Cloud Run, ...
Development
Python or NodeJS script which uses an official client library for pushing to Elimity Insights' API
HTTP server in Go or NodeJS which uses the official SDK for communication with Elimity Insights
In general we recommend all new customers to use gateway-based imports when possible.
Authenticating with gateways
In most circumstances authentication for gateway-based imports is already sufficiently covered by the combination of HTTPS and credentials for the underlying data source. However, sometimes additional protection of the gateway's HTTP(S) endpoint is necessary (e.g. if the gateway itself implicitly authenticates to the underlying source). For these use cases, Elimity Insights supports including OAuth2 access tokens in its requests to gateways. More specifically, our on-prem customers can enable this feature with the EnableGatewayOAuth2 configuration option. For SaaS customers we can provide a fully configured setup upon request. The diagram below depicts the authentication flow in more detail:
As you can see, in this case Elimity Insights includes access tokens from our own OAuth2 server at auth.elimity.com. More specifically, we'll make sure those tokens adhere to the JWT standard and have the following custom claims:
aud
"gateway"
base_url
Base URL of the Elimity Insights tenant that requested the access token, e.g. "https://example.elimity.com"
gateway_url
URL of the gateway for which Elimity Insights has requested the access token, e.g. "https://gateway.example.com"
iss
"https://auth.elimity.com/"
source_id
Unique identifier of the source for which to import into Elimity Insights, e.g. "42"
Last updated