«

Access Gateway (Legacy)

The access gateway is deprecated and is no longer available in Datomic Cloud. This page only applies to Datomic 781-9041 and lower.

The access gateway provides two capabilities: client access from outside the VPC, and Datomic analytics. If you are planning a new system, you should not use the access gateway. Instead, you should use an API Gateway for client access, and bring your own analytics cluster.

The access gateway is an EC2 instance that is managed by the primary compute group, runs in the same VPC as your Datomic compute groups.

Client Access

Allow Inbound Traffic to the Access Gateway

This section only applies to Datomic 781-9041 and lower.

Datomic runs in a private VPC. To allow access from outside the Datomic VPC (e.g. for developers), you must add an inbound rule to the access gateway security group.

NOTE: Access gateway instances are secured by a keypair that is accessible to Datomic administrators. If you want to further restrict access by IP also, you can enter a specific IP address or range of addresses for Source in the instructions below.

  • Navigate to the Security Groups section of the AWS EC2 Management Console
  • Click on the access gateway security group for your Datomic Cloud system, named <system-name>-bastion
  • Click the "Inbound" tab in the Security Group details at the bottom of the console
  • Click "Edit Inbound Rules" to display the Edit inbound rules dialog box
  • Add an entry with the following parameters:
    • Type: SSH
    • Protocol: TCP
    • Port Range: 22
    • Source: Select "Anywhere" from the source dropdown
  • Accept the defaults for the other entries, and click "Save Rules"

Prerequisites

This section only applies to Datomic 781-9041 and lower.

This page assumes that an administrator has already:

Before you configure Client API connectivity to a Datomic database from outside the VPC, you need to:

Connect to the Access Gateway

To connect to the access gateway, run the datomic access command to create an SOCKS proxy connection, passing you system name:

datomic client access <system>

The script will continue to run once launched.

Test Access Gateway Connection

Run the following command to test your system's ability to reach Datomic through the SOCKS proxy, replacing [system], [region]. and [port] with your system-name, region and port.

curl -x socks5h://localhost:[port] http://entry.[system].[region].datomic.net:8182/

On success, this command will return text like:

{:s3-auth-path [bucket-name]}

Any other response indicates a failure to connect. Carefully review the prerequisites and steps on this page. If you feel that you have completed each step correctly, check the troubleshooting documentation.

If you are trying Datomic for the first time, a good next step is the Client API Tutorial.

Analytics

This section only applies to Datomic 781-9041 and lower.

For the initial setup of analytics support, you need to do three things:

After you configure analytics support, you can connect analytics tools.

Later, you may revisit the configuration for the following reasons:

  • Update the access gateway instance type to match your performance needs
  • Update the catalog to change the databases exposed by analytics
  • Update metaschemas to change the mapping from Datomic databases to SQL tables

Enable

Analytics requires an access gateway instance with enough memory to run the analytics process. To enable analytics support:

You can choose a different access gateway instance type at any time. Note that analytics support will be unavailable while the new instance launches (typically for a few minutes).

Updating Instance Type

You can change the access gateway instance type from the CloudFormation console.

  • Select your compute stack via the checkbox or radio button
  • Click the "Actions" button and select "Update Stack"
  • On the "Select Template" screen, choose "Use current template"
  • On the "Specify Details" screen, change your access gateway instance type
  • On the "Options" screen, leave all options unchanged
  • On the "Review" screen, click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names" and click "Update"

Wait for the template to report UPDATE_COMPLETE. This can take a few minutes.

Configuration Directory

Analytics support requires you to configure a catalog and zero or more metaschema files locally, and then sync them to your running Datomic system.

The configuration directory must contain two subdirectories, datomic and catalog:

my-config-dir/
├── catalog
│   └── my-system.properties
└── datomic
    └── my-metaschema.edn

Populate the catalog and metaschema files as described below.

Catalog

The catalog subdirectory of your local configuration directory should contain a [your-catalog-name].properties file with the contents:

connector.name=datomic

#optional - limit exposed dbs by listing them here:
#datomic.databases=[thisdb thatdb]

Replace [your-catalog-name] with a name of your choice. (Since the catalog corresponds to a Datomic system, your Datomic system name is often a good choice.) The catalog name will be used in any subsequent configurations or URIs that mention a "catalog" or "Presto catalog" name.

The optional list of exposed databases (datomic.databases) enables you to select only specific databases from this system as accessible through the access gateway. If you leave this list commented, all databases in the system will be accessible via analytics. Limiting the set of exposed databases to those specifically desired for analytics is highly recommended.

After you change the catalog, you must sync the configuration files and then restart the access gateway.

Enabling JDBC Metadata

Many analytics tools provide the ability to automatically explore all of the tables in your system. Such exploration involves issuing one (or many!) JDBC metadata queries, forcing the load of every table in your system.

Because these automatic queries can be so expensive, they are disabled by default. To enable JDBC metadata queries, you must explicitly enumerate the Datomic databases you want to query with the datomic.databases property described above.

Metaschema

Place zero or more metaschema EDN files in the datomic subdirectory of your local configuration directory.

After you call sync, your metaschema changes will be available in a minute or less. This facilitates the interactive development of your metaschema:

  • You do not need to restart the access gateway
  • You do not need to restart any analytics tools

If your analytics tool scans the database to discover the schema, you may need to re-run this scan manually to pick up schema changes.

Syncing Configuration

The analytics sync command in the datomic tools updates analytics configuration to match your local configuration directory. Call sync with your system name and configuration dir:

datomic analytics sync <system> <path-to-config-dir>

The access gateway performs queries against your Datomic system, using the primary compute group by default. If you find that analytics queries are competing for resources with other uses of your system, you can create a dedicated query group for analytics, and configure analytics to use this dedicated query group with the -q option to sync

datomic analytics sync <system> <path-to-config-dir> -q <query-group-name>

Restarting Access Gateway

To restart the access gateway, pass restart and your system name to the datomic gateway CLI tool.

datomic gateway restart <system>