Configuration

For initial setup of analytics support you need to do three things:

After you configure analytics support, you can connect analytics tools.

Later, you may revisit configuration for the following reasons:

  • Update the access gateway instance type to match your performance needs.
  • Update the catalog to change the databases exposed by analytics.
  • Update metaschemas to change the mapping from Datomic databases to SQL tables.

Enable

Analytics requires an access gateway instance with enough memory to run the analytics process. To enable analytics support:

You can choose a different access-gateway instance type at any time. Note that analytics support will be unavailable while the new instance launches (typically for a few minutes).

Updating Instance Type

You can change the access gateway instance type from the CloudFormation console.

  • Select your compute stack via the checkbox or radio button.
  • Click the "Actions" button and select "Update Stack"
  • On the "Select Template" screen, choose "Use current template"
  • On the "Specify Details" screen, change your access gateway instance type.
  • On the "Options" screen, leave all options unchanged.
  • On the "Review" screen, click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names" and click "Update".

Wait for the template to report UPDATE_COMPLETE. This can take a few minutes.

Configuration Directory

Analytics support requires you to configure a catalog and zero or more metaschema files locally, and then sync them to your running Datomic system.

The configuration directory must contain two subdirectories: datomic and catalog:

my-config-dir/
├── catalog
│   └── my-system.properties
└── datomic
    └── my-metaschema.edn

Populate the catalog and metaschema files as described below.

Catalog

The catalog subdirectory of your local configuration directory should contain a [your-catalog-name].properties file with the contents:

connector.name=datomic

#optional - limit exposed dbs by listing them here:
#datomic.databases=[thisdb thatdb]

Replace [your-catalog-name] with a name of your choice. (Since the catalog corresponds to a Datomic system, your Datomic system name is a often a good choice.) The catalog name will be used in any subsequent configurations or URIs that mention a "catalog" or "presto catalog" name.

The optional list of exposed databases (datomic.databases) enables you to select only specific databases from this system as accessible through the access gateway. If you leave this list commented, all databases in the system will be accessible via analytics. Limiting the set of exposed databases to those specifically desired for analytics is highly recommended.

After you change the catalog, you must sync the configuration files and then restart the access gateway.

Metaschema

Place zero or more metaschema edn files in the datomic subdirectory of your local configuration directory.

After you call sync, your metaschema changes will be available in a minute or less. This facilitates interactive development of your metaschema:

  • You do not need to restart the access gateway.
  • You do not need to restart any analytics tools.

If your analytics tool scans the database to discover the schema, you may need to re-run this scan manually to pick up schema changes.

Syncing Configuration

The sync command in the datomic-analytics CLI tool updates analytics configuration to match your local configuration directory. Call sync with your system name and configuration dir:

bash datomic-analytics sync <system> <path-to-config-dir>

The access gateway performs queries against your Datomic system, using the primary compute group by default. If you find that analytics queries are competing for resources with other uses of your system, you can create a dedicated query group for analytics, and configure analytics to use this dedicated query group with the -q option to sync

bash datomic-analytics sync -q <query-group-name> <system> <path-to-config-dir>

Restarting Access Gateway

To restart the access gateway, pass restart and your system name to the datomic-gateway CLI tool.

bash datomic-gateway restart <system>