For initial setup of analytics support you need to do three things:
- Enable analytics in CloudFormation.
- Create a local configuration directory and sync it to your Datomic system.
- Restart the access gateway to pick up the initial configuration.
After you configure analytics support, you can connect analytics tools.
Later, you may revisit configuration for the following reasons:
- Update the access gateway instance type to match your performance needs.
- Update the catalog to change the databases exposed by analytics.
- Update metaschemas to change the mapping from Datomic databases to SQL tables.
Analytics requires an access gateway instance with enough memory to run the analytics process. To enable analytics support:
- Ensure that your Datomic system has separate storage and compute stacks.
- Ensure that your compute group specifies an access-gateway instance type that is larger than `nano`.
You can choose a different access-gateway instance type at any time. Note that analytics support will be unavailable while the new instance launches (typically for a few minutes).
Updating Instance Type
You can change the access gateway instance type from the CloudFormation console.
- Select your compute stack via the checkbox or radio button.
- Click the "Actions" button and select "Update Stack"
- On the "Select Template" screen, choose "Use current template"
- On the "Specify Details" screen, change your access gateway instance type.
- On the "Options" screen, leave all options unchanged.
- On the "Review" screen, click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names" and click "Update".
Wait for the template to report UPDATE_COMPLETE. This can take a few minutes.
The configuration directory must contain two subdirectories:
my-config-dir/ ├── catalog │ └── my-system.properties └── datomic └── my-metaschema.edn
Populate the catalog and metaschema files as described below.
catalog subdirectory of your local configuration directory
should contain a
[your-catalog-name].properties file with the contents:
connector.name=datomic #optional - limit exposed dbs by listing them here: #datomic.databases=[thisdb thatdb]
[your-catalog-name] with a name of your choice. (Since the
catalog corresponds to a Datomic system, your Datomic system name is a
often a good choice.) The catalog name will be used in any subsequent
configurations or URIs that mention a "catalog" or "presto catalog"
The optional list of exposed databases (
you to select only specific databases from this system as accessible
through the access gateway. If you leave this list commented, all
databases in the system will be accessible via analytics. Limiting the
set of exposed databases to those specifically desired for analytics
is highly recommended.
Enabling JDBC Metadata
Many analytics tools provide the ability to automatically explore all of the tables in your system. Such exploration involves issuing one (or many!) JDBC metadata queries, forcing the load of every table in your system.
Because these automatic queries can be so expensive, they are disabled by
default. To enable JDBC metadata queries, you must explicitly enumerate the Datomic
databases you want to query with the
After you call sync, your metaschema changes will be available in a minute or less. This facilitates interactive development of your metaschema:
- You do not need to restart the access gateway.
- You do not need to restart any analytics tools.
If your analytics tool scans the database to discover the schema, you may need to re-run this scan manually to pick up schema changes.
bash datomic-analytics sync <system> <path-to-config-dir>
The access gateway performs queries against your Datomic system, using
the primary compute group by default. If you find that analytics
queries are competing for resources with other uses of your system,
you can create a dedicated query group for analytics, and
configure analytics to use this dedicated query group with the
bash datomic-analytics sync -q <query-group-name> <system> <path-to-config-dir>