How To

Install the AWS CLI

Install the AWS Command Line Interface.. Make sure you have version 1.11.170 or later. If you are unfamiliar with managing Python environments, first try using the bundled AWS CLI installer. You can check your version using:

aws --version

NOTE: Running the datomic-socks-proxy script with an earlier version of the AWS CLI will result in a generic error message and failure of the script.

If you require multiple versions of the AWS CLI to be installed, then please investigate using Python's virtual environments, or the third-party virtualenv solution.

Manage AWS Access Keys for Datomic

The Datomic CLI tools and ion deployment tools require that you have a set of AWS access key with permissions to access Datomic. To create these keys, you must have an authorized IAM user. With this user you can then:

  1. Create access keys
  2. Add access keys to your environment

Datomic supports the use of named profiles as a credentials source. Additional information about IAM Users and access keys can be found in the AWS Security Credentials documentation .

Install Datomic CLI Tools

The Datomic CLI tools are distributed as a zip file containing several executable scripts. Download the latest version of the tools from the releases page and unzip the scripts to a location of your choosing.

The CLI tools zip includes:


The access gateway runs inside a Datomic system's private VPC. It acts as an entry point for Client API and analytics connections from outside the VPC.

While the gateway runs automatically based on your CloudFormation template, there are some situations where you may want to explicitly control it:

  • You can start or stop the gateway to control access or EC2 instance costs.
  • You can restart the gateway to recover from a process failure.
  • You can restart the gateway to pick up new analytics catalog settings.

You can control the access gateway with the datomic-gateway CLI tool. The script takes a command and a system name.

bash datomic-gateway <command> <system>

and supports the commands:

  • up : Start the access gateway instance.
  • down : Stop the access gateway instance.
  • restart : Restart the access gateway instance.

The script can also take several optional arguments:

-p <aws-profile> -- name of an AWS Credentials Profile to use
-r <aws-region>  -- AWS Region in which the Datomic system is running
--instance-wait  -- return from the script only once the change completes

If --instance-wait is set, the script will wait for confirmation that the command has completed. This can a few minutes. Otherwise the script will return immediately.

Monitor the Access Gateway

You can monitor the status of the access gateway in the AWS EC2 Management Console. The access gateway is named SystemName-bastion.

  • When the Instance State is running, the access gateway is available for use.
  • An Instance State of terminated indicates the access gateway has terminated. You will incur no future charges for that instance.


The datomic-access CLI tool uses ssh to allow connections from outside the Datomic VPC through the access gateway to a Datomic system.

datomic-access directions requires two arguments, a connection type and a system-name.

  • The client connection type connects to a Client API endpoint.
  • The analytics connection type connects to an Analytics endpoint.

The script can also take several optional arguments:

-p <aws-profile>         name of an AWS Credentials Profile to use
-r <aws-region>          AWS Region in which the Datomic system is running
--port <port>            Local port to use for the proxy, default 8182
--no-host-key-checking   disable host key checking

The --no-host-key-checking flag disables ssh host key checking, i.e. -o StrictHostKeyChecking=no. This is useful for automated scripts where there is no user to respond to interactive prompts.

If you do not specify --no-host-key-checking, you will sometimes need to repond yes to the following prompt, which will appear each time the access gateway is running on a new IP address:

The authenticity of host <access gateway-server-address> can't be established.
RSA key fingerprint is <an SSH key>.
Are you sure you want to continue connecting (yes/no)?

The datomic-access script continues to run once launched. If the script terminates (as it might for e.g. connection loss after laptop sleep), you will need to restart it.


That datomic-analytics CLI tool configures Datomic analytics support. Currently there is one option, sync, which syncs your local configuration directory to the access gateway. Call sync with your system-name and your local configuration directory:

bash datomic-analytics sync <system> <path-to-config-dir>

The datomic-anlaytics script also takes several optional arguments:

-p <aws-profile> -- name of an AWS Credentials Profile to use
-r <aws-region> -- AWS Region in which the Datomic system is running
-q <query-group> -- which query group to use (primary is the default)

Control Shell Scripts

Some Datomic CLI tools (e.g. the datomic-access script) continue to run in the foreground once you launch them. For interactive use, the easiest way to manage such tools is simply to kill them with Ctrl-C when they are no longer needed.

If you are building automation around scripts, you can or course use all the ordinary Unix facilities, e.g.

  • You can use pkill [script-name] to kill a script from another terminal.
  • You can use e.g. nohup [script-command] [script-args] \~ to launch a script in the background, directing its output to nohup.out.

Find Datomic System Name

Every Datomic system resides in a single AWS region, and has a unique system name within that region. The system name is the Stack Name used to launch the Datomic storage stack.

If you did not start Datomic yourself, you can find the available systems in a region by browsing the CloudFormation console.

You can list the names of all running Datomic nodes with the following AWS CLI command, replacing [region] with the region you want to query:

aws ec2 describe-instances --region [region] --filters "Name=tag-key,Values=datomic:tx-group" "Name=instance-state-name,Values=running" --query 'Reservations[*].Instances[*].[Tags[?Key==`datomic:system`].Value]' --output text

Find Compute Group Name

  • If you start a Datomic system with split stacks, then it has one or more compute groups. A compute group's name is the name of the CloudFormation stack you used to create the group.
  • If you start a system using the AWS Marketplace template, Datomic generates a unique name for your compute stack based on your system name, of the form $(SystemName)-Compute-$(GeneratedId).

You can browse all your CloudFormation stacks in the AWS CloudFormation console .

Find Datomic S3 Bucket

Every Datomic system has its own S3 bucket for configuration and data storage.

You can find this S3 bucket by browsing to the Outputs tab of your storage stack in the AWS Console and looking for the the S3DatomicArn output.

Upgrade Base Schema

Every Datomic database starts with a base schema, whose identifiers are prefixed with :db. When the Datomic team enhances the base schema, all databases created with new versions of Datomic get the enhancements automatically.

Existing databases do not automatically incorporate enhancements to the base schema. In order to upgrade existing databases to use new base schema, you must first upgrade all compute groups (primary and query) to the minimum version of Datomic required for the base schema features you want:

Feature Minimum Datomic Version Released
tuples 480-8770 June 27, 2019
attribute predicates 480-8770 June 27, 2019
entity specs 480-8770 June 27, 2019
db.type/symbol 480-8770 June 27, 2019

One you have upgraded all compute groups, you can upgrade an existing database to use the latest base schema by passing the :upgrade-schema action to administer-system. For example:

(d/administer-system client {:db-name "movies"
                             :action :upgrade-schema})

Upgrading a schema is an idempotent operation, so it will not harm your data to call it more than once. That said, you should treat administer-system as a potentially expensive operation. Call it only when you actually need it, not before every call to connect.

Once you have used a base schema feature in a particular database, do not run any compute group for that system on versions of Datomic older than the features used. (See table above.)

Update a CloudFormation Parameter

Datomic's CloudFormation parameter settings are carefully planned to adhere to AWS best practices and provide a robust and performant system. You should change CloudFormation parameters only:

  • when the Datomic documentation specifically instructs you to do so
  • when the Datomic support team specifically instructs you to do so

To update a CloudFormation parameter:

  • Select your stack via the checkbox or radio button.
  • Click the "Actions" button and select "Update Stack"
  • On the "Select Template" screen, choose "Use current template"
  • On the "Specify Details" screen, change your parameter(s).
  • On the "Options" screen, leave all options unchanged.
  • On the "Review" screen, click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names."

Convert from Solo to Production

To update your system from Solo to Production topology:

  • make sure you have raised your i3.large instance limit
  • click the name of your solo stack in

CloudFormation console. On the stack page, Click "Update Stack":


On the following page, select "Specify an Amazon S3 template URL:" and enter the CloudFormation template URL for the latest production template you wish to upgrade to (see Release page for the latest production template) and click "Next".

  • On the "Specify Details" screen, leave all options unchanged.
  • On the "Options" screen, leave all options unchanged.
  • On the "Review" screen, click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names."

Wait for the template to report UPDATE_COMPLETE. This can take five or more minutes. You can refresh the CloudFormation dashboard to see progress. In the EC2 Dashboard you'll note that two i3.large instances with your stack name are running and a t2.small instance has been terminated. Your production stack is now ready for use.