Compute Templates

This page is a reference for compute group CloudFormation templates. Because the primary compute and query group templates are so similar, they are both covered here, with differences noted where relevant. This page is organized to follow the steps in CloudFormation's web interface for creating or updating a stack.

Stack operations

You can create, update, or delete a compute group.

Creating a Compute Group

To create a new compute group, choose "Create stack (with new resources)" in CloudFormation. Then in the "Prepare template" section, choose "Template is ready" and proceed to specify template.

Updating a Compute Group

You can update a template to do one or both of the following:

  • upgrade to a newer version of Datomic
  • modify template parameters

To update a compute group template, select an existing compute group stack in CloudFormation. Then chooose "Update Stack".

  • If you are upgrading Datomic, choose "Replace current template" and proceed to specify template below.
  • If you are only modifying template parameters, choose "Use current template" and proceed to parameters below.

Deleting a Compute Group

Deleting a compute group deletes all that group's resources: EC2 instances, Application Load Balancer, and API Gateways. It does not delete your Datomic system or any data, and you can recreate the compute group again later. To delete a Datomic compute group:

  1. Find your compute group stack in CloudFormation.
  2. Right-click on your query group stack and select "Delete Stack".

Specify template

Datomic CloudFormation templates are stored in S3. To specify a compute template:

  • Choose a template source of "Amazon S3 URL".(copy) or query group template (TODO: link to copy).
  • Set the Amazon S3 URL to the latest primary compute template: or

Specify stack details

The compute group stack details include the stack name and the stack parameters.

Stack Name

You will use the compute stack name to distinguish your compute group from other groups on the same system and from compute groups for other systems. A good choice is "$(System)-(Role)", where $(System) is your system name, and $(Role) is the role this group serves, e.g. "primary compute", an application name, or a particular task or load profile. For example, the "inventory" system might have "inventory-primary" for its primary compute group, and "inventory-analytics" for a query group serving analytics.

Parameters

Datomic configuration is managed via CloudFormation parameters, minimizing the surface area of AWS you must engage to run a system. As your needs grow, you may revisit and adjust the parameters that control the amount of compute resources, i.e. Instance Type and the Auto Scaling Configuration. Most other parameters are likely to be set once when starting a compute group and never revisited again. That said, except where explicitly noted, you can change any of these parameters by performing a rolling stack update.

Each parameter is summarized in the table below and then described in detail. Compute stack parameters are named by a Label in the CloudFormation UI, and under the Parameters key in CloudFormation JSON.

Parameter (JSON) Label (UI) Value
SystemName System Name system name
InstanceType Instance Type EC2 instance type
KeyName AWS EC2 Key Pair AWS keypair
ApplicationName Application Name CodeDeploy application name
EnvironmentMap Environment Map ion application configuration
PreloadDatabase Preload Database database name
MetricsLevel Metrics Level default/none/basic/detailed
DesiredCapacity Desired Capacity number or "Default"
MinSize Minimum Instances number or "Default"
MaxSize Maximum Instances number or "Default"
MinInstancesInService Minimum Number of Instances During Update number or "Default"
ClientAPI Client API yes/no
Ions Ions yes/no
NodePolicyARN IAM Managed Policy for Instances IAM policy ARN
OverrideSettings Override Settings edn map
Instance Type
Type of EC2 instance for compute nodes. See Instance Sizes.
AWS EC2 Key Pair
Key pair for ssh access to nodes.
Application Name
Ion CodeDeploy application name. This name cannot be changed later.
Environment Map
Ion environment map.
Preload Database
All compute group instances will load this database when they start, before accepting requests.
Metrics Level
Level of metrics detail. "default" sets metric level based on instance size, and should be suitable for most users.
Auto Scaling Configuration
Each of the Auto Scaling configuration settings allows you to set a specific number, or "Default" to choose a default based on your instance size.
Desired capacity
AWS Auto Scaling desired capacity or "Default"
Minimum group instances
AWS Auto Scaling minimum size or "Default".
Maximum group instances
AWS Auto Scaling maximum size or "Default".
Minimum number of instances during update
Number of instances that must be in service within the AWS while AWS CloudFormation updates instances, should be at least 1 less than maximum. A number or "Default".
Client API
Manage an AWS API gateway for Datomic Client API access. Most users should choose "yes". (If "no", you will have to configure VPC access for clients.)
Ions
Manage an AWS API Gateway for ion web applications. Most users should choose "yes". (If "no", HTTP direct ions will be inaccessible.)
Existing IAM managed policy for instances
(Optional) Name of a IAM managed policy to assign to instance nodes.
Support Configuration
Leave blank unless directed by Datomic Support.

Configure stack options

The "Configure stack options" screen has no Datomic-specific settings. Most users should accept all the defaults.

Review

The review step does two things. It shows you the choices you made in the previous steps, and it has a "Capabilities" section at the bottom where you must acknowledge the capabilities needed to run the stack.

When you create or update a Datomic compute stack, you can simply scroll down to the Capabilities section, acknowledge the capabilities being used, and then create the stack.

Wait for the stack

After you create or update a compute stack, CloudFormation will create and/or update AWS resources to match your specifications. This can take up to 25 minutes. You can refresh the CloudFormation dashboard to follow the stack's progress, which should end with CREATE_COMPLETE or UPDATE_COMPLETE.