Upgrading

Important:

The AWS Marketplace configuration of Datomic Cloud utilizes a master template in order to facilitate one-click deployment. On your first upgrade you will move to separate cloud formation stacks for storage and compute so you can manage their upgrades and lifecycles independently, per AWS best practice guidelines.

What Versions Am I Running?

After your first upgrade, a running Datomic system will include one or more CloudFormation templates, each running a specific DatomicCFTVersion.

To determine the DatomicCFTVersion a template is running:

  • Select the name of your system stack in the CloudFormation console
  • On the Outputs tab, find the value for the DatomicCFTVersion key.

Choosing to Upgrade

The releases page provides three resources that can help you decide when and how to upgrade:

  • The Critical Notices section contains critical notices for all users. You should always read this section.
  • The Release History table includes a summary column with a brief description of each release.
  • The Release Notes provide a comprehensive list of changes in each release.

How to Upgrade

Before any ugprade, read the critical notices for all releases between your current release(s) and the release(s) you are upgrading to. These notices may override the generic instructions below.

Unless otherwise directed by the critical notices, choose the first row in the table below whose criteria apply to you, and follow the Upgrade Path instructions.

CriteriaUpgrade Path
System has never been upgraded beforeFirst Upgrade
Upgrading compute template onlyCompute-Only Upgrade
Upgrading storage template onlyStorage-Only Upgrade

First Upgrade Ever

Your first upgrade will bring the entire system down and back up. After the first upgrade, other upgrades will typically be rolling upgrades with no downtime.

This upgrade process converts your Master Template Stack to the setup described in the Production Setup documentation.

Remove original stack

This process removes the Master stack, but will reuse the existing storage in a later step to maintain your storage assets.

  1. Select the root stack for your system in the CloudFormation console. The root stack will have a Stack Name that is the same as your system name.
  2. Choose "Delete Stack" from the "Actions" menu. Confirm this in the Delete Stack popup, then wait for the stack deletion to complete. This can take 10 minutes or more.

Create and Upgrade Storage Stack

Creating a new storage stack that utilizes existing storage from the original master template.

  1. Choose "Create Stack", and then paste in the template URL for the Storage Stack you want from the Releases Page.
  2. Click the Next button, and fill in the template's parameter values as specified below.
    StackName
    Enter your system name, i.e. the name of the stack you just deleted.
    Reuse existing storage
    Set this to "True".

    This CloudFormation Template creates a VPC in which to run Datomic Cloud. Configure the settings for the VPC in the VPC Configuration section. See the AWS VPC Guide for details on specifying the CIDR blocks. Unless you know you need to change the CIDR block settings, you should accept the default configuration.

    VPC CIDR block
    The CIDR block to assign to the VPC. Accept the defaults.
    First, Second, and Third CIDR Blocks.
    Datomic will configure three subnets in the VPC. These three CIDR Blocks must be subsets of the VPC CIDR Block, and they must not overlap with each other. Accept the defaults.
  3. Click the Next button to move to the "Options" screen. Leave all settings unchanged.
  4. Click the Next button to move to the "Review" screen. Under "Capabilities", click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names."
  5. Click "Create" to launch the Storage stack.
  6. Wait for the template to report CREATE_COMPLETE. This can take up to 25 minutes. You can refresh the CloudFormation dashboard to see progress.

Create and Upgrade Compute Stack

  1. Choose "Create Stack" a second time
  2. Go to Releases and click click the file icon for the compute release you wish to use. Selection between Solo and Production depends on factors explained in the Architecture document
  3. Click the Next button, and fill in the template's parameter values as specified below.
    StackName
    Use the name "$(System)-compute", where System is your system name.
    SystemName
    Enter your system name.
    Start bastion?
    Set this to "Yes" so that the template will start a bastion for developer internet access.
    AWS EC2 Key Pair
    The key pair to assign to compute nodes. Select a key pair for ssh access to nodes and the bastion host.
    Application Name
    If you supply the name of an existing application, this compute group will be a deployment group of that application. Otherwise, a new application will be created with the provided name.
    Environment Map
    Ion environment map.
    Preload Database
    All compute group instances will load this database when they start.
    Existing IAM managed policy for node
    Optional. Leave this blank.
  4. Click the Next button to move to the "Options" screen. Leave all settings unchanged.
  5. Click the Next button to move to the "Review" screen. Under "Capabilities", click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names."
  6. Click "Create" to launch the Compute stack.
  7. Verify that your upgraded system is available using the dashboard.
  8. (Optional) To allow developers to access the bastion from outside the VPC, you must follow the steps to Allow Inbound Bastion Traffic. You must repeat these steps if you previously did them, as deleting the compute stack removes that access.

Parameter Upgrade

You can upgrade a CloudFormation stack to change template parameters, while leaving the template itself unchanged. To update a stack parameter:

  • Select your stack via the checkbox or radio button.
  • Click the "Actions" button and select "Update Stack"
  • On the "Select Template" screen, choose "Use current template"
  • On the "Specify Details" screen, change your parameter(s).
  • On the "Options" screen, leave all options unchanged.
  • On the "Review" screen, click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names."

Compute-Only Upgrade

To update a Compute Stack:

  • Open the CloudFormation console
  • Select your stack via the checkbox or radio button.
  • Click the "Actions" button and select "Update Stack"
  • Select "Specify an Amazon S3 template URL:" and enter the CloudFormation template URL for the version that you wish to upgrade to (see Release page for all versions) then click "Next".
  • On the "Specify Details" screen, leave all options unchanged.
  • On the "Options" screen, leave all options unchanged.
  • On the "Review" screen, click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names."

Storage-Only Upgrade

To update your Storage Resources:

  • Open the CloudFormation console
  • Select your stack via the checkbox or radio button.
  • Click the "Actions" button and select "Update Stack"
  • Select "Specify an Amazon S3 template URL:" and enter the CloudFormation template URL for the version you wish to upgrade to (see Release page for all versions) then click "Next".
  • On the "Specify Details" screen, set the "Reuse Existing Storage" option to true: ../images/reuse-existing-storage-true.png
  • On the "Options" screen, leave all options unchanged.
  • On the "Review" screen, click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names" and click "Update".

Upgrading from Solo to Production

You must raise your i3.large instance limit before launching the Production Topology.

The Production Topology is designed for production use. It provides high availability, horizontally scalable processing, load balanching, and auto scaling. To update your system stack from solo to production, click the name of your solo stack in CloudFormation console. On the stack page, Click "Update Stack":

../images/updatestack.png

On the following page, select "Specify an Amazon S3 template URL:" and enter the CloudFormation template URL for the latest production template you wish to upgrade to (see Release page for the latest production template) and click "Next".

  • On the "Specify Details" screen, leave all options unchanged.
  • On the "Options" screen, leave all options unchanged.
  • On the "Review" screen, click the checkbox stating "I acknowledge that AWS CloudFormation might create IAM resources with custom names."

Wait for the template to report UPDATE_COMPLETE. This can take five or more minutes. You can refresh the CloudFormation dashboard to see progress. In the EC2 Dashboard you'll note that two i3.large instances with your stack name are running and a t2.small instance has been terminated. Your production stack is now ready for use.

Why Multiple Stacks?

A Datomic System consists of Storage Resources plus one or more Compute Stacks. These resources have distinct lifecycles and are upgraded separately (in particular Storage upgrades are less common than Compute upgrades) and so each has its own CloudFormation stack per AWS best practice guidelines.

The Marketplace install wraps a Storage template and Compute template in a single master template. This makes the initial setup slightly easier, but is less desirable for ongoing operations. Use the first upgrade instructions convert from the marketplace master stack to separate stacks.

The Master Template setup also obfuscates various errors, making it more difficult to debug issues.

Future Datomic Systems should be created as described in Production Setup.