Scaling

In the production topology, a Datomic Cloud system scales horizontally in three ways:

  • arbitrary and automatic scaling of durable storage
  • user-initiated scaling to handle additional databases
  • elastic autoscaling to handle additional read load

../images/scaling.png

Automatic Storage Scaling

Datomic stores database data in three places:

  • S3
  • EFS
  • DynamoDB

All three of these storages scale automatically, and do not require any configruation.

Explicit Database Scaling

Transactions are distributed across the primary compute nodes. If you are writing to a large number of different databases, you can increase the size of the primary compute cluster by configuring its AutoScaling Group.

Elastic Read Scaling

Note: Query Groups are coming soon.

Larger systems consist of a primary compute cluster for transactions, plus one or more query groups for various read loads. This provides two additional forms of horizontal scaling:

  1. You can explicitly start and stop query groups, and then route particular loads to them. This allows you to make after-the-fact decisions to adapt to new usage patterns. For example, you might have a query group dedicated to a range of data, or to a different priority of consumer, or to a single hot query. Note that your data is never partitioned, and no up-front decisions need to me made.
  2. Within a query group, you can set an autoscaling trigger so that the group grows and shrinks based on load.