Datomic Cloud Change Log

2021/07/13 884-9095 - Compute Update

  • Enhancement: Many new EC2 instance sizes.
  • Enhancement: Lower pricing for all instance sizes.
  • Enhancement: API Gateways for client and application internet access.
  • Enhancement: Run analytics anywhere.
  • Upgrade: Compute nodes have been upgraded from Java 8 to Java 11.
  • Replaced: The client access gateway is no longer available; you can connect directly to the new client API gateway.
  • Replaced: The analytics gateway is no longer available; you can run your own analytics node or cluster.
  • Replaced: The solo tier is no longer available; the production compute stack now scales all the way down to t3.small.
  • Replaced: The socks proxy is no longer available; clients can connect directly to the client API Gateway.
  • Replaced: Each compute group now uses an ALB instead of an NLB.

This upgrade will cause up to one minute of downtime for each compute group. Make sure to perform this upgrade at a time that minimizes the impact of these disruptions.

In addition, make sure to read the upgrade instructions below for each of the following scenarios that apply to your system today: solo systems, production systems, developer connections, application connections, analytics connections, and web application (ion) connections.

884-9095 for Solo Users

Datomic no longer has a Solo compute stack. If you were using Solo you can upgrade to Production at no additional cost by peforming the following steps:

  • Your web applications no longer need to go through a web lambda proxy. They can (and must) instead use HTTP direct. HTTP direct is higher performance and simplifies your code, which no longer needs to call ionize.
  • Perform a storage upgrade.
  • Perform a compute upgrade, selecting the t3.small instance size (the default).

884-9095 for Production Users

If you are running Production, you now have a set of t3 instance sizes to choose from. Many users will be able to provision smaller, less expensive instances:

  • Unless your compute group needs valcache, you should switch from your current i3 instance to the matching-sized t3 instance.
  • If you believe you were already overprovisioned with i3.large, you should consider a smaller t3 instance size.

884-9095 for Developer Connections

884-9095 significantly simplifies developer connections. If you connect to Datomic with a socks proxy to the access gateway, remove all of those steps from your workflow. Instead:

  • When you upgrade a compute group, enable the client API gateway.
  • Configure your clients to connect directly to the client API gateway endpoint.

884-9095 for Application Connections

884-9095 significantly simplies application connections. If your application connections require managing security groups, VPC endpoints, or VPC peering, remove all of these steps from your workflow. Instead:

  • When upgrade a compute group, enable the client API gateway.
  • Configure your application clients to connect directly to the client API gateway endpoint.

884-9095 for Ion Applications with API Gateway

If you manually created an API Gateway for your ion application on a previous release of Datomic, that gateway will no longer work. You should note any custom configuration for refererence and then delete the gateway. Then, let Datomic create the API Gateway for you (preferred), or follow the updated instructions for creating your own API Gateway.

884-9095 for Ion Applications using Web Lambda Proxies

If you were using Web Lambda Proxies, then you should change to HTTP Direct or lambdas.

  • Move the function to the :http-direct or :lambdas key in your ion-config.edn
  • The ionize function should not be used.
  • Expect the appropriate input data for the type of entry point (HTTP Direct or Lambda)

884-9095 for Analytics Users

If you were using Datomic analytics via the access gateway, you will now need to start your own analytics cluster. Configure analytics on any machine or cluster of machines (locally or in EC2), and connect to the client API gateway endpoint.

2021/07/13 884-9095 - Storage Update

  • Enhancement: The storage template now sets DDB provisioning to fit within the AWS free tier if your usage is low enough.

2021/07/13 - 0.10.89 - CLI Tools

  • Add describe-groups system command

2021/07/13 0.8.113 - Client-Cloud Update

  • Add support for hmac through API Gateway

2021/07/13 0.9.290 - ion-dev

  • Update dep list for Datomic Cloud 884-9095
  • Enhancement: Improved integration between ions and AWS Application Load Balancers for the cloud version release.

2021/03/02 - 781-9041 - Compute Update

Storage Solo Compute Production Compute Query Groups
  • Upgrade: AWS Lambda Runtimes moved to NodeJs 12.x
  • Enhancement: New AWS Region - ca-central-1
  • Upgrade Jetty server to version 9.4.36.v20210114

2021/02/23 0.9.282 - ion-dev && 0.9.50 - ion

Ion Ion dev
  • ion-dev
    • Improvement: Limit how long to wait for a cluster node to gracefully shutdown.
    • Improvement: Increase performance of Ion request handler
    • Fix: Suppress noisy AsyncContext Alert caused by Ion timeout
  • ion
    • Improvement: get-params now allows arbitrary number of parameters

2021/02/23 - 772-9034 - Compute Update

Storage Solo Compute Production Compute Query Groups

2021/01/20 - 0.8.105 - Client-Cloud Update

client-cloud
  • Improvment: Make client dynaload thread safe

2020/11/30 732-8992 - Compute Update

Storage Solo Compute Production Compute Query Groups
  • New: Change the scale of a BigDecimal attribute in a transaction.
  • Upgrade to presto 346, fixing a bug that prevents analytics gateway from launching.
  • Fix: Reenable query counts metric in dashboard.
  • Fix: Query correctly treats range functions as functions (not as predicates).
  • New region: eu-north-1

2020/09/23 715-8973 - Compute Update

Storage Solo Compute Production Compute Query Groups
  • New region: ap-south-1
  • Fix: Improve storage garbage collection.
  • Fix: Prevent situation where tx-range sometimes returned one extra transaction prior to a specified instant.

2020/08/07 704-8957 - Compute Update

Storage Solo Compute Production Compute Query Groups
  • New feature: Cancel
  • New region: `eu-west-2`
  • Fix: Prevent race condition in valcache that could lead to a failed read
  • Improvement: Better entity predicate error messages
  • Improvement: Better indexing performance
  • Upgrade: The version of the Presto server running on the access gateway is now 338. This includes an upgrade to Java 11 on the access gateway.
  • Upgrade: The AMI is now based on the 6/17/2020 version of Linux 2 base AMI
  • Upgrade: AWS SDK for Java to version 1.11.826
  • Upgrade: core.async to version 1.3.610

2020/08/07 0.9.276 - ion-dev && 0.9.48 - ion

Ion Ion dev
  • New feature: Cancel
  • cloud-deps update

2020/07/21 - 0.8.102 - Client-Cloud Update

client-cloud
  • Fix: datomic.api.client.async/client now accepts :dev-local as a :server-type

2020/07/17 - 0.8.101 - Client-Cloud Update

client-cloud

2020/05/14 - 668-8927 - Compute Update

Storage Solo Compute Production Compute Query Groups
  • New feature: qseq
  • New feature: index-pull
  • New feature: xform
  • Enhancement: Improve index efficiency.
  • Enhancement: Improve performance of pull in queries.
  • Enhancement: Improve internal record keeping of active databases that could lead to spurious error messages.
  • Enhancement: Base AMI is now sourced from Amazon Linux 2, and includes recent security updates

Upgrade: As recommended by AWS, we have upgraded our base AMI to Amazon Linux 2 [https://aws.amazon.com/blogs/aws/update-on-amazon-linux-ami-end-of-life/]. This new AMI includes a breaking change to the linux netcat utility that Ions use to deploy new code to a cluster. We encourage all users of Ions to upgrade to version 0.9.265 or later of ion-dev prior to upgrading to this release.

2020/05/14 - 0.9.265 - ion-dev && 0.9.43 - ion

Ion Ion dev

2020/05/14 - 0.8.96 - Client-Cloud Update

client-cloud
  • New feature: qseq
  • New feature: index-pull
  • Updated jetty dependencies from 9.4.24.v20191120 to 9.4.27.v20200227.

2020/04/28 - 0.10.82 - CLI Tools

CLI Tools
0.10.82
  • Enhancement: Add `–ssho` option to the gateway restart command to pass ssh configuration options as if using `ssh -o <option>`

2020/02/21 - 616-8879 - Compute Update

Storage Solo Compute Production Compute Query Groups
  • New feature: `[:db/retract eid aid]` will retract all values for an eid/aid combination.
  • Critical Fix: regression introduced in 569-8835 where `datoms`, `seek-datoms`, and `index-range` return incorrect results if `Iterable.iterator` is called more than once on the returned value.
  • Fix: respect `nil` as a limit in query `pull` expression.
  • Fix: respect `false` as a default in query `pull` expressions for boolean-valued attributes.
  • Fix: allow arbitrary `java.util.List` (not just Clojure vectors) for lookup refs.

2020/02/19 - 0.10.81 - CLI Tools

CLI Tools
0.10.81

2020/02/03 - 0.9.251 - ion-dev

Ion dev
  • Increase timeout for how long a deployment will wait for dependencies to sync on an instance

2020/01/20 - 589-8846 - Storage and Compute Update

Storage Solo Compute Production Compute Query Groups
  • New region: `sa-east-1`
  • Better tagging of resources created by Datomic
  • Fixed problem using with-db from ions

2019/11/26 - CLI Tools - 0.9.33

CLI Tools
0.9.33
  • Note: This release requires a version of Datomic Cloud 569-8835 or newer
  • Enhancement: datomic-gateway restart now just restarts the process running on the access gateway, instead of restarting the whole instance. This results in faster restart times when making configuration changes. Restarting a bastion-only access gateway has no effect.
  • Enhancement: Automatically retrieve the hostkey from the access gateway. There will no longer be a warning about the authenticity of the host.

2019/11/26 - 569-8835 - Storage and Compute Update

Storage Solo Compute Production Compute Query Groups
  • Fix: resolve tempids for reference attributes inside tuples.
  • Enhancement: performance improvement for range predicates in analytics support
  • Enhancement: SSH public keys automatically handled by access gateway and proxy script
  • Enhancement: AutoScalingGroup details exposed as CloudFormation Outputs
  • Enhancement: JDBC metadata support in analytics
  • Upgrade: AWS Lambda Runtimes moved to NodeJs 10.x

2019/11/19 - 0.9.247 - ion-dev

Ion dev
  • Better error message for invalid ion-config.edn.
  • Update commands to match the preferred approach of installing ion-dev in your user deps.edn
  • Use slf4j-simple to prevent console warning when invoking tools.

2019/11/14 - 0.8.81 - Client-Cloud Update

client-cloud
  • Documentation string updates

2019/11/12 - 0.9.240 - ion-dev

Ion dev
  • Upgrade Clojure tools.deps.alpha to 0.8.584

2019/11/06 - CLI Tools - 0.9.21

CLI Tools
0.9.21
  • Enhancement: Added --ssho to allow any SSH -o option to be passed, removed --no-host-key-checking option
  • --instance-wait option renamed to --wait

2019/10/01 - 535-8812 - Compute Template Update

Storage Solo Compute Production Compute Query Groups
  • Fix: prevent exception thrown when handling COUNT(*) clauses in analytics support.
  • Fix: do not create redundant `:db.install/attribute` datoms for idempotent schema operations.
  • Update transactor Clojure dependency to 1.10.1.
  • Enhancement: New AWS Region - ap-northeast-1.

2019/10/01 - CLI Tools - 0.9.11

CLI Tools
0.9.11

2019/10/01 - 512-8806 - Storage and Compute Template Update

Storage Solo Compute Production Compute Query Groups

2019/08/15 - 0.9.234 - ion-dev && 0.9.35 - ion

Ion Ion dev
  • Enhancement: allow for the inclusion of the target dir in the :paths list.

2019/08/06 - 482-8794 - Compute Template Update

Storage Solo Compute Production Compute Query Groups
  • New: Upgrade from t2 to t3 instances for bastion and query groups.
  • Fix: tx-range now returns a map with a :data key, consistent with the docs.
  • Fix: Correctly handle boolean attributes in composite tuples.
  • Fix: Prevent erroneous attempts to use Datomic On-Prem's excision feature.

2019/07/09 - 480-8772 - Compute Template Update

Storage Solo Compute Production Compute Query Groups
  • Fixed problem where databases that change an attribute from `:db.cardinality/one` to`:db.cardinality/many` may become unavailable after a process restart.

2019/06/27 - 480-8770 - Compute Template Update

Storage Solo Compute Production Compute Query Groups

2019/06/18 - 0.8.78 - Client-Cloud Update

client-cloud
  • New feature: Return Maps.
  • New feature: Reverse references in nav (try it in REBL).

2019/06/04 - ion-dev 0.9.231

Ion dev
  • Improvement: Fixed issue where some older Java libraries could not be loaded in an ion application

2019/05/16 - 477-8741 - Compute Template Update - HTTP Direct

Storage Solo Compute Production Compute Query Groups

The 477-8741 release contains fixes, enhancements, and dependency updates:

  • Enhancement: HTTP Direct integration for ions.
  • Enhancement: Improved integration between ions and AWS Network Load Balancers. This eliminates transient errors during rolling deployments.
  • Enhancement: Improved integration between ions and AWS Lambda. This also eliminates transient errors.
  • Fix: Issue where cluster nodes could become unresponsive when serving multiple databases or a burst of `tx-range` queries.
  • Fix: Nested queries could deadlock the query pool on Solo nodes.
  • Fix: Increased direct memory on Production nodes to prevent out-of-direct-memory errors.
  • Update: Version 1.7.26 of org.slf4j libraries.

2019/05/15 - 0.9.229 - ion-dev && 0.9.34 - ion

Ion Ion dev
  • New Feature: HTTP Direct
  • Enhancement: Improved integration between ions and AWS NLBs

2019/04/25 - 470-8654.1 - Compute, Storage, and Query Group Update

Storage Solo Compute Production Compute Query Groups

Note Our Cloud Formation templates use node.js lambdas to create support functions for Cloud Formation. April 30th 2019 is the end of life date for AWS Lambda node.js runtime that we currently use.

Effective May 1, 2019, Datomic Cloud CloudFormation templates older than 4/25/2019 will no longer execute. Datomic systems launched with those templates will continue to run, but any template operations (launching or upgrading CloudFormation stacks) will fail.

All Datomic Cloud users should upgrade to 470-8654.1 at their earliest convenience.

For more information see the AWS runtime support policy.

  • Critical: Updated to Node.js 8.10 runtime. All Datomic Cloud users should upgrade at their earliest convenience.

2019/02/22 - 470-8654 - Compute Template Update

Storage Solo Compute Production Compute Query Groups

Note If you have created a VPC Endpoint using the provided CloudFormation Template, you will need to delete that CloudFormation Stack prior to upgrading to this version of Datomic Cloud.

Attempting to upgrade to or past this release without deleting the Stack will result in a failed update with the message: Export <SystemName>-VpcEndpointServiceName cannot be deleted as it is in use by <SystemName>-vpc-endpoint

See the troubleshooting documentation for more information.

The 470-8654 release contains fixes, enhancements, and dependency updates:

  • Fix: Production nodes now use the documented -Xss values for starting the JVM, preventing stack overflow when compiling complex ion applications.
  • Fix: Fixed bug where ion connections could fail to become aware of recent transactions in a timely manner.
  • Fix: Allow bigint values in transaction data.
  • Fix: Corrected allowlist handling of function calls from datalog rules.
  • Enhancement: New AWS Region - ap-southeast-1.
  • Enhancement: Improved valcache cleanup algorithm.
  • Enhancement: Reduce storage overhead of transactions.
  • Enhancement: Defer creation of VPC Endpoint Service until specifically needed for cross-VPC client access.
  • Update to version 2.9.8 of Jackson.
  • Update to version 1.11.479 of the AWS SDK for Java.

2018/12/10 - 454-8573 - Compute Template Update

Storage Solo Compute Production Compute Query Groups

The 454-8573 release contains a number of performance and availability enhancements:

2018/12/10 - 0.9.186 - ion-dev && 0.9.28 - ion

Ion Ion dev
  • Enhancement: Increase CodeDeploy timeout to 5 minutes.
  • Enhancement: Include doc strings for user facing functions in build artifact

2018/11/28 - 0.8.71 - Client-Cloud update

client-cloud
  • Add compatibility with com.cognitect/aws-api
  • Add Datafy
  • Upgraded http-client to 0.1.87
  • Improved error reporting

2018/10/10 - 441-8505 - Critical Update - Compute Template Update

Storage Solo Compute Production Compute Query Groups

Release 441-8505 includes critical updates. All users of the Production Topology and Query Groups should upgrade to 441-8505 immediately. The Solo Topology is not affected.

  • Critical: fixes a problem that can cause portions of the log to become inaccessible. Please update as soon as possible.

2018/09/07 - 441-8477 - Compute Template Update

Storage Solo Compute Production Compute Query Groups
  • New feature: query groups.
  • New instance type option: i3.xlarge for Production.
  • Improved memory settings: more stack space for Solo, more heap space for Production.
  • Bugfix: fixed bug that could prevent a Production node from beginning indexing jobs.

2018/08/21 - 0.8.63 - Client-Cloud update

client-cloud
  • Bugfix: fixed bug that could cause a with database query to go to the wrong node in a production cluster.
  • Bugfix: fixed Jetty configuration that could cause a client to prevent JVM from shutting down.
  • Upgraded transit-clj to 0.8.313
  • :query-group parameter is no longer required in the client arg-map

2018/08/15 - 0.9.176 - ion-dev && 0.9.26 - ion

Ion Ion dev

2018/08/15 - 409-8407 - Compute Template Update

Storage Solo Compute Production Compute
  • Bugfix: allow retraction of :db/unique attributes
  • Bugfix: coerce Integer values to long when needed in transactions
  • Enhancement: better memory utilization allows larger query results
  • Enhancement: automatically rollback deployments when an ion application fails to load
  • Added support for ap-southeast-2 (Sydney) region

2018/07/10 - 0.9.173 - ion-dev && 0.9.14 - ion

Ion Ion dev
  • New Feature: datomic.ion.cast library for monitoring ions
  • Bugfix: fixed race condition in ion code loading that could allow ion invocation before namespace completely loaded
  • Enhancement: warn on dependency conflicts
  • Improvement: prefer shell-friendly symbols instead of strings as arguments to datomic.ion.dev CLI commands
  • Improvement: better error messaging when deploying to the wrong region.
  • Improvement: list available deploy groups in push output
  • Improvement: enforce the requirement for :uname when project has a :local/root in deps.edn.

2018/07/02 - 0.8.56 - Client Cloud Update

2018/06/29 - 402-8396 - Compute Template Update

Storage Solo Compute Production Compute
  • Upgraded AWS libs to 1.11.349
  • Upgraded Jackson libs to 2.9.5
  • Fixed cache problem where all d/with databases deriving from a common initial call to d/with-db had the same common value.

2018/06/06 - 0.8.54 - client-cloud Update

client-cloud

2018/06/06 - 397-8384 - Storage and Compute Template Update

Storage Solo Compute Production Compute
  • Enhancement: Datomic Ions
  • Improvement: Replaced Application Load Balancer with Network Load Balancer. If your applications run in a separate VPC you will need to configure a VPC endpoint.

2018/02/21 - 303-8300 - Storage and Compute Template Update

Storage Solo Compute Production Compute
  • Bugfix: doubles and floats allowed in transactions
  • Bugfix: avoid unnecessary ":AdopterSkippedOlder" alert when creating new database
  • Update: latest Amazon Linux patches
  • Improvement: better error handling in Storage template
  • Improvement: reduce memcached timeout

2018/01/15 - 297-8291 - Initial Release

Storage Solo Compute Production Compute