Datomic Cloud Releases

Note Our Cloud Formation templates use node.js lambdas to create support functions for Cloud Formation. May 31, 2021 is the end of life date for AWS Lambda node.js runtime that we currently use.

Effective June 1, 2021, Datomic Cloud CloudFormation templates older than March 3, 2021 will no longer execute. Datomic systems launched with those templates will continue to run, but any template operations (launching or upgrading CloudFormation stacks) will fail.

All Datomic Cloud users should upgrade to 781-9041 at their earliest convenience.

For more information see the AWS runtime support policy.

• Upgrade: AWS Lambda Runtimes moved to NodeJs 12.x
• Enhancement: New AWS Region - ca-central-1
• Upgrade Jetty server to version 9.4.36.v20210114

• ion-dev
  • Improvement: Limit how long to wait for a cluster node to gracefully shutdown.
  • Improvement: Increase performance of Ion request handler
  • Fix: Suppress noisy AsyncContext Alert caused by Ion timeout
• ion
  • Improvement: get-params now allows arbitrary number of parameters

• Fix: Attribute Predicates are now applied to assertions only.
• Upgrade to presto 348 for Datomic analytics.

• Improvment: Make client dynaload thread safe

• New: Change the scale of a BigDecimal attribute in a transaction.
• Upgrade to presto 346, fixing a bug that prevents analytics gateway from launching.
• Fix: Reenable query counts metric in dashboard.
• Fix: Query correctly treats range functions as functions (not as predicates).
• New region: eu-north-1

• New region: ap-south-1
• Fix: Improve storage garbage collection.
• Fix: Prevent situation where tx-range sometimes returned one extra transaction prior to a specified instant.

• New feature: Cancel
• New region: `eu-west-2`
• Fix: Prevent race condition in valcache that could lead to a failed read
• Improvement: Better entity predicate error messages
• Improvement: Better indexing performance
• Upgrade: The version of the Presto server running on the access gateway is now 338. This includes an upgrade to Java 11 on the access gateway.
• Upgrade: The AMI is now based on the 6/17/2020 version of Linux 2 base AMI
• Upgrade: AWS SDK for Java to version 1.11.826
• Upgrade: core.async to version 1.3.610

• New feature: Cancel
• cloud-deps update

• Fix: datomic.api.client.async/client now accepts :dev-local as a :server-type

• New feature: dev-local

• New feature: qseq
• New feature: index-pull
• New feature: xform
• Enhancement: Improve index efficiency.
• Enhancement: Improve performance of pull in queries.
• Enhancement: Improve internal record keeping of active databases that could lead to spurious error messages.
• Enhancement: Base AMI is now sourced from Amazon Linux 2, and includes recent security updates

Upgrade: As recommended by AWS, we have upgraded our base AMI to Amazon Linux 2 [https://aws.amazon.com/blogs/aws/update-on-amazon-linux-ami-end-of-life/]. This new AMI includes a breaking change to the linux netcat utility that Ions use to deploy new code to a cluster. We encourage all users of Ions to upgrade to version 0.9.265 or later of ion-dev prior to upgrading to this release.

• Enhancement: Calculate transitive list of dependencies in `:dependency-conflict` map
• Update dep list for Cloud 668-8927 release

• New feature: qseq
• New feature: index-pull
• Updated jetty dependencies from 9.4.24.v20191120 to 9.4.27.v20200227.

• Enhancement: Add `–ssho` option to the gateway restart command to pass ssh configuration options as if using `ssh -o <option>`

• New feature: `[:db/retract eid aid]` will retract all values for an eid/aid combination.
• Critical Fix: regression introduced in 569-8835 where `datoms`, `seek-datoms`, and `index-range` return incorrect results if `Iterable.iterator` is called more than once on the returned value.
• Fix: respect `nil` as a limit in query `pull` expression.
• Fix: respect `false` as a default in query `pull` expressions for boolean-valued attributes.
• Fix: allow arbitrary `java.util.List` (not just Clojure vectors) for lookup refs.

• Many new features - see docs

• Increase timeout for how long a deployment will wait for dependencies to sync on an instance

• New region: `sa-east-1`
• Better tagging of resources created by Datomic
• Fixed problem using with-db from ions

• Note: This release requires a version of Datomic Cloud 569-8835 or newer
• Enhancement: datomic-gateway restart now just restarts the process running on the access gateway, instead of restarting the whole instance. This results in faster restart times when making configuration changes. Restarting a bastion-only access gateway has no effect.
• Enhancement: Automatically retrieve the hostkey from the access gateway. There will no longer be a warning about the authenticity of the host.

• Fix: resolve tempids for reference attributes inside tuples.
• Enhancement: performance improvement for range predicates in analytics support
• Enhancement: SSH public keys automatically handled by access gateway and proxy script
• Enhancement: AutoScalingGroup details exposed as CloudFormation Outputs
• Enhancement: JDBC metadata support in analytics
• Upgrade: AWS Lambda Runtimes moved to NodeJs 10.x

• Better error message for invalid ion-config.edn.
• Update commands to match the preferred approach of installing ion-dev in your user deps.edn
• Use slf4j-simple to prevent console warning when invoking tools.

• Documentation string updates

• Upgrade Clojure tools.deps.alpha to 0.8.584

• Enhancement: Added --ssho to allow any SSH -o option to be passed, removed --no-host-key-checking option
• --instance-wait option renamed to --wait

• Fix: prevent exception thrown when handling COUNT(*) clauses in analytics support.
• Fix: do not create redundant `:db.install/attribute` datoms for idempotent schema operations.
• Update transactor Clojure dependency to 1.10.1.
• Enhancement: New AWS Region - ap-northeast-1.

• Initial release of CLI tools

• New feature preview: Analytics. See https://docs.datomic.com/cloud/analytics/analytics-concepts.html.
• Fix: Allow Datomic Cloud to launch correctly in accounts that have a large number of existing ASGs.

• Enhancement: allow for the inclusion of the target dir in the :paths list.

• New: Upgrade from t2 to t3 instances for bastion and query groups.
• Fix: tx-range now returns a map with a :data key, consistent with the docs.
• Fix: Correctly handle boolean attributes in composite tuples.
• Fix: Prevent erroneous attempts to use Datomic On-Prem's excision feature.

• Fixed problem where databases that change an attribute from `:db.cardinality/one` to`:db.cardinality/many` may become unavailable after a process restart.

• New feature: Tuples.
• New feature: Attribute predicates.
• New feature: Entity specs.
• New feature: Return maps.
• Fix: `sample` aggregate could hang query thread.

• New feature: Return Maps.
• New feature: Reverse references in nav (try it in REBL).

• Improvement: Fixed issue where some older Java libraries could not be loaded in an ion application

The 477-8741 release contains fixes, enhancements, and dependency updates:

• Enhancement: HTTP Direct integration for ions.
• Enhancement: Improved integration between ions and AWS Network Load Balancers. This eliminates transient errors during rolling deployments.
• Enhancement: Improved integration between ions and AWS Lambda. This also eliminates transient errors.
• Fix: Issue where cluster nodes could become unresponsive when serving multiple databases or a burst of `tx-range` queries.
• Fix: Nested queries could deadlock the query pool on Solo nodes.
• Fix: Increased direct memory on Production nodes to prevent out-of-direct-memory errors.
• Update: Version 1.7.26 of org.slf4j libraries.

• New Feature: HTTP Direct
• Enhancement: Improved integration between ions and AWS NLBs

Note Our Cloud Formation templates use node.js lambdas to create support functions for Cloud Formation. April 30th 2019 is the end of life date for AWS Lambda node.js runtime that we currently use.

Effective May 1, 2019, Datomic Cloud CloudFormation templates older than 4/25/2019 will no longer execute. Datomic systems launched with those templates will continue to run, but any template operations (launching or upgrading CloudFormation stacks) will fail.

All Datomic Cloud users should upgrade to 470-8654.1 at their earliest convenience.

For more information see the AWS runtime support policy.

• Critical: Updated to Node.js 8.10 runtime. All Datomic Cloud users should upgrade at their earliest convenience.

Note If you have created a VPC Endpoint using the provided CloudFormation Template, you will need to delete that CloudFormation Stack prior to upgrading to this version of Datomic Cloud.

Attempting to upgrade to or past this release without deleting the Stack will result in a failed update with the message: Export <SystemName>-VpcEndpointServiceName cannot be deleted as it is in use by <SystemName>-vpc-endpoint

See the troubleshooting documentation for more information.

The 470-8654 release contains fixes, enhancements, and dependency updates:

• Fix: Production nodes now use the documented -Xss values for starting the JVM, preventing stack overflow when compiling complex ion applications.
• Fix: Fixed bug where ion connections could fail to become aware of recent transactions in a timely manner.
• Fix: Allow bigint values in transaction data.
• Fix: Corrected allowlist handling of function calls from datalog rules.
• Enhancement: New AWS Region - ap-southeast-1.
• Enhancement: Improved valcache cleanup algorithm.
• Enhancement: Reduce storage overhead of transactions.
• Enhancement: Defer creation of VPC Endpoint Service until specifically needed for cross-VPC client access.
• Update to version 2.9.8 of Jackson.
• Update to version 1.11.479 of the AWS SDK for Java.

The 454-8573 release contains a number of performance and availability enhancements:

• Enhancement: You can configure systems to preload a database before serving requests, eliminating a source of unavailable anomalies.
• Enhancement: Ion deployments load active databases before serving requests, eliminating a source of unavailable anomalies.
• Enhancement: Improved throughput for transactions initiated by ion applications running the Production Topology.
• Enhancement: Improved performance for systems that create and delete many databases (e.g. test systems)
• Enhancement: Improvements to logging.

• Enhancement: Increase CodeDeploy timeout to 5 minutes.
• Enhancement: Include doc strings for user facing functions in build artifact

• Add compatibility with com.cognitect/aws-api
• Add Datafy
• Upgraded http-client to 0.1.87
• Improved error reporting

Release 441-8505 includes critical updates. All users of the Production Topology and Query Groups should upgrade to 441-8505 immediately. The Solo Topology is not affected.

• Critical: fixes a problem that can cause portions of the log to become inaccessible. Please update as soon as possible.

• New feature: query groups.
• New instance type option: i3.xlarge for Production.
• Improved memory settings: more stack space for Solo, more heap space for Production.
• Bugfix: fixed bug that could prevent a Production node from beginning indexing jobs.

• Bugfix: fixed bug that could cause a with database query to go to the wrong node in a production cluster.
• Bugfix: fixed Jetty configuration that could cause a client to prevent JVM from shutting down.
• Upgraded transit-clj to 0.8.313
• :query-group parameter is no longer required in the client arg-map

• Enhancement: parameters library for working with the AWS Systems Manager Parameter Store

• Bugfix: allow retraction of :db/unique attributes
• Bugfix: coerce Integer values to long when needed in transactions
• Enhancement: better memory utilization allows larger query results
• Enhancement: automatically rollback deployments when an ion application fails to load
• Added support for ap-southeast-2 (Sydney) region

• New Feature: datomic.ion.cast library for monitoring ions
• Bugfix: fixed race condition in ion code loading that could allow ion invocation before namespace completely loaded
• Enhancement: warn on dependency conflicts
• Improvement: prefer shell-friendly symbols instead of strings as arguments to datomic.ion.dev CLI commands
• Improvement: better error messaging when deploying to the wrong region.
• Improvement: list available deploy groups in push output
• Improvement: enforce the requirement for :uname when project has a :local/root in deps.edn.

• Enhancement: added sync to client API. See https://docs.datomic.com/cloud/transcactions/client-synchronization.html
• Better error message when unable to connect to cluster or proxy.

• Upgraded AWS libs to 1.11.349
• Upgraded Jackson libs to 2.9.5
• Fixed cache problem where all d/with databases deriving from a common initial call to d/with-db had the same common value.

• Enhancement: added :server-type :ion
• Enhancement: ensure recentness of d/db conn

• Enhancement: Datomic Ions
• Improvement: Replaced Application Load Balancer with Network Load Balancer. If your applications run in a separate VPC you will need to configure a VPC endpoint.

• Bugfix: doubles and floats allowed in transactions
• Bugfix: avoid unnecessary ":AdopterSkippedOlder" alert when creating new database
• Update: latest Amazon Linux patches
• Improvement: better error handling in Storage template
• Improvement: reduce memcached timeout