Caching

Datomic Cloud's cache hierarchy includes the object cache, valcache, and EFS cache.

../images/cache-hierarchy.png

Datomic caches:

  • contain segments of index or log, typically a few thousand datoms per segment
  • contain only immutable data
  • are always consistent
  • are transparent to application code
  • require no configuration
  • are for performance only, and have no bearing on transactional guarantees

Segments are always encrypted in transit and at rest.

Object Cache

Nodes maintain an LRU cache of segments as Java objects. When a node needs a segment, Datomic looks in the object cache first. If a segment is unavailable in the object cache, Datomic will look next to the valcache.

Because each process maintains its own object cache, a process will automatically adjust over time to its workload.

Valcache

Primary Compute Nodes maintain a valcache. Valcache implements an immutable subset of the memcached API and maintains an LRU cache of backed by fast local SSDs.

If a segment is unavailable in valcache, Datomic will look next to the EFS cache.

EFS Cache

The EFS cache contains the entirety of all indexes for all databases. Datomic uses the EFS cache to populate the smaller and faster valcache and object cache, without the latency of reading from S3.

If a segment is unavailable in the EFS cache, Datomic will load the segment from S3 and repair the EFS cache.