Reducing Latency with Transaction Hints

Datomic creates a new database value by accreting a database with a set of new information. In order to accomplish this goal, Datomic reads the db-before to resolve identities in datoms, maintain composite tuple attributes, uphold invariants (e.g. unique values, cardinality-1 retractions, redundancy elimination), as well as to expand information sets or assert application invariants expressed through transaction functions and entity predicates.

Such reads against a db-before bound the minimum transaction latency, and thus it is important for data to be locally available before it is necessary. Datomic strives to prefetch reads as soon as it knows necessity. However, certain data dependencies limit the amount of prefetching possible: checking datom invariants cannot commence until entity IDs are resolved, and transaction functions are opaque and can do arbitrary things.

With Transaction Hints, Peers can analyze a transaction to convey hints that enable the transactor to prefetch data earlier and exhaustively. This can significantly increase transaction performance and is observable through io-stats and tx-stats. Though this requires speculatively running the transaction on the Peer prior to the transactor, Peers can independently and concurrently generate hints that reduce queueing and service time in the transactor.

(d/with db tx-data :return-hints true)

(d/transact conn tx-data :hints (:hints ret-of-with))