Processing Transactions

Submitting Transactions

Transactions can be submitted to a Connection via either the Synchronous or Asynchronous API.

Transactions take a map with the following keys

key required usage
:tx-data yes the data to be transacted
:timeout no timeout after which client stops waiting for a response

The :tx-data format is fully described in the Transaction Data Reference. The following example shows a simple transaction using the Synchronous API.

;; load the Synchronous API with prefix 'd'
(require '[datomic.client.api :as d])

;; Get a connection
(def conn (->
            {:server-type :ion
             :region "region" ;; e.g. us-east-1
             :system "system-name"
             :endpoint "http://entry.system-name.region.datomic.net:8182/"
             :proxy-port 8182}
            (d/client)
            (d/connect {:db-name "database name"})))

;; transact a movie
(def result (d/transact conn {:tx-data [{:db/id "goonies"
                                         :movie/title "The Goonies"
                                         :movie/genre "action/adventure"
                                         :movie/release-year 1985}]}))
;; what id was assigned to The Goonies?
(-> result :tempids (get "goonies"))

Transaction Results

If a transaction completes successfully, data is durably committed to the database, and the transaction API returns a transaction report. A transaction report is a map with the following keys:

key usage
:db-before database value before the transaction
:db-after database value after the transaction
:tx-data datoms produced by the transaction
:tempids map from temporary ids to assigned ids

Both :db-before and :db-after are database values that can be passed to the various query APIs.

The :tx-data member of a transaction report contains the set of datoms created by a transaction (both assertions and retractions). The :tx-data can be used as an input source for a subsequent query.

The :tempids can be used to recover the ids of newly-created entities. This is useful for tasks that add to the same entity across multiple transactions.

Continuing from the transaction in the previous section:

;; Database value before transaction
(def db-before (:db-before result))

;; Database value after transaction
(def db-after (:db-after result))

;; Query attempt on previous db value
(d/q '[:find ?e
       :where [?e :movie/release-year 1985]]
     db-before)

;; Result: empty list, since the data didn't exist before the transaction
[]

;; Query attempt on new db value
(d/q '[:find ?e
       :where [?e :movie/release-year 1985]]
     db-after)

;; Result (your value may vary)
[[8945626603585608]]

;; Grab the temp id for the "goonies" datom /for this transaction/
;; Further "goonies" transactions may not have the same id
(def temp-id (-> result :tempids (get "goonies")))

;; Utilize the temp id against the "after" state of the db
(d/pull db-after '[:movie/title] temp-id)

;; Result
#:movie{:title "The Goonies"}

;; Or against the current db state
(d/pull (d/db conn) '[:movie/title] temp-id)

;; Result
#:movie{:title "The Goonies"}

Transaction Anomalies

Transaction failures are reported as anomalies. For example, the structurally invalid transaction below causes the synchronous API to throw an exception:

(d/transact conn {:tx-data [[:this "does not" :make "sense"]]})
=> ExceptionInfo Unable to resolve data function: :this  clojure.core/ex-info (core.clj:4725)

You can use ex-data to extract the anomaly map from the exception:

;; continuing previous example
(ex-data *e)
{:cognitect.anomalies/category :cognitect.anomalies/incorrect, 
 :cognitect.anomalies/message "Unable to resolve data function: :this", 
 ...}

Tempid Resolution

When a transaction containing temporary ids is processed, each unique temporary id is mapped to an actual entity id. If a given temporary id is used more than once in a given transaction, all instances are mapped to the same actual entity id.

If an entity does not have a unique attribute, then Datomic will assign a new entity id for that entity.

The :tempids key of a transaction result has a mapping from the tempids passed in to the actual ids assigned. The example below demonstrates the resolution of tempids "item-1" and "item-2".

(-> conn
    (d/transact {:tx-data [[:db/add "item-1" :inv/color :green]
                           [:db/add "item-1" :inv/sku "SKU-2001"]
                           [:db/add "item-1" :inv/size :large]
                           {:db/id "item-2"
                            :inv/sku "SKU-2002"
                            :inv/size :small}]})
    :tempids)
=> {"item-2" 2410129488085138, "item-1" 39292147530203281}

Tempids are meaningful only inside the scope of a single transaction,
e.g. the tempid "item-1" will not necessarily refer to the same entity
in two different transactions.

Unique Identities

If an entity has a :db.unique/identity attribute, Datomic will upsert. First Datomic will look for an existing entity with the same value for that unique attribute, and use that id. If no such entity exists, Datomic will assign a new entity id.

The transaction below upserts SKU-42 via tempid "foo" returning the actual entity id:

;; Schema
(d/transact conn {:tx-data [{:db/ident :inv/sku
                             :db/cardinality :db.cardinality/one
                             :db/valueType :db.type/string
                             :db/unique :db.unique/identity}]})

(-> conn
    (d/transact {:tx-data [[:db/add "foo" :inv/sku "SKU-42"]]})
    :tempids
    (get "foo"))

;; Result
49460431063875660

Any subsequent transaction data about :inv/sku "SKU-42" will automatically resolve to the same entity id, regardless of the tempid used. The following transaction uses "bar", and resolves the same entity id of 49460431063875660:

(-> conn
    (d/transact {:tx-data [[:db/add "bar" :inv/sku "SKU-42"]]})
    :tempids
    (get "bar"))

;; Result
49460431063875660

Unique Values

If an entity has a :db.unique/value attribute, Datomic will look for an existing existing entity with the same value for that unique attribute. If such an entity exists, the transaction will abort with an ::anom/conflict. Otherwise, Datomic will assign a new entity id.

In the example below, a :reservation/code must be unique and can never be assigned again. The attempt to use "HQJ43P" on a second entity causes an anomaly.

(d/transact conn {:tx-data [{:db/ident :reservation/code
                             :db/valueType :db.type/string
                             :db/cardinality :db.cardinality/one
                             :db/unique :db.unique/value}]})
(d/transact conn {:tx-data [[:db/add "" :reservation/code "HQJ43P"]]})

(d/transact conn {:tx-data [[:db/add "" :reservation/code "HQJ43P"]]})
;; Result
ExceptionInfo Unique conflict: :reservation/code,
      value: HQJ43P already held by: 35650565019009171
                    asserted for: 26929238787489940

;; Other non-duplicate values work fine
(d/transact conn {:tx-data [[:db/add "" :reservation/code "HJ1337"]]})

;; Result
{:db-before {:database-id ...} ... }

Timeouts

As with all asynchronous operations, Datomic client libraries provide a way to control how long an application waits for a transaction to complete. In Clojure, you can specify a specific timeout when you wait for a transaction to complete.

The example below sets a very low timeout of 1 msec, leading (at least in some tests) to a timeout:

(d/transact conn {:tx-data [[:db/add "" :db/doc "might not succeed!"]]
                  :timeout 1})

;; Result
#:cognitect.anomalies{:category :cognitect.anomalies/interrupted,
                      :message "Datomic Client Timeout"}

When a client times out while waiting for a transaction to complete, the client does not know whether the transaction succeeded and will need to query a recent value of the database to discover what happened. For the previous example, you could use a query like:

(d/q '[:find ?e ?when
      :where [?e :db/doc "might not succeed!" ?tx]
             [?tx :db/txInstant ?when]]
     (d/db conn))

Reified Transactions

When Datomic processes a transaction, it creates a transaction entity to represent it. By default, this entity has two attributes, :db/txInstant, whose value is the instant that the transaction was processed, and :db/txUUID, a Datomic-assigned transaction identifier. In addition, every datom in a transaction refers to the transaction entity through its :tx field.

You can add additional attributes to a transaction entity to capture other useful information, such as the purpose of the transaction, the application that executed it, the provenance of the data it added, or the user who caused it to execute, or any other information that might be useful for auditing purposes.

To annotate the current transaction, include an add statement (either map or list) that uses a :db/id with the value "datomic.tx".

You can query for transaction entities with particular attributes and values the same way you would query for any other entity in the system (see query reference).

This example below create a :data/source attribute to indicate where data came from, and then sets that attribute on a transaction.

(d/transact conn {:tx-data [{:db/ident :data/source
                             :db/valueType :db.type/string
                             :db/cardinality :db.cardinality/one
                             :db/doc "URI this transaction was imported from"}]})
(d/transact conn {:tx-data [{:db/id "datomic.tx"
                             :data/source "http://example.com/catalog-2_29_2012.xml"}
                            {:inv/sku "SKU-42" :inv/color :green :inv/size :large}]})

Explicit :db/txInstant

You can set :db/txInstant explicitly, overriding Datomic's clock time. When you do, you must choose a :db/txInstant value that is not older than any existing transaction, and not newer than the Datomic's clock time. This capability enables initial imports of existing data that has its own timestamps.

The transaction below might occur in an import of historical data, adding information about SKU-42 with a transaction time back in 2001. Note that for this transaction to work, everything already in the database, including e.g. the schema itself, must also have been backdated.

(d/transact conn {:tx-data [{:inv/sku "SKU-42"
                             :inv/color :green}
                            [:db/add "datomic.tx" :db/txInstant #inst "2001"]]})

Redundancy Elimination

A datom is redundant with the current value of the database if there is a matching datom that differs only by transaction id. If a transaction would produce redundant datoms, those datoms are filtered out, and do not appear a second time in either the indexes or the transaction log.

For example, imagine that SKU-42 does not exist and you transact the following:

(-> conn
    (d/transact {:tx-data [{:inv/sku "SKU-42"
                            :inv/color :green}]})
    :tx-data)

The resulting tx-data will contain three datoms: the two datoms about the new SKU-42 entity, and the transaction's :db/txInstant:

e a v
13194139533320 50 #inst "2017-09-03T16:10:38.279-00:00"
49460431063875660 64 "SKU-42"
49460431063875660 65 25235990880714817

If you immediately re-run the same transaction, the two datoms about SKU-42 (entity 49460431063875660 above) are now redundant, and will not be added to the database again.

e a v
13194139533322 50 #inst "2017-09-03T16:11:43.332-00:00"

Note that transactions are never entirely redundant, because the transaction's :db/txInstant is always a new fact.