«

Processing Transactions

Submitting Transactions

Transactions can be submitted to a Connection via either the Synchronous or Asynchronous API.

Transactions take a map with the following keys

key required usage
:tx-data yes the data to be transacted
:timeout no timeout after which client stops waiting for a response

The :tx-data format is fully described in the Transaction Data Reference. The following example shows a simple transaction using the Synchronous API.

;; load the Synchronous API with alias 'd'
(require '[datomic.client.api :as d])

;; Get a connection
(def conn (->
            {:server-type :ion
             :region "region" ;; e.g. us-east-1
             :system "system-name"
             :endpoint "http://entry.system-name.region.datomic.net:8182/"
             :proxy-port 8182}
            (d/client)
            (d/connect {:db-name "database name"})))

;; transact a movie
(def result (d/transact conn {:tx-data [{:db/id "goonies"
                                         :movie/title "The Goonies"
                                         :movie/genre "action/adventure"
                                         :movie/release-year 1985}]}))
;; what id was assigned to The Goonies?
(-> result :tempids (get "goonies"))

Nested maps in transactions

Often, a group of related entities are created or modified in the same transaction. Nested maps allow you to specify these related entities together in a single map. If an entity map contains a nested map as a value for a reference attribute, Datomic will expand the nested map into its own entity. Nested map expansion is governed by two rules:

  • If the nested map does not include a :db/id, Datomic will assign a :db/id automatically, using the same partition as the :db/id of the outer entity.
  • Either the reference to the nested map must be a component attribute, or the nested map must include a unique attribute. This constraint prevents the accidental creation of easily-orphaned entities that have no identity or relation to other entities.

As an example, the following data uses nested maps to specify two line items belonging to an order:

[{:db/id order-id
  :order/lineItems [{:lineItem/product chocolate
                     :lineItem/quantity 1}
                    {:lineItem/product whisky
                     :lineItem/quantity 2}]}]

Notice that

  • The two line items do not need to specify a :db/id, and will automatically get ids in the same partition as the order-id entity.
  • Since the line items do not have unique ids, you can infer that :order/lineItems must be a component attribute. (This is sensible given the domain. Line items have no independent existence outside orders.)

Nested maps are often much more convenient than their equivalent flat expansions. The data below shows the same information order and line items as three independent entity maps.

[{:db/id order-id
  :order/lineItems [item-1-id, item-2-id]}
 {:db/id item-1-id
  :lineItem/product chocolate
  :lineItem/quantity 1}
 {:db/id item-2-id
  :lineItem/product whisky
  :lineItem/quantity 2}]

In addition to being more verbose, this form requires additional work to explicitly manage item-1-id and item-2-id and their connections to the containing order.

Transaction Results

If a transaction completes successfully, data is durably committed to the database, and the transaction API returns a transaction report. A transaction report is a map with the following keys:

key usage
:db-before database value before the transaction
:db-after database value after the transaction
:tx-data datoms produced by the transaction
:tempids map from temporary ids to assigned ids

Both :db-before and :db-after are database values that can be passed to the various query APIs.

The :tx-data member of a transaction report contains the set of datoms created by a transaction (both assertions and retractions). The :tx-data can be used as an input source for a subsequent query.

The :tempids can be used to recover the ids of newly-created entities. This is useful for tasks that add to the same entity across multiple transactions.

Continuing from the transaction in the previous section:

;; Database value before transaction
(def db-before (:db-before result))

;; Database value after transaction
(def db-after (:db-after result))

;; Query attempt on previous db value
(d/q '[:find ?e
       :where [?e :movie/release-year 1985]]
     db-before)

=>
[] ;; empty list, since the data didn't exist before the transaction
;; Query attempt on new db value
(d/q '[:find ?e
       :where [?e :movie/release-year 1985]]
     db-after)

=>
[[8945626603585608]] ;; (your value may vary)
;; Grab the temp id for the "goonies" datom /for this transaction/
;; Further "goonies" transactions may not have the same id
(def temp-id (-> result :tempids (get "goonies")))

;; Utilize the temp id against the "after" state of the db
(d/pull db-after '[:movie/title] temp-id)
=>
#:movie{:title "The Goonies"}
;; Or against the current db state
(d/pull (d/db conn) '[:movie/title] temp-id)
=>
#:movie{:title "The Goonies"}

Transaction Anomalies

Transaction failures are reported as anomalies. For example, the structurally invalid transaction below causes the synchronous API to throw an exception:

(d/transact conn {:tx-data [[:this "does not" :make "sense"]]})
=>
ExceptionInfo Unable to resolve data function: :this  clojure.core/ex-info (core.clj:4725)

You can use ex-data to extract the anomaly map from the exception:

;; continuing previous example
(ex-data *e)
=>
{:cognitect.anomalies/category :cognitect.anomalies/incorrect, 
 :cognitect.anomalies/message "Unable to resolve data function: :this", 
 ...}

Tempid Resolution

When a transaction containing temporary ids is processed, each unique temporary id is mapped to an actual entity id. If a given temporary id is used more than once in a given transaction, all instances are mapped to the same actual entity id.

If an entity does not have a unique attribute, then Datomic will assign a new entity id for that entity.

The :tempids key of a transaction result has a mapping from the tempids passed in to the actual ids assigned. The example below demonstrates the resolution of tempids "item-1" and "item-2".

(-> conn
    (d/transact {:tx-data [[:db/add "item-1" :inv/color :green]
                           [:db/add "item-1" :inv/sku "SKU-2001"]
                           [:db/add "item-1" :inv/size :large]
                           {:db/id "item-2"
                            :inv/sku "SKU-2002"
                            :inv/size :small}]})
    :tempids)
=>
{"item-2" 2410129488085138, "item-1" 39292147530203281}

;; Tempids are meaningful only inside the scope of a single transaction,
;; e.g. the tempid "item-1" will not necessarily refer to the same entity
;; in two different transactions.

Unique Identities

If an entity has a :db.unique/identity attribute, Datomic will upsert. First Datomic will look for an existing entity with the same value for that unique attribute, and use that id. If no such entity exists, Datomic will assign a new entity id.

The transaction below upserts SKU-42 via tempid "foo" returning the actual entity id:

;; Schema
(d/transact conn {:tx-data [{:db/ident :inv/sku
                             :db/cardinality :db.cardinality/one
                             :db/valueType :db.type/string
                             :db/unique :db.unique/identity}]})

(-> conn
    (d/transact {:tx-data [[:db/add "foo" :inv/sku "SKU-42"]]})
    :tempids
    (get "foo"))
=>
49460431063875660

Any subsequent transaction data about :inv/sku "SKU-42" will automatically resolve to the same entity id, regardless of the tempid used. The following transaction uses "bar", and resolves the same entity id of 49460431063875660:

(-> conn
    (d/transact {:tx-data [[:db/add "bar" :inv/sku "SKU-42"]]})
    :tempids
    (get "bar"))
=>
49460431063875660

Unique Values

If an entity has a :db.unique/value attribute, Datomic will look for an existing entity with the same value for that unique attribute. If such an entity exists, the transaction will abort with an :cognitect.anomalies/conflict. Otherwise, Datomic will assign a new entity id.

In the example below, a :reservation/code must be unique and can never be assigned again. The attempt to use "HQJ43P" on a second entity causes an anomaly.

(d/transact conn {:tx-data [{:db/ident :reservation/code
                             :db/valueType :db.type/string
                             :db/cardinality :db.cardinality/one
                             :db/unique :db.unique/value}]})
(d/transact conn {:tx-data [[:db/add "" :reservation/code "HQJ43P"]]})

(d/transact conn {:tx-data [[:db/add "" :reservation/code "HQJ43P"]]})
=>
ExceptionInfo Unique conflict: :reservation/code,
      value: HQJ43P already held by: 35650565019009171
                    asserted for: 26929238787489940

;; Other non-duplicate values work fine
(d/transact conn {:tx-data [[:db/add "" :reservation/code "HJ1337"]]})
=>
{:db-before {:database-id ...} ... }

Timeouts

As with all asynchronous operations, Datomic client libraries provide a way to control how long an application waits for a transaction to complete. In Clojure, you can specify a specific timeout when you wait for a transaction to complete.

The example below sets a very low timeout of 1 msec, leading (at least in some tests) to a timeout:

(d/transact conn {:tx-data [[:db/add "" :db/doc "might not succeed!"]]
                  :timeout 1})
=>
#:cognitect.anomalies{:category :cognitect.anomalies/interrupted,
                      :message "Datomic Client Timeout"}

When a client times out while waiting for a transaction to complete, the client does not know whether the transaction succeeded and will need to query a recent value of the database to discover what happened. For the previous example, you could use a query like:

(d/q '[:find ?e ?when
      :where [?e :db/doc "might not succeed!" ?tx]
             [?tx :db/txInstant ?when]]
     (d/db conn))

Reified Transactions

When Datomic processes a transaction, it creates a transaction entity to represent it. By default this entity has a :db/txInstant attribute whose value is the instant that the transaction was processed. Every datom in a transaction refers to the transaction entity through its :tx field.

You can add additional attributes to a transaction entity to capture other useful information, such as the purpose of the transaction, the application that executed it, the provenance of the data it added, the user who caused it to execute, or any other information that might be useful for auditing purposes.

To annotate the current transaction, include an add statement (either map or list) that uses a :db/id with the value "datomic.tx".

You can query for transaction entities with particular attributes and values the same way you would query for any other entity in the system (see query reference).

This example below create a :data/source attribute to indicate where data came from, and then sets that attribute on a transaction.

(d/transact conn {:tx-data [{:db/ident :data/source
                             :db/valueType :db.type/string
                             :db/cardinality :db.cardinality/one
                             :db/doc "URI this transaction was imported from"}]})
(d/transact conn {:tx-data [{:db/id "datomic.tx"
                             :data/source "http://example.com/catalog-2_29_2012.xml"}
                            {:inv/sku "SKU-42" :inv/color :green :inv/size :large}]})

Explicit :db/txInstant

You can set :db/txInstant explicitly, overriding Datomic's clock time. When you do, you must choose a :db/txInstant value that is not older than any existing transaction, and not newer than the Datomic's clock time. This capability enables initial imports of existing data that has its own timestamps.

The transaction below might occur in an import of historical data, adding information about SKU-42 with a transaction time back in 2001. Note that for this transaction to work, everything already in the database, including e.g. the schema itself, must also have been backdated.

(d/transact conn {:tx-data [{:inv/sku "SKU-42"
                             :inv/color :green}
                            [:db/add "datomic.tx" :db/txInstant #inst "2001"]]})

Redundancy Elimination

A datom is redundant with the current value of the database if there is a matching datom that differs only by transaction id. If a transaction would produce redundant datoms, those datoms are filtered out, and do not appear a second time in either the indexes or the transaction log.

For example, imagine that SKU-42 does not exist and you transact the following:

(-> conn
    (d/transact {:tx-data [{:inv/sku "SKU-42"
                            :inv/color :green}]})
    :tx-data)

The resulting tx-data will contain three datoms: the two datoms about the new SKU-42 entity, and the transaction's :db/txInstant:

e a v
13194139533320 50 #inst "2017-09-03T16:10:38.279-00:00"
49460431063875660 64 "SKU-42"
49460431063875660 65 25235990880714817

If you immediately re-run the same transaction, the two datoms about SKU-42 (entity 49460431063875660 above) are now redundant, and will not be added to the database again.

e a v
13194139533322 50 #inst "2017-09-03T16:11:43.332-00:00"

Note that transactions are never entirely redundant, because the transaction's :db/txInstant is always a new fact.

Cancel

datomic.ion/cancel cancels the current Datomic query or transaction, and ​throws an ex-info with an anomaly to the original caller.

cancel requires a map with the key :cognitect.anomalies/category, which has valid values of:

  • :cognitect.anomalies/incorrect
  • :cognitect.anomalies/conflict

When :cognitect.anomalies/message is provided, the message will be used as the Exception's detail message.

All other keys should be namespace-qualified and all data passed to cancel must be transit-serializable.

The example below uses a transaction function to ensure that users always have a name and email. The first transaction succeeds, but the second is canceled since :address is passed instead of :email.

;; Transaction function
(defn add-user [db umap]
  (if (every? umap [:name :email])
    [{:user/name (:name umap)
      :user/email (:email umap)}]
    (datomic.ion/cancel {:cognitect.anomalies/category :cognitect.anomalies/incorrect
                         :cognitect.anomalies/message "User map must contain :email and :name"})))
​
(d/transact conn {:tx-data ['(datomic.sample.txfns/add-user {:name "Marshall" :email "test@test.com"})]})
=> ;; tx result map
(d/transact conn {:tx-data ['(datomic.sample.txfns/add-user {:name "Marshall" :address "test@test.com"})]})
=> Execution error (ExceptionInfo) at datomic.client.api.async/ares (async.clj:58).