Processing Transactions
Table of Contents
Submitting Transactions
Transactions can be submitted to a Connection via either the Synchronous or Asynchronous API.
Transactions take a map with the following keys
key | required | usage |
---|---|---|
:tx-data | yes | the data to be transacted |
:timeout | no | timeout after which client stops waiting for a response |
The :tx-data
format is fully described in the Transaction Data
Reference. The following example shows a simple transaction using the
Synchronous API.
;; load the Synchronous API with alias 'd' (require '[datomic.client.api :as d]) ;; Get a connection (def conn (-> {:server-type :ion :region "region" ;; e.g. us-east-1 :system "system-name" :endpoint "http://entry.system-name.region.datomic.net:8182/" :proxy-port 8182} (d/client) (d/connect {:db-name "database name"}))) ;; transact a movie (def result (d/transact conn {:tx-data [{:db/id "goonies" :movie/title "The Goonies" :movie/genre "action/adventure" :movie/release-year 1985}]})) ;; what id was assigned to The Goonies? (-> result :tempids (get "goonies"))
Nested maps in transactions
Often, a group of related entities are created or modified in the same transaction. Nested maps allow you to specify these related entities together in a single map. If an entity map contains a nested map as a value for a reference attribute, Datomic will expand the nested map into its own entity. Nested map expansion is governed by two rules:
- If the nested map does not include a :db/id, Datomic will assign a :db/id automatically, using the same partition as the :db/id of the outer entity.
- Either the reference to the nested map must be a component attribute, or the nested map must include a unique attribute. This constraint prevents the accidental creation of easily-orphaned entities that have no identity or relation to other entities.
As an example, the following data uses nested maps to specify two line items belonging to an order:
[{:db/id order-id :order/lineItems [{:lineItem/product chocolate :lineItem/quantity 1} {:lineItem/product whisky :lineItem/quantity 2}]}]
Notice that
- The two line items do not need to specify a :db/id, and will automatically get ids in the same partition as the order-id entity.
- Since the line items do not have unique ids, you can infer that :order/lineItems must be a component attribute. (This is sensible given the domain. Line items have no independent existence outside orders.)
Nested maps are often much more convenient than their equivalent flat expansions. The data below shows the same information order and line items as three independent entity maps.
[{:db/id order-id :order/lineItems [item-1-id, item-2-id]} {:db/id item-1-id :lineItem/product chocolate :lineItem/quantity 1} {:db/id item-2-id :lineItem/product whisky :lineItem/quantity 2}]
In addition to being more verbose, this form requires additional work to explicitly manage item-1-id and item-2-id and their connections to the containing order.
Transaction Results
If a transaction completes successfully, data is durably committed to the database, and the transaction API returns a transaction report. A transaction report is a map with the following keys:
key | usage |
---|---|
:db-before | database value before the transaction |
:db-after | database value after the transaction |
:tx-data | datoms produced by the transaction |
:tempids | map from temporary ids to assigned ids |
Both :db-before
and :db-after
are database values that can be
passed to the various query APIs.
The :tx-data
member of a transaction report contains the set of
datoms created by a transaction (both assertions and retractions). The
:tx-data
can be used as an input source for a subsequent query.
The :tempids
can be used to recover the ids of newly-created
entities. This is useful for tasks that add to the same entity across
multiple transactions.
Continuing from the transaction in the previous section:
;; Database value before transaction (def db-before (:db-before result)) ;; Database value after transaction (def db-after (:db-after result)) ;; Query attempt on previous db value (d/q '[:find ?e :where [?e :movie/release-year 1985]] db-before)
=> [] ;; empty list, since the data didn't exist before the transaction
;; Query attempt on new db value (d/q '[:find ?e :where [?e :movie/release-year 1985]] db-after)
=> [[8945626603585608]] ;; (your value may vary)
;; Grab the temp id for the "goonies" datom /for this transaction/ ;; Further "goonies" transactions may not have the same id (def temp-id (-> result :tempids (get "goonies"))) ;; Utilize the temp id against the "after" state of the db (d/pull db-after '[:movie/title] temp-id)
=> #:movie{:title "The Goonies"}
;; Or against the current db state (d/pull (d/db conn) '[:movie/title] temp-id)
=> #:movie{:title "The Goonies"}
Transaction Anomalies
Transaction failures are reported as anomalies. For example, the structurally invalid transaction below causes the synchronous API to throw an exception:
(d/transact conn {:tx-data [[:this "does not" :make "sense"]]})
=> ExceptionInfo Unable to resolve data function: :this clojure.core/ex-info (core.clj:4725)
You can use ex-data to extract the anomaly map from the exception:
;; continuing previous example (ex-data *e)
=> {:cognitect.anomalies/category :cognitect.anomalies/incorrect, :cognitect.anomalies/message "Unable to resolve data function: :this", ...}
Tempid Resolution
When a transaction containing temporary ids is processed, each unique temporary id is mapped to an actual entity id. If a given temporary id is used more than once in a given transaction, all instances are mapped to the same actual entity id.
If an entity does not have a unique attribute, then Datomic will assign a new entity id for that entity.
The :tempids
key of a transaction result has a mapping from the
tempids passed in to the actual ids assigned. The example below
demonstrates the resolution of tempids "item-1" and "item-2".
(-> conn (d/transact {:tx-data [[:db/add "item-1" :inv/color :green] [:db/add "item-1" :inv/sku "SKU-2001"] [:db/add "item-1" :inv/size :large] {:db/id "item-2" :inv/sku "SKU-2002" :inv/size :small}]}) :tempids)
=> {"item-2" 2410129488085138, "item-1" 39292147530203281} ;; Tempids are meaningful only inside the scope of a single transaction, ;; e.g. the tempid "item-1" will not necessarily refer to the same entity ;; in two different transactions.
Unique Identities
If an entity has a :db.unique/identity
attribute, Datomic will
upsert. First Datomic will look for an existing entity with the same
value for that unique attribute, and use that id. If no such entity
exists, Datomic will assign a new entity id.
The transaction below upserts SKU-42 via tempid "foo" returning the actual entity id:
;; Schema (d/transact conn {:tx-data [{:db/ident :inv/sku :db/cardinality :db.cardinality/one :db/valueType :db.type/string :db/unique :db.unique/identity}]}) (-> conn (d/transact {:tx-data [[:db/add "foo" :inv/sku "SKU-42"]]}) :tempids (get "foo"))
=> 49460431063875660
Any subsequent transaction data about :inv/sku
"SKU-42" will
automatically resolve to the same entity id, regardless of the tempid
used. The following transaction uses "bar", and resolves the same
entity id of 49460431063875660:
(-> conn (d/transact {:tx-data [[:db/add "bar" :inv/sku "SKU-42"]]}) :tempids (get "bar"))
=> 49460431063875660
Unique Values
If an entity has a :db.unique/value
attribute, Datomic will look for
an existing entity with the same value for that unique
attribute. If such an entity exists, the transaction will abort with an
:cognitect.anomalies/conflict
. Otherwise, Datomic will assign a new entity id.
In the example below, a :reservation/code
must be unique and can
never be assigned again. The attempt to use "HQJ43P" on a second
entity causes an anomaly.
(d/transact conn {:tx-data [{:db/ident :reservation/code :db/valueType :db.type/string :db/cardinality :db.cardinality/one :db/unique :db.unique/value}]}) (d/transact conn {:tx-data [[:db/add "" :reservation/code "HQJ43P"]]}) (d/transact conn {:tx-data [[:db/add "" :reservation/code "HQJ43P"]]})
=> ExceptionInfo Unique conflict: :reservation/code, value: HQJ43P already held by: 35650565019009171 asserted for: 26929238787489940
;; Other non-duplicate values work fine (d/transact conn {:tx-data [[:db/add "" :reservation/code "HJ1337"]]})
=> {:db-before {:database-id ...} ... }
Timeouts
As with all asynchronous operations, Datomic client libraries provide a way to control how long an application waits for a transaction to complete. In Clojure, you can specify a specific timeout when you wait for a transaction to complete.
The example below sets a very low timeout of 1 msec, leading (at least in some tests) to a timeout:
(d/transact conn {:tx-data [[:db/add "" :db/doc "might not succeed!"]] :timeout 1})
=> #:cognitect.anomalies{:category :cognitect.anomalies/interrupted, :message "Datomic Client Timeout"}
When a client times out while waiting for a transaction to complete, the client does not know whether the transaction succeeded and will need to query a recent value of the database to discover what happened. For the previous example, you could use a query like:
(d/q '[:find ?e ?when :where [?e :db/doc "might not succeed!" ?tx] [?tx :db/txInstant ?when]] (d/db conn))
Reified Transactions
When Datomic processes a transaction, it creates a transaction entity to
represent it. By default this entity has a :db/txInstant
attribute
whose value is the instant that the transaction was processed.
Every datom in a transaction refers to the transaction
entity through its :tx
field.
You can add additional attributes to a transaction entity to capture other useful information, such as the purpose of the transaction, the application that executed it, the provenance of the data it added, the user who caused it to execute, or any other information that might be useful for auditing purposes.
To annotate the current transaction, include an add statement
(either map or list) that uses a :db/id
with the value "datomic.tx".
You can query for transaction entities with particular attributes and values the same way you would query for any other entity in the system (see query reference).
This example below create a :data/source
attribute to indicate where
data came from, and then sets that attribute on a transaction.
(d/transact conn {:tx-data [{:db/ident :data/source :db/valueType :db.type/string :db/cardinality :db.cardinality/one :db/doc "URI this transaction was imported from"}]}) (d/transact conn {:tx-data [{:db/id "datomic.tx" :data/source "http://example.com/catalog-2_29_2012.xml"} {:inv/sku "SKU-42" :inv/color :green :inv/size :large}]})
Explicit :db/txInstant
You can set :db/txInstant
explicitly, overriding Datomic's clock time.
When you do, you must choose a :db/txInstant
value that is not older
than any existing transaction, and not newer than the Datomic's clock
time. This capability enables initial imports of existing data that has
its own timestamps.
The transaction below might occur in an import of historical data, adding information about SKU-42 with a transaction time back in 2001. Note that for this transaction to work, everything already in the database, including e.g. the schema itself, must also have been backdated.
(d/transact conn {:tx-data [{:inv/sku "SKU-42" :inv/color :green} [:db/add "datomic.tx" :db/txInstant #inst "2001"]]})
Redundancy Elimination
A datom is redundant with the current value of the database if there is a matching datom that differs only by transaction id. If a transaction would produce redundant datoms, those datoms are filtered out, and do not appear a second time in either the indexes or the transaction log.
For example, imagine that SKU-42 does not exist and you transact the following:
(-> conn (d/transact {:tx-data [{:inv/sku "SKU-42" :inv/color :green}]}) :tx-data)
The resulting tx-data will contain three datoms: the two datoms about the new SKU-42 entity, and the transaction's :db/txInstant:
e | a | v |
---|---|---|
13194139533320 | 50 | #inst "2017-09-03T16:10:38.279-00:00" |
49460431063875660 | 64 | "SKU-42" |
49460431063875660 | 65 | 25235990880714817 |
If you immediately re-run the same transaction, the two datoms about SKU-42 (entity 49460431063875660 above) are now redundant, and will not be added to the database again.
e | a | v |
---|---|---|
13194139533322 | 50 | #inst "2017-09-03T16:11:43.332-00:00" |
Note that transactions are never entirely redundant, because the transaction's
:db/txInstant
is always a new fact.
Cancel
datomic.ion/cancel
cancels the current Datomic query or transaction, and throws an ex-info with an anomaly to the original caller.
cancel
requires a map with the key :cognitect.anomalies/category
, which has valid values of:
:cognitect.anomalies/incorrect
:cognitect.anomalies/conflict
When :cognitect.anomalies/message
is provided, the message will be used as the Exception's detail message.
All other keys should be namespace-qualified and all data passed to cancel must be transit-serializable.
The example below uses a transaction function to ensure that users always have a name
and email
.
The first transaction succeeds, but the second is canceled since :address
is passed instead of :email
.
;; Transaction function (defn add-user [db umap] (if (every? umap [:name :email]) [{:user/name (:name umap) :user/email (:email umap)}] (datomic.ion/cancel {:cognitect.anomalies/category :cognitect.anomalies/incorrect :cognitect.anomalies/message "User map must contain :email and :name"}))) (d/transact conn {:tx-data ['(datomic.sample.txfns/add-user {:name "Marshall" :email "test@test.com"})]})
=> ;; tx result map
(d/transact conn {:tx-data ['(datomic.sample.txfns/add-user {:name "Marshall" :address "test@test.com"})]})
=> Execution error (ExceptionInfo) at datomic.client.api.async/ares (async.clj:58).