Client API

API Docs

Synchronous API

Synchronous API functions have the following common semantics:

  • they block the calling thread if they access a remote resource
  • they return a value
  • they indicate anomalies by throwing an exception

The following example shows a simple transaction using the Synchronous API.

;; load the Synchronous API with prefix 'd'
(require '[datomic.client.api :as d])

;; transact a movie
(def result (d/transact conn {:db/id "goonies"
                              :movie/title "The Goonies"
                              :movie/genre "action/adventure"
                              :movie/release-year 1985}))

;; what id was assigned to The Goonies?
(-> result :tempids (get "goonies"))

Asynchronous API

The Asynchronous API version differs from the Synchronous API in that

  • they return a core.async channel if they access a remote resource
  • they never block the calling thead
  • they place result(s) on the channel
  • they place anomalies on the channel

You must use a channel-taking operator such as <!! to retrieve the result of an Asynchronous operation, and you must explicitly check for anomalies:

;; load the Asynchronous API with prefix 'd', plus core.async and anomaly APIs
(require '[clojure.core.async :refer (<!!)]
         '[cognitect.anomalies :as anom]
         '[datomic.client.api.async :as d])

;; transact a movie
(def result (<!! (d/transact conn {:db/id "goonies"
                                   :movie/title "The Goonies"
                                   :movie/genre "action/adventure"
                                   :movie/release-year 1985})))

;; what id was assigned to The Goonies?
(when-not (::anom/anomaly result)
  (-> result :tempids (get "goonies")))

Client Object

All use of the Client API begins by creating a client. The args map for creating a client expects the following keys:

KeyValue
:server-type:cloud
:regionAWS region
:systemyour system name
:query-groupyour system name
:endpointcreated by CloudFormation
:timeoutmsec timeout, optional, default 60000

Database Operations

The client object can be used to create, list, and delete databases.

Create and delete operations take effect immediately. After a database is deleted, a Primary Compute Node will asynchronously delete all resources associated with a database.

Connection

All use of a database is through a connection object. The connect API returns a connection, which is then passed as an argument to the transaction and query APIs.

Database values remember their association with a connection, so APIs that work against a single database (e.g. datoms) do not need to redundantly specify a connection argument.

Connections do not require any special handling on the client side: Connections are thread safe, do not need to be pooled, and do not need to be closed when not in use. Connections are cached automatically, so creating the same connection multiple times is inexpensive.

Offset and Limit

Client APIs that can return an arbitrary number of results allow you to request only part of these results by specifying the following optional keys in the args map:

KeyMeaningDefault
:offsetnumber of results to omit from the beginning0
:limitmaximum number of results to return1000

You can specify a :limit of -1 to request all results.

Offset and Limit Example

The full mbrainz example has over 600,000 names. The following query uses :offset and :limit to return 2 names starting with the 10th item in the query result.

;; query
{:query '[:find ?name
          :where [_ :artist/name ?name]]
          :args [db]
          :offset 10
          :limit 2}

;; result
[["Kenneth Ishak & The Freedom Machines"] 
 ["The Plastik"]]

Chunked Async Results

The sync API is designed for convenience and returns a single, fully-realized set of results. The async API provides more flexibility by placing the results on a channel one chunk at a time. You can control the chunk size via a key in the args map:

KeyMeaningDefaultMax
:chunkmax results in a single chunk100010000

Processing results by async transduction is the most efficient way to deal with large result sets.

Chunked Results Example

If you wanted to know the average length of an artist name in the mbrainz data set, you could use the following async transduction to process all 600,000+ names in chunks of 10,000.

(defn averager
  "Reducing fn that calculates average of its inputs"
  ([] [0 0])
  ([[n total]]
     (/ total (double n)))
  ([[n total] count]
     [(inc n) (+ total count)]))

(def first-counter
  "Transform for chunked query results that gets the count of
the first item from each result tuple."
  (comp (halt-when ::anom/category)
        cat
        (map first)
        (map count)))


(->> (d/q {:query '[:find ?name
                    :where [_ :artist/name ?name]]
           :chunk 10000
           :args [db]})
     (a/transduce first-counter averager (averager))
     <!!)

;; check that averager actually averages
(transduce identity averager (averager) [5 4])
=> 4.5

;; check that first-counter returns count of first element
;; in tuple
(into [] first-counter [[["Wilma" "Flintstone"]
                         ["Fred" "Flintsone"]]])
=> [5 4]

;; check that the reducing-fn and xform work together
(transduce first-counter averager (averager)
           [[["Wilma" "Flintstone"]
             ["Fred" "Flintsone"]]])
=> 4.5

The decomposition of async programs into named reducing fns and transforms such as averager and first-counter makes it easy to test the parts of an async program entirely synchronously, and without the need for mocking and stubbing:

;; check that first-counter returns counts-of-firsts
(into [] first-counter [[["Wilma" "Flintstone"]
                         ["Fred" "Flintsone"]]])
=> [5 4]

;; check that averager actually averages
(transduce identity averager (averager) [5 4])
=> 4.5

;; check that the reducing-fn and xform work together
(transduce first-counter averager (averager)
           [[["Wilma" "Flintstone"]
             ["Fred" "Flintsone"]]])
=> 4.5


Handling Errors

Errors are represented as an anomalies map. In the sync API, you can retrieve the specific anomaly as the ex-data of an exception. In the async API, the anomaly will be placed on the channel.

Sync API Error Example

The query below is incorrect because the :find clause uses a variable name ?nomen that is not bound by the query. The sync API will throw an exception:

(require '[datomic.client.api :as d])
(d/q '[:find ?nomen
       :where [_ :artist/name ?name]]
     db)
=> ExceptionInfo Query is referencing unbound variables: #{?nomen} 

You can discover the category of anomaly by inspecting the ex-data of the exception:

(ex-data *e)
=> {:cognitect.anomalies/category 
    :cognitect.anomalies/incorrect, 
    :cognitect.anomalies/message 
    "Query is referencing unbound variables: #{?nomen}", 
    ...}

Async API Error Example

With the Async API, the same anomaly appears as a map on channel:

(require '[datomic.client.api.async :as d])
(<!! (d/q {:query '[:find ?nomen
                    :where [_ :artist/name ?name]]
           :args [db]}))
=> {:cognitect.anomalies/category 
    :cognitect.anomalies/incorrect, 
    :cognitect.anomalies/message 
    "Query is referencing unbound variables: #{?nomen}", 
    ...}

Timeouts

All APIs that communicate with a remote process enforce a timeout that you can specify via the args map:

KeyMeaningDefault
:timeoutmax msec wait before giving up60000

The call to create a client takes a :timeout which establishes the default for all API calls through that client.

API timeouts cause an anomaly with category ::anom/unavailable, as shown below:

(d/q {:query '[:find (count ?name)
               :where [_ :artist/name ?name]]
      :args [db]
      :timeout 1})
(ex-data *e)
=> #:cognitect.anomalies{:category :cognitect.anomalies/unavailable, 
                         :message "Total timeout elapsed"}