Getting Started

This page will walk you through connecting to and using a Datomic Cloud Database.

Prerequisites

This tutorial requires:

Install Clojure

Install the AWS CLI

Install the AWS Command Line Interface. Make sure you have version 1.11.170 or later. If you are unfamiliar with managing Python environments, first try using the bundled AWS CLI installer. You can check your version using:

aws --version

NOTE: Running the datomic-socks-proxy script with an earlier version of the AWS CLI will result in a generic error message and failure of the script.

If you require multiple versions of the AWS CLI to be installed, then please investigate using Python's virtual environments, or the third-party virtualenv solution.

Configure AWS Access Keys

If you have not done so already:

  1. Create access keys for your Datomic user
  2. Add access keys to your environment

Datomic supports the use of AWS profiles as a credentials source. Additional information about IAM Users and access keys can be found in the AWS Security Credentials documentation .

Start a SOCKS proxy

In this step you will start a SOCKS proxy from your local dev machine to a bastion host that is running in AWS. If you are developing from an EC2 instance inside the same VPC as Datomic, you can skip this step.

To start the proxy

  • Download datomic-socks-proxy
    • In macOS the file may be downloaded as datomic-socks-proxy.dms. This is normal. Rename it to datomic-socks-proxy to continue.
  • run the script with bash datomic-socks-proxy <system-name>

Additional details on the SOCKS proxy script are provided in the Operations Documentation.

If you don't know your system name, the names of all running Datomic systems in your AWS account can be found with the following AWS CLI command (you can add --profile <profile-name> to run in the context of an AWS Credentials Profile):

aws ec2 describe-instances --filters "Name=tag-key,Values=datomic:tx-group" "Name=instance-state-name,Values=running" --query 'Reservations[*].Instances[*].[Tags[?Key==`datomic:system`].Value]' --output text

Depending on your local system settings, you may be prompted by SSH with:

The authenticity of host <bastion-server-address> can't be established.
RSA key fingerprint is <an SSH key>.
Are you sure you want to continue connecting (yes/no)?

Enter yes to continue establishing the proxy.

NOTE The datomic-socks-proxy will be launched in the foreground. If you need to kill it for some reason, create a new tab then use the command pkill datomic-socks-proxy or if pkill is not available on your system then utilize your system's process management to kill the process.

If you wish to launch the process in the background then use nohup bash datomic-socks-proxy <system-name> &. The command nohup will launch the process in the background and all output will be sent to the file named "nohup.out". If the job exits unexpectedly, you can type cat nohup.out in the directory that you ran the process to see the error message. The nohup.out file can also be read with your favorite text editor.

Test Bastion Connection

See the Arcitecture page for a complete picture of how the SOCKS proxy reaches Datomic

Run the following command to test your system's ability to reach Datomic through the SOCKS proxy, replacing the environment variables with the same values you used when launching the SOCKS proxy:

export DATOMIC_SYSTEM=my-system
export DATOMIC_REGION=us-east-1
export DATOMIC_SOCKS_PORT=8182

curl -x socks5h://localhost:$DATOMIC_SOCKS_PORT http://entry.$DATOMIC_SYSTEM.$DATOMIC_REGION.datomic.net:8182/

It should return:

{:s3-auth-path <bucket-name>}

An error response indicates that your Datomic system is not available at the specified port and address, or that your proxy port is incorrect. Verify that your Primary Stack and your SOCKS proxy are running and correctly configured.

Additional common issues with proxy configuration are discussed in the troubleshooting documentation.

Congratulations! You have successfully configured a SOCKS proxy through the bastion server to allow local access to your Datomic system. Now you can begin to use Datomic.

Add Client Library Dependency

The Datomic Client library is provided via Maven Central.

Clojure CLI

To use the Client library from a Clojure CLI REPL, add the following to your deps.edn dependencies map:

com.datomic/client-cloud {:mvn/version "0.8.71"}

Maven

To retrieve the Client library for a Maven project, add the following snippet inside the <dependencies> block of your pom.xml file:

<dependency>
 <groupId>com.datomic</groupId>
 <artifactId>client-cloud</artifactId>
 <version>0.8.71</version>
</dependency>

Leiningen

To include the Client library in a Leiningen project, add the following snippet to your project.clj file:

;; in collection under :dependencies key
[com.datomic/client-cloud "0.8.71"]

Make sure that Clojure dependency is set to at least [org.clojure/clojure "1.9.0"].

Connect and Use Datomic

Create a Database

To interact with Datomic Cloud, you must first specify a system connection, with:

  • :endpoint is the URI where your system can be reached, "http://entry.<system-name>.<region>.datomic.net:8182" where <system-name> is the same as the Stack Name you used to start the CloudFormation Storage Stack. This endpoint is labeled as EndpointAddress in the Cloudformation Output of the compute stack.
  • :region is the AWS region in which you've started Datomic Cloud.
  • :system is the SystemName

First we'll create a client with datomic.client.api/client. We'll use this client as we explore the Datomic below.

In your REPL, execute:

(require '[datomic.client.api :as d])

(def cfg {:server-type :ion
          :region "<your AWS Region>" ;; e.g. us-east-1
          :system "<system-name>"
          :creds-profile "<your_aws_profile_if_not_using_the_default>"
          :endpoint "http://entry.<system-name>.<region>.datomic.net:8182/"
          :proxy-port 8182})

(def client (d/client cfg))

You may need to use a local SSH tunnel port. Adjust the :proxy-port value to your local tunneling port if necessary.

Warnings may occur. Do not be alarmed as they will not affect functionality during this tutorial.

Given a functioning system, you can use the admin API to create a new database with datomic.client.api/create-database.

(d/create-database client {:db-name "movies"})

Now you're ready to connect to your newly created database using dataomic.client.api/connect.

(def conn (d/connect client {:db-name "movies"}))

The next step will be to define some schema for your new database.

Schema defines the set of possible attributes that can be associated with an entity. We'll need to provide 3 attributes: db/ident, db/valueType and db/cardinality. db/doc will also be provided for documentation.

Transact Schema

Now we need to create a schema.

Define the following small schema for a database about movies:

(def movie-schema [{:db/ident :movie/title
                    :db/valueType :db.type/string
                    :db/cardinality :db.cardinality/one
                    :db/doc "The title of the movie"}

                   {:db/ident :movie/genre
                    :db/valueType :db.type/string
                    :db/cardinality :db.cardinality/one
                    :db/doc "The genre of the movie"}

                   {:db/ident :movie/release-year
                    :db/valueType :db.type/long
                    :db/cardinality :db.cardinality/one
                    :db/doc "The year the movie was released in theaters"}])

Now transact the schema using datomic.client.api/transact.

(d/transact conn {:tx-data movie-schema})

You should get back a response like the following:

{:db-before ..., 
 :db-after ..., 
 :tx-data [...], ;; data added to the database
 :tempids {}}

Transact Data

Now you can define some movies to add to the database utilizing the schema we defined earlier:

(def first-movies [{:movie/title "The Goonies"
                    :movie/genre "action/adventure"
                    :movie/release-year 1985}
                   {:movie/title "Commando"
                    :movie/genre "thriller/action"
                    :movie/release-year 1985}
                   {:movie/title "Repo Man"
                    :movie/genre "punk dystopia"
                    :movie/release-year 1984}])

Transact the movies into the database:

(d/transact conn {:tx-data first-movies})

You should see a response similar to:

=> {:db-before ... 
    :db-after ...
    :tx-data [...], 
    :tempids {}}

Query

Get a current value for the database with datomic.client.api/db:

(def db (d/db conn))  

Now create a query for all movie titles:

(def all-titles-q '[:find ?movie-title 
                    :where [_ :movie/title ?movie-title]])

And execute the query with the value of the database using datomic.client.ap/q:

(d/q all-titles-q db)

You should see:

=> [["Commando"] ["The Goonies"] ["Repo Man"]]

Delete a Database

When you are done with this tutorial, you can use datomic.client.api/delete-database to delete the movies database:

(d/delete-database client {:db-name "movies"})

Troubleshooting

If you encounter issues connecting to or using your system, review the common client errors described in the troubleshooting documentation.

Next Steps

You have successfully connected to and used Datomic. For a more in depth exploration, see the Tutorial.

The Client API Documentation provides more details on the Datomic Client API itself.