Ions Reference

This reference covers everything you need to develop and user ions:

Requirements

Ion applications require the following:

Ion deps.edn

Ion applications are described by a standard Clojure deps.edn file. Add:

  • com.datomic/ion as a runtime dependency
  • com.datomic/ion-dev under a :dev alias
  • a "datomic-cloud" entry under your :mvn/repos
;; add the following to get the latest ion release
{:deps {com.datomic/ion {:mvn/version "0.9.16"}}
 :mvn/repos {"datomic-cloud" {:url "s3://datomic-releases-1fc2183a/maven/releases"}}
 :aliases
  {:dev {:extra-deps {com.datomic/ion-dev {:mvn/version "0.9.175"}}}}}

See ion-starter's deps.edn for an example.

Dev

Ion Function Signatures

An ion function is a plain function, with a signature based on its intended use:

ion typeinput argsreturn
transaction fndb + datatransaction data
query fndatadata
lambda:input JSON, :context mapString, InputStream, ByteBuffer, or File
web serviceweb requestweb response

To write an ion, implement a function with one of these signatures. A single ion application can have any number of ions of any types.

The datomic.ion.starter namespace has examples of each type of ion signature.

ion-config.edn

An ion application must have a resource named datomic/ion-config.edn on its classpath. This resource tells Datomic how to invoke your application:

  • the :app-name key names the CodeDeploy application. If you did not override this name when creating your Datomic compute group, it will default to your system name.
  • the :allow key whitelists ions that Datomic is allowed to call. When you deploy an application, Datomic will automatically require all the namespaces mentioned in the :allow section.
  • the :lambdas section configures lambda entry points for your application.

The example below is taken from the ion-starter project.

{:allow [;; transaction function
         datomic.ion.starter/inc-attr

         ;; query function
         datomic.ion.starter/modes

         ;; lambda handler
         datomic.ion.starter/echo

         ;; web application
         datomic.ion.starter/get-tutorial-schema    
         ]            
 :lambdas {:echo
           {:fn datomic.ion.starter/echo
            :description "Echos input"}
           :get-tutorial-schema
           {:fn datomic.ion.starter/get-tutorial-schema
            :description "returns the schema for the Datomic docs tutorial"}}
 :app-name "<YOUR-APP-HERE>"}

Developing Lambdas

Lambda Code

A lambda ion takes a map with two keys:

  • :input is a String containing the input JSON payload.
  • :context contains all the data fields of the AWS Lambda context as a Clojure map, with all keys (inluding nested and dynamic map keys) converted to keywords.

A lambda ion can return any of String, ByteBuffer, InputStream, or File; or it can throw an exception to signal an error.

For example, the following echo function simply echos back its invocation:

(defn echo
  [{:keys [context input]}]
  input)

Lambda Configuration

To configure a lambda, you must add the following entries in ion-config.edn:

  • add the lambda function name under the :allow key
  • add an entry under the lambdas key, where the key is the lambda name, and the value is a map describing the lambda.

A lambda configuration map supports the following keys

keyrequiredvalueexampledefault
:fnyessymboldatomic.ion.starter/echo
:descriptionnostring"echo"""
:timeout-secsnopositive int6060
:concurrency-limitnopositive int or :none2020
:integrationno:api-gateway/proxy:api-gateway/proxy

:concurrency-limit configures the concurrent executions of the lambda. Setting it to :none configures no reserved concurrent executions.

The :integration key customizes integration between the lambda and AWS. The only value currently supported is :api-gateway/proxy, which causes the AWS Lambda proxy to confrom to the special conventions for AWS Lambda proxies.

:server-type :ion

When you are developing ions locally, you will want to connect to a Datomic cloud system as a client. But when you deploy ions, you will want the same code to utilize an in-memory implement of client. The :ion server-type implements this behavior, connecting as a client when remote, and providing an in-memory implemention of client when running in Datomic Cloud.

The following example shows how to create a client for an ion application:

(require '[datomic.client.api :as d])

(def cfg {:server-type :ion
          :region "<your AWS Region>" ;; e.g. us-east-1
          :system "<system-name>"
          :query-group "<system-name>"
          :endpoint "http://entry.<system-name>.<region>.datomic.net:8182/"
          :proxy-port <local-port for SSH tunnel to bastion>})

(def client (d/client cfg))

Push

Push Arguments

The datomic.ion.dev main function takes the following arguments for push:

keywordrequiredvalueexample
:opyes:push:push
:unamenostring"janes-wip"
:creds-profilenostring"janes-profile"
:regionnostring"us-east-1"

You can specify a Named Profile by passing it as :creds-profile. You can override your environment's default region by providing :region.

Unreproducible Push

As a development convenience, Ions support an unreproducible push, allowing you to deploy work-in-progress code not committed to git. Since there is no git SHA, you must specify an unrepro name (:uname) when pushing. You are responsible for making the uname unique within your org+region+app.

It is ok to push to the same :uname repeatedly, so long as you understand that the last push wins.

Push Outputs

Push will ensure that git and maven libs exist under

s3://$(datomic-code-bucket)/datomic/libs

and will create a CodeDeploy application revision located at

s3://$(datomic-code-bucket)/datomic/apps/$(app-name)/$(git-sha).zip

or at

s3://$(datomic-code-bucket)/datomic/apps/$(app-name)/$(uname).zip

if a :uname is specified.

On success, push returns a map with the following keys:

keywordrequiredvalue
:revnogit SHA for the commit that was pushed
:unamenounreproducible name for the push
:deploy-groupsyeslist of available groups for deploy
:deploy-commandyessample command for deploy
:docnodocumentation
:dependency-conflictsnomap describing conflicts

Dependency Conflicts

Because ions run on the same classpath as Datomic Cloud, it is possible for ion dependencies to conflict with Datomic's own dependencies. If this happens:

  • Datomic's dependencies will be used
  • The return from push will warn you with a :dependency-conflicts map
  • You can add the :deps from the conflicts map to your local project so that you can test against the libraries used by Datomic Cloud.

The Datomic team works to keep Datomic's dependencies up-to-date. If you are unable to resolve a dependency conflict, please contact support.

Deploy

Once you have pushed an application, you can deploy it as many times as you want, possibly to different compute groups.

Deploy Arguments

The datomic.ion.dev main function takes the following arguments for deploy:

keywordrequiredvalueexample
:opyes:deploy:deploy
:groupyescompute group name"my-datomic-compute"
:revnooutput from push"6468765f843d70e01a7a2e483405c5fcc9aa0883"
:unamenoinput to push"janes-wip"
:creds-profilenostring"janes-profile"
:regionnostring"us-east-1"

The :group key takes compute group name as a value. If you followed the upgrade instructions and are running separate storage and compute stacks the name of your compute group is the name you gave the compute stack when you created it.

However, if you launched from AWS Marketplace, you are running a nested stack with an automatically generated compute stack name. To find the name of your compute stack, go to the AWS CloudFormation console and look through the list of Stack Names. The name of your compute stack will be $(SystemName)-Compute-$(GeneratedId). Use that whole stack name as the parameter to :group.

You can specify a Named Profile by passing it as :creds-profile. You can override your environment's default region by providing :region.

Deploy Outputs

When you deploy an ion application, Datomic will use an AWS Step Machine to

  • CodeDeploy the code for the application onto each instance in the compute group. This deployment is one-at-a-time, so the group will remain available with N-1 members active throughout the deployment.
  • Ensure that all the lambdas requested via the ion-config.edn map exist and are configured correctly.

Deploy runs a step function that deploys the application to CodeDeploy and ensures that the lambdas are correctly configured.

Deploy Status

You can monitor the deployment from the command line or from the AWS Console.

Deploy Status Arguments

keywordrequiredvalueexample
:opyes:deploy-status:deploy-status
:execution-arnyesoutput from deploy"arn:aws:states:us-east-1:123456789012:execution:datomic-compute:datomic-compute-1526506240469"
:creds-profilenostring"janes-profile"
:regionnostring"us-east-1"

You can specify a Named Profile by passing it as :creds-profile. You can override your environment's default region by providing :region.

Deploy Status Outputs

Deploy Status returns the status of the Code Deploy and the ion deploy as a map:

{:deploy-status "SUCCEEDED"
 :code-deploy-status "SUCCEEDED"}

See the Step Functions reference for possible values of the status keys.

You can monitor the deployment in near real-time from the Step Functions Console. Look for a state machine named "datomic-$(group)".

Invoking Lambdas

When you deploy an application with lambda ions, Datomic will create AWS Lambdas named:

$(group)-$(name)

where group is the :group key you used to invoke deploy, and name is the name under the :lambdas key in ion-config.edn.

Developing Web Services

Web Service Code

A web ion is a function that takes the following input map:

keyreq'dvalueexample
:bodynoan input stream"hello"
:headersyesmap string->string{"x-foo" "bar"}
:protocolyesHTTP protocol"HTTP/1.1"
:remote-addryescaller host"example.com"
:request-methodyesHTTP verb as keyword:get
:schemeno
:server-nameyesserver host name"example.com"
:server-portnoTCP port443
:uriyeshost name

and returns

keyreq'dvalueexample
:bodynostreamable"goodbye"
:headersyesmap string->string{"x-foo" "bar"}
:statusyesHTTP status code200

where the :body can be an InputStream, String, or File.

The input and output maps are mostly compatible with the Clojure Ring Spec, and many web applications should be able to run unmodified as web ions.

The ion-starter project includes an example web ion.

Web Service Configuration

Web services are exposed as lambda proxies through the AWS API Gateway. The ion library includes a helper function datomic.ion.lambda.api-gateway/ionize that will convert a web ion into a lambda ion, as demonstrated in the ion-starter project:

(def get-tutorial-schema
  "API Gateway web service ion for tutorial-schema-handler."
  (apigw/ionize tutorial-schema-handler))

The lambda ion is then configured like any other lambda. You can then use the API Gateway to:

  • deploy an API to the internet
  • manage content encodings and compression
  • validate requests
  • configure access control
  • issue and manage API keys

The ions tutorial includes an example that deploys a web ion to the internet.