Schema

The facts that a Datomic database stores are represented by datoms. Each datom is an addition or retraction of a relation between an entity, an attribute, a value, and a transaction. The set of possible attributes a datom can specify is defined by a database's schema.

Each Datomic database has a schema that describes the set of attributes that can be associated with entities. A schema only defines the characteristics of the attributes themselves. It does not define which attributes can be associated with which entities. Decisions about which attributes apply to which entities are made by an application.

This gives applications a great degree of freedom to evolve over time. For example, an application that wants to model a person as an entity does not have to decide up front whether the person is an employee or a customer. It can associate a combination of attributes describing customers and attributes describing employees with the same entity. An application can determine whether an entity represents a particular abstraction, customer or employee, simply by looking for the presence of the appropriate attributes.

Attributes

Schema attributes are defined using the same data model used for application data. That is, attributes are themselves entities with associated attributes. Datomic defines a set of built-in system attributes that are used to define new attributes.

Required schema attributes

Every new attribute is described by three required attributes:

  • :db/ident specifies the unique name of an attribute. It's value is a namespaced keyword with the lexical form :<namespace>/<name>. It is possible to define a name without a namespace, as in :<name>, but a namespace is preferred in order to avoid naming collisions. Namespaces can be hierarchical, with segments separated by ".", as in :<namespace>.<nested-namespace>/<name>. The :db namespace is reserved for use by Datomic itself.

  • :db/valueType specifies the type of value that can be associated with an attribute. The type is expressed as a keyword. Allowable values are listed below.

    :db.type/keyword - Value type for keywords. Keywords are used as names, and are interned for efficiency. Keywords map to the native interned-name type in languages that support them.

    :db.type/string - Value type for strings.

    :db.type/boolean - Boolean value type.

    :db.type/long - Fixed integer value type. Same semantics as a Java long: 64 bits wide, two's complement binary representation.

    :db.type/bigint - Value type for arbitrary precision integers. Maps to java.math.BigInteger on Java platforms.

    :db.type/float - Floating point value type. Same semantics as a Java float: single-precision 32-bit IEEE 754 floating point.

    :db.type/double - Floating point value type. Same semantics as a Java double: double-precision 64-bit IEEE 754 floating point.

    :db.type/bigdec - Value type for arbitrary precision floating point numbers. Maps to java.math.BigDecimal on Java platforms.

    :db.type/ref - Value type for references. All references from one entity to another are through attributes with this value type.

    :db.type/instant - Value type for instants in time. Stored internally as a number of milliseconds since midnight, January 1, 1970 UTC. Maps to java.util.Date on Java platforms.

    :db.type/uuid - Value type for UUIDs. Maps to java.util.UUID on Java platforms.

    :db.type/uri - Value type for URIs. Maps to java.net.URI on Java platforms.

    :db.type/bytes - Value type for small binary data. Maps to byte array on Java platforms.

  • :db/cardinality specifies whether an attribute associates a single value or a set of values with an entity. The values allowed for :db/cardinality are:

    :db.cardinality/one - the attribute is single valued, it associates a single value with an entity

    :db.cardinality/many - the attribute is multi valued, it associates a set of values with an entity

    Transactions can add or retract individual values for multi-valued attributes.

Optional schema attributes

In addition to these three required attributes, there are several optional attributes that can be associated with an attribute definitions:

  • :db/doc specifies a documentation string.
  • :db/unique - specifies a uniqueness constraint for the values of an attribute. Setting an attribute :db/unique also implies :db/index. The values allowed for :db/unique are:

    :db.unique/value - the attribute value is unique to each entity; attempts to insert a duplicate value for a different entity id will fail

    :db.unique/identity - the attribute value is unique to each entity and "upsert" is enabled; attempts to insert a duplicate value for a temporary entity id will cause all attributes associated with that temporary id to be merged with the entity already in the database.

    :db/unique defaults to nil.

  • :db/index specifies a boolean value indicating that an index should be generated for this attribute. Defaults to false.
  • :db/fulltext specifies a boolean value indicating that an eventually consistent fulltext search index should be generated for the attribute. Defaults to false.

  • :db/isComponent specifies that an attribute whose type is :db.type/ref refers to a subcomponent of the entity to which the attribute is applied. When you retract an entity with :db.fn/retractEntity, all subcomponents are also retracted. When you touch an entity, all its subcomponent entities are touched recursively. Defaults to nil.

  • :db/noHistory specifies a boolean value indicating whether past values of an attribute should not be retained. Defaults to false.

    The purpose of :db/noHistory is to conserve storage, not to make semantic guarantees about removing information. The effect of :db/noHistory happens in the background, and some amount of history may be visible even for attributes with :db/noHistory set to true.

External keys

All entities in a database have an internal key, the entity id. You can use :db/unique and :db/index together to define an attribute to represent an external key.

An entity may have any number of external keys.

External keys must be single attributes, multi-attribute keys are not supported.

Installing an attribute definition

Attribute definitions are represented as data structures (see Data Structures) that can be submitted to a database as part of a transaction. It is idiomatic to use the transaction map data structure, as shown in the following example. It defines an attribute named :person/name of type :db.type/string with :db.cardinality/one that is intended to capture a person's name.

{:db/id #db/id[:db.part/db]
 :db/ident :person/name
 :db/valueType :db.type/string
 :db/cardinality :db.cardinality/one
 :db/doc "A person's name"
 :db.install/_attribute :db.part/db}

Every attribute definition must include :db/id, :db/ident, :db/valueType, and :db/cardinality. In addition, every attribute must be installed, by creating a :db.install attribute reference from :db.part/db to the new attribute id. The example above takes advantage of reverse attribute navigation: the underscore in the name :db.install/_attribute reverses the direction of the attribute reference, creating the needed reference from :db.part/db to the new attribute.

Attribute installation is atomic. All the required information for an attribute must be specified in a single transaction, after which the attribute will be immediately available for use.

Partitions

All entities created in a database reside within a partition. Partitions group data together, providing locality of reference when executing queries across a collection of entities. In general, you want to group entities based on how you'll use them. Entities you'll often query across - like the community-related entities in our sample data - should be in the same partition to increase query performance. Different logical groups of entities should be in different partitions. Partitions are discussed in more detail in the Indexes topic.

There are three partitions built into Datomic.

PartitionPurpose
:db.part/dbSystem partition, used for schema
:db.part/txTransaction partition
:db.part/userUser partition, for application entities

The attributes you create for your schema must live in the :db.part/db partition.

The :db.part/tx partition is used only for transaction entities, which are created automatically by transactions.

You can use :db.part/user for your own entities, but this is only appropriate for experimentation.

Your schema should create one or more partitions appropriate for your application.

Creating new partitions

A partition is represented by an entity with a :db/ident attribute, as shown below.

{:db/id #db/id[:db.part/db]
 :db/ident :communities}

As with attribute definitions, there are two steps to installing a new partition in a database. The first is to create the partition entity. The second is to associate the new partition entity with the built-in system partition entity, :db.part/db. Specifically, the new partition must be added as a value of the :db.part/db entity's :db.install/partition attribute. This linkage is what tells the database that the new entity is actually a partition definition.

Here is a complete transaction for installing the :communities partition. It includes the map that defines the partition, followed by the statement adding it as a value of :db.part/db's :db.install/partition attribute.

[{:db/id #db/id[:db.part/db -1]
  :db/ident :communities}
 [:db/add :db.part/db :db.install/partition #db/id[:db.part/db -1]]]

Here's the brief version using the "reverse reference" mechanism (see Installing an attribute definition discussion above).

[{:db/id #db/id[:db.part/db],
  :db/ident :communities,
  :db.install/_partition :db.part/db}]

Enums

Many databases need to represent a set of enumerated values. In Datomic, it's idiomatic to represent enumerated values as entities with a :db/ident attribute.

For example, this transaction defines a :person/gender attribute, and two enumerated values, :person.gender/female and :person.gender/male.

[{:db/id #db/id[:db.part/db -1]
  :db/ident :person/gender
  :db/valueType :db.type/ref
  :db/cardinality :db.cardinality/one
  :db/doc "A person's gender"
  :db.install/_attribute :db.part/db}

 {:db/id #db/id[:db.part/user -2]
  :db/ident :person.gender/female}

 {:db/id #db/id[:db.part/user -3]
  :db/ident :person.gender/male}]

The use of :db/ident on the enumerated value entities makes it possible to refer to the entities using their identity keywords, :person.gender/female and :person.gender/male, respectively.

This transaction creates a person named "Bob" with the gender :person.gender/male.

[{:db/id #db/id[:db.part/user]
  :person/name "Bob"
  :person/gender :person.gender/male}]

The :person/gender attribute is of type :db.type/ref, so its value must be a reference to another entity. The :person.gender/male keyword can be used as a value because it is the identity of an entity, as specified by :db/ident in the previous transaction.

Schema limits

Maximum number of schema elements

Datomic enforces the total number of schema elements to be fewer than 2^20, which includes schema attributes, value types and partitions. Attempts to create more than 2^20 schema elements will fail.

Namespace restrictions

The :db namespace, and all :db.* namespaces, are reserved for use by Datomic. With the exception of :db/doc and :db/ident, you should not use Datomic attributes on your own entities.

The Seattle sample schema

The Seattle sample in the Datomic package includes a complete schema definition, samples/seattle/seattle-schema.edn, which is described in detail in the tutorial.

Schema Alteration

Because Datomic maintains a single set of physical indexes, and supports query across time, a db value utilizes the single schema associated with its basis, i.e. before any call to asOf/since/history, for all views of that db (including asOf/since/history). Thus traveling back in time does not take the working schema back in time, as the infrastructure to support it may no longer exist. Many alterations are backwards compatible - any nuances are detailed separately below.

Datomic provides two categories of alterations you can make to existing schemas. The first is renaming a :db/ident. Idents provide names for things - specifically schema attributes and enumerated values. Both types of names can be altered via a schema alteration.

The second category of alteration is altering schema attributes of attributes. The supported attributes to alter include :db/cardinality, :db/isComponent, :db/noHistory, :db/index and :db/unique. You cannot alter :db/valueType or :db/fulltext.

Backup Before Altering Schema

Schema alterations are not irrevocable, but they can be difficult to undo. For example, if you drop a unique constraint and then add non-unique data, you will have to find and retract the non-unique data before re-enabling the unique constraint.

Given the difficulty of fixing schema mistakes, you are encouraged to backup a database before making schema changes.

Renaming an Identity

To rename a :db/ident, submit a transaction with the :db/id and the value of the new :db/ident. This alteration will take place synchronously, and will be immediately visible in the database returned by the transaction.

Both the new ident and the old ident will refer to the entity. entid will return the same entity id for both the new and the old ident. ident will return the new ident, so there is asymmetry between what entid returns and what ident returns for the old ident. :db/ident in the entity map, returned from entity, will be the new ident.

This example alters the :db/ident of :person/name to be :person/full-name.

[{:db/id :person/name
  :db/ident :person/full-name}]

We don't recommend repurposing a :db/ident, but if you find you need to use the name for a different purpose, you can install the name again as described in the Installing an attribute definition section. This repurposed :db/ident will cease to point to the entity it was previously pointing to and ident will return the newly installed entity.

Altering Schema Attributes

When you want to alter existing schema attributes, you can do so using the :db.alter/attribute attribute. Altering an attribute is similar to installing an attribute: you submit a transaction with the new facts, including the :db/id and the new value(s) of the attribute(s) and associate them with the built-in database partition, using :db.alter/attribute. You can also retract optional schema attributes with :db.alter/attribute.

All alterations happen synchronously, except for adding an AVET index. If you want to know when the AVET index is available, call syncSchema. In order to add :db/unique, you must first have an AVET index including that attribute.

The possible alterations are listed below, with more details about each one in the sections following the table.

FromToEffect on Attribute
:db/cardinality :db.cardinality/one:db/cardinality :db.cardinality/manyAllow multiple values per entity
:db/cardinality :db.cardinality/many:db/cardinality :db.cardinality/oneLimit to single value per entity
:db/isComponent true:db/isComponent false or unsetDo not treat as a component
:db/isComponent false or unset:db/isComponent trueTreat as a component
:db/noHistory true:db/noHistory false or unsetKeep history
:db/noHistory false or unset:db/noHistory trueDo not keep history
:db/index false or unset, :db/unique unset:db/index trueKeep AVET index, do not enforce unique
:db/index true, :db/unique unset:db/index false or unsetDrop AVET index, do not enforce unique
:db/index true, :db/unique unset:db/unique setKeep AVET index, enforce unique
:db/index false or unset, :db/unique unset:db/unique setKeep AVET index, enforce unique
:db/index false or unset, :db/unique set:db/unique unsetDrop AVET index, stop enforcing unique
:db/index true, :db/unique set:db/unique unsetKeep AVET index, stop enforcing unique
:db/unique :db.unique/identity:db/unique :db.unique/valueKeep AVET index, enforce unique value
:db/unique :db.unique/value:db/unique :db.unique/identityKeep AVET index, enforce unique identity

Altering Cardinality

When altering from a multivalued attribute to a single valued attribute there must be at most a single value for every entity in the set of current assertions, otherwise the alteration will not be accepted and the transaction will fail.

After you alter the cardinality of an attribute, entity lookups will return values specified by the new cardinality - either a single value in the case of :db.cardinality/one or a set of values in the case of :db.cardinality/many. This includes queries against historical databases. An entity from an asOf, since or history database that has an attribute with multiple values will return a single one of those values if the schema attribute has been altered to be single-valued.

The example below alters the cardinality of :person/favorite-food to many and the cardinality of :person/email to single.

[{:db/id :person/favorite-food
  :db/cardinality :db.cardinality/many
  :db.alter/_attribute :db.part/db}
 {:db/id :person/email
  :db/cardinality :db.cardinality/one
  :db.alter/_attribute :db.part/db]

Altering Component Attribute

Altering a component attribute will cause Datomic to start enforcing the semantics for the new component value of the attribute. For example, when you set :db/isComponent to true, values of that attribute will be included recursively by Entity.touch.

The example below alters :order/line-items to be a component, and :customer/orders to be a component.

[{:db/id :order/line-items
  :db/isComponent true
  :db.alter/_attribute :db.part/db}]
[{:db/id :customer/orders
  :db/isComponent false
  :db.alter/_attribute :db.part/db}]

You can also revoke component status by retracting the :db/isComponent attribute. The example below revokes component status for :customer/orders via retraction.

[[:db/retract :customer/orders :db/isComponent true]
 [:db/add :db.part/db :db.alter/attribute :customer/orders]]

Altering noHistory Attribute

Altering an attribute's :db/noHistory to false or retracting it will cause Datomic to start retaining history on that attribute. Setting :db/noHistory to true will cause Datomic to stop retaining history. Note that :db/noHistory controls the operation of future indexing jobs, and does nothing to current historical values. See excision if you need to excise history.

The example below alters :person/encrypted-password to stop retaining history forward, and alters :person/address to start retaining history.

[{:db/id :person/encrypted-password
  :db/noHistory true
  :db.alter/_attribute :db.part/db}]
[{:db/id :person/address
  :db/noHistory false
  :db.alter/_attribute :db.part/db}]

The example below uses retraction to stop retaining history for :person/address.

[[:db/retract :person/address :db/noHistory false]
 [:db/add :db.part/db :db.alter/attribute :person/address]]

Altering AVET index and uniqueness attributes

The effects of the :db/index and :db/unique attributes are correlated, so the two will be described together.

Removing an AVET index from an attribute

If :db/index is false or unset and :db/unique is unset, Datomic will not keep an AVET index.

The example below will retract the unique identity constraint and drop the AVET index for :person/external-id.

[[:db/retract :person/external-id :db/unique :db.unique/identity]
 [:db/retract :person/external-id :db/index true]
 [:db/add :db.part/db :db.alter/attribute :person/external-id]]

Adding an AVET index to an attribute

If :db/index is true or :db/unique is set, Datomic will keep an AVET index. When you alter an existing attribute to have an AVET index, the AVET index may not be available immediately. To find out when the AVET index is available, call syncSchema.

The example below will keep an AVET index for :person/external-id.

[{:db/id :person/external-id
  :db/index true
  :db.alter/_attribute :db.part/db}]

Altering an attribute to be unique

If :db/unique is set, Datomic will enforce a uniqueness constraint on the attribute.

In order to add a unique constraint to an attribute, Datomic must already be maintaining an AVET index on the attribute, or the attribute must have never had any values asserted. Furthemore, if there are values present for that attribute, they must be unique in the set of current assertions. If either of these constraints are not met, the alteration will not be accepted and the transaction will fail.

Adding a unique constraint does not change history, therefore historical databases may contain non-unique values. Code that expects to find a unique value may find multiple values when querying against history.

The example below will alter the attribute :person/external-id to be a unique identity.

[{:db/id :person/external-id
  :db/unique :db.unique/identity
  :db.alter/_attribute :db.part/db}]

Removing a uniqueness constraint

If db:unique is unset, Datomic will stop enforcing the uniqueness constraint.

The example below will retract the unique value constraint on :person/external-id.

[[:db/retract :person/external-id :db/unique :db.unique/value]
 [:db/add :db.part/db :db.alter/attribute :person/external-id]]