Using temporality

Overview

Fauna documents are immutable, which means that once a document is created it never changes. If you update a document Fauna creates a new version of the document reflecting the changes and the original is left intact. With Fauna’s temporality features, it is possible to see the change history of a document, retrieve older versions, and perform queries based on document timestamps.

The history_days attribute

How long Fauna retains old versions of updated is determined at the collection level with the history_days attribute. history_days defaults to 30, but it is possible to specify a different value when you create a collection:

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.CreateCollection({ name: 'temporal' })
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  ref: Collection("temporal"),
  ts: 1649181192120000,
  history_days: 30,
  name: 'temporal'
}
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn:  52

  •   bytesOut: 148

  • computeOps:   1

  •    readOps:   0

  •   writeOps:   1

  •  readBytes: 810

  • writeBytes: 334

  •  queryTime: 7ms

  •    retries:   0

It is also possible to update the history_days attribute of an existing collection:

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.Update(q.Collection('fruit'), { history_days: 5 })
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  ref: Collection("fruit"),
  ts: 1641495115580000,
  history_days: 5,
  name: "fruit"
}
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn:   72

  •   bytesOut:  141

  • computeOps:    1

  •    readOps:    0

  •   writeOps:    1

  •  readBytes:  129

  • writeBytes:   95

  •  queryTime: 18ms

  •    retries:    0

After the period of time specified by history_days expires, Fauna removes any version of a document which has aged out. The most recent version of a document is not affected.

Documents may also include a ttl (time to live) attribute. Documents which have aged out of the database due to a ttl expiration cannot be retrieved with temporal queries. See Create for more information about the ttl attribute.

Document removal is handled by a background task, so once a document "expires" due to the history_days setting on a collection or the ttl field on a document, it could be some time (hours or days) before the removal occurs. There is no guarantee that removal actually occurs.

Document history

You can see a document’s change history with the Events function. To illustrate this feature, let’s create a new document in the fruit collection:

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.Create(
    q.Ref(q.Collection('temporal'), '1'),
    {
      data: {
        test: 'temporality',
      },
    }
  )
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  ref: Ref(Collection("temporal"), "1"),
  ts: 1648840223890000,
  data: { test: 'temporality' }
}
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn:  116

  •   bytesOut:  175

  • computeOps:    1

  •    readOps:    0

  •   writeOps:    1

  •  readBytes:   14

  • writeBytes:  202

  •  queryTime: 12ms

  •    retries:    0

Now let’s update the document:

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.Update(
    q.Ref(q.Collection('fruit'), '1'),
    {
      data: {
        quantity: 50,
      },
    },
  )
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
    ref: Ref(Collection("fruit"), "1"),
    ts: 1641328117890000,
    data: { type: "apple", color: ["red", "green"], quantity: 50 }
}
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn:  106

  •   bytesOut:  204

  • computeOps:    1

  •    readOps:    0

  •   writeOps:    1

  •  readBytes:  351

  • writeBytes:  116

  •  queryTime: 27ms

  •    retries:    0

Now we can use Events to see the change history. The following example uses Events with the Paginate function to list all the document’s change events.

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.Paginate(q.Events(q.Ref(q.Collection('fruit'), '1')))
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  data: [
    {
      ts: 1644607325240000,
      action: 'create',
      document: Ref(Collection("fruit"), "1"),
      data: { type: 'apple', color: [ 'red', 'green' ], quantity: 15 }
    },
    {
      ts: 1644607325510000,
      action: 'update',
      document: Ref(Collection("fruit"), "1"),
      data: { quantity: 50 }
    }
  ]
}
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn:  63

  •   bytesOut: 971

  • computeOps:   1

  •    readOps:   1

  •   writeOps:   0

  •  readBytes: 453

  • writeBytes:   0

  •  queryTime: 5ms

  •    retries:   0

You can retrieve a specific version of a document by including its timestamp with the Get function:

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.Get(q.Ref(q.Collection('fruit'), '1'), 1641838482800000)
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  ref: Ref(Collection("fruit"), "1"),
  ts: 1641838482800000,
  data: {
    type: "apple",
    color: ["red", "green"],
    quantity: 50
  }
}
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn:  69

  •   bytesOut: 204

  • computeOps:   1

  •    readOps:   1

  •   writeOps:   0

  •  readBytes:  88

  • writeBytes:   0

  •  queryTime: 4ms

  •    retries:   0

Timestamp queries

You can use the At function to retrieve data from a specified point in time. To see this feature in action, let’s add a few more documents to our collection.

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.Foreach(
    [
      ['2', 'mango', ['green', 'yellow'], 20],
      ['3', 'melon', ['green'], 100],
      ['4', 'pear', ['yellow', 'brown'], 60],
    ],
    q.Lambda(
      'new_fruit',
      q.Create(
        q.Ref(q.Collection('fruit'), q.Select(0, q.Var('new_fruit'))),
        {
          data: {
            type: q.Select(1, q.Var('new_fruit')),
            color: q.Select(2, q.Var('new_fruit')),
            quantity: q.Select(3, q.Var('new_fruit')),
          },
        }
      )
    )
  )
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
[
  [ '2', 'mango', [ 'green', 'yellow' ], 20 ],
  [ '3', 'melon', [ 'green' ], 100 ],
  [ '4', 'pear', [ 'yellow', 'brown' ], 60 ]
]
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn:  430

  •   bytesOut:  127

  • computeOps:    1

  •    readOps:    0

  •   writeOps:    0

  •  readBytes:    0

  • writeBytes:    0

  •  queryTime: 11ms

  •    retries:    0

These new documents all have the same timestamp, because they were all added as part of the same transaction.

Now that we have several documents in our collection, let’s create an index so we can easily retrieve them all at once.

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.CreateIndex({ name: 'all_fruit', source: q.Collection('fruit') })
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  ref: Index("all_fruit"),
  ts: 1641593136380000,
  active: true,
  serialized: true,
  name: 'all_fruit',
  source: Collection("fruit"),
  partitions: 8
}
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn:    80

  •   bytesOut:   251

  • computeOps:     1

  •    readOps:     0

  •   writeOps:     5

  •  readBytes: 2,279

  • writeBytes: 1,681

  •  queryTime:  48ms

  •    retries:     0

This index has no terms, so it includes all the documents in the fruit collection.

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.Map(
    q.Paginate(
      q.Match(q.Index('all_fruit'))
    ),
    q.Lambda('X', q.Get(q.Var('X')))
  )
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  data: [
    {
      ref: Ref(Collection("fruit"), "1"),
      ts: 1641578425510000,
      data: { type: 'apple', color: [ 'red', 'green' ], quantity: 50 }
    },
    {
      ref: Ref(Collection("fruit"), "2"),
      ts: 1641592193100000,
      data: { type: 'mango', color: [ 'green', 'yellow' ], quantity: 20 }
    },
    {
      ref: Ref(Collection("fruit"), "3"),
      ts: 1641592193100000,
      data: { type: 'melon', color: [ 'green' ], quantity: 100 }
    },
    {
      ref: Ref(Collection("fruit"), "4"),
      ts: 1641592193100000,
      data: { type: 'pear', color: [ 'yellow', 'brown' ], quantity: 60 }
    },
  ]
}
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn:  118

  •   bytesOut:  607

  • computeOps:    1

  •    readOps:   11

  •   writeOps:    0

  •  readBytes:  909

  • writeBytes:    0

  •  queryTime: 11ms

  •    retries:    0

To query a database at a particular point in time, you can use the At function. At adds a condition to a query which matches only documents which were created at or before a specified timestamp.

The C# version of this example is not currently available.
The Go version of this example is not currently available.
The Java version of this example is not currently available.
client.query(
  q.At(
    1641838482800000,
    q.Map(
      q.Paginate(
        q.Match(q.Index('all_fruit'))
      ),
      q.Lambda('X', q.Get(q.Var('X')))
    )
  )
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  data: [
    {
      ref: Ref(Collection("fruit"), "1"),
      ts: 1641578425510000,
      data: { type: 'apple', color: [ 'red', 'green' ], quantity: 50 }
    }
  ]
}
The Python version of this example is not currently available.
The Shell version of this example is not currently available.
Query metrics:
  •    bytesIn: 149

  •   bytesOut: 215

  • computeOps:   1

  •    readOps:   9

  •   writeOps:   0

  •  readBytes: 434

  • writeBytes:   0

  •  queryTime: 9ms

  •    retries:   0

Any documents which have been removed from the database due to either history_days or ttl expiration cannot be retrieved with temporal queries.

Conclusion

Temporality offers developers a rich set of features for working with versioned documents and querying collections based on document timestamps. See the following resources for more information about temporality:

Was this article helpful? 

We're sorry to hear that.
Tell us how we can improve!
Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!