Event streaming

Streaming is supported only in FQL V4 at this time and not yet supported in FQL v10.

Streaming is a feature where client code can subscribe to event notifications. A subscription is an open connection to Fauna and change notification messages are sent over the connection as they occur.

Avoid running a query to fetch a document followed by establishing a stream. Multiple events might have modified the document before stream startup, which can lead to inaccurate document data representation in your application.

Streaming mode comparison

There are two kinds of event streaming:

  • Document streaming

    Client code subscribes to a document reference. When the referenced document is created, updated, or deleted, an event notification is sent to the subscriber.

  • Set streaming

    Client code subscribes to a set reference. For example:

    • Documents(Collection("spells")) is the set of documents in the spells collection.

    • Match(Index("elements_of_spells")) is the set of documents in the term-less index elements_of_spells.

    • Match(Index("spells_by_element"), "fire") is the set of spells documents whose element field matches fire.

    When one or more documents enter or exit the set by creation, deletion, or field value change for a set defined by an index terms or values field, an event notification is sent to the subscriber.

Streaming mode sequence comparison

Polling sequence:

A sequence diagram demonstrating the communications during database polling

Streaming sequence:

A sequence diagram demonstrating the communications during database streaming

The diagrams show that with polling the client executes more queries to discover when a document is updated. For a streaming client, the subscription happens once and events are broadcast to the client when the subscribed document is updated.

Streaming might be a better alternative than polling for pay-as-you-go pricing because polling is more expensive.

Streaming works with HTTP/1.x, but HTTP/2 is more efficient if your client environment supports it.

There is a cost in compute operations to hold a stream open, or to repeatedly start a stream that fails.

See Billing for details.

Events

Fauna streaming uses a protocol inspired by HTTP Server-Sent Events (SSE). Fauna streams events for a subscribed document to the client, keeping the connection open when possible to minimize transmission delays. Client event handling in supported drivers is similar to WebSockets but streams are unidirectional in that the client can’t send events to the server using the stream.

Similar to the SSE protocol, events are communicated over a text-based channel. Each event is formatted as a single-line JSON object. Unlike SSE, Fauna appends a line ending, \r\n, to delimit payloads, which helps with JSON parsing when network middleware splits the event payload into multiple packets.

Events are communicated one at a time regardless of volume.

Document streaming events

This document streaming event example has the JSON formatted for easy structure identification:

{
  "action": "update", (1)
  "document": { (2)
    "ref": {
      "@ref": {
        "id": "1",
        "collection": {
          "@ref": {
            "id": "Scores",
            "collection": {
              "@ref": {
                "id": "collections"
              }
            }
          }
        }
      }
    },
    "ts": 1643910577140000,
    "data": {
      "scores": [
        1,
        11,
        17
      ],
      "foo": "bar",
      "final": true
    }
  },
  "diff": { (3)
    "ref": {
      "@ref": {
        "id": "1",
        "collection": {
          "@ref": {
            "id": "Scores",
            "collection": {
              "@ref": {
                "id": "collections"
              }
            }
          }
        }
      }
    },
    "ts": 1643910577140000,
    "data": {
      "foo": "bar",
      "final": true
    }
  },
  "prev": { (4)
    "ref": {
      "@ref": {
        "id": "1",
        "collection": {
          "@ref": {
            "id": "Scores",
            "collection": {
              "@ref": {
                "id": "collections"
              }
            }
          }
        }
      }
    },
    "ts": 1643910545600000,
    "data": {
      "scores": [
        1,
        11,
        17
      ]
    }
  }
}

Annotations:

1 The action field shows that this event is an update.
2 The document field shows the document definition for the event.
3 The diff field shows the difference between the document and prev fields. In this case, the final field is added with the value of true.
4 The prev field shows the previous document definition before the current event.

The outermost structure is a metadata event wrapper that has the following document streaming event fields:

Field Description

type

Event payload type. One of:

  • start: An event marking the start of the stream.

    Use the txn field as the stream starting timestamp. txn values for other events in the same stream are guaranteed to come after the start event txn.

    Two streams for the same document are not guaranteed to have the same txn timestamp.

  • version: An event that has information about a given document.

  • error: An event in response to an error with the stream.

  • history_rewrite: An event that has information about a historical change, such as when the subscribed document history is revised.

txn

Timestamp of the transaction emitting the event.

event

Value describing the event:

For the start event type, the value is a timestamp.

For other event types, the value is an object that has the following fields:

  • action: the type of event. One of:

    • create: Occurs when a document is created.

    • update: Occurs when an existing document is updated.

    • delete: Occurs when an existing document is deleted.

  • document: An object that has the subscribed document details. For update events, only the modified fields are included.

The document ts field is the document timestamp expressed as a Long. It is often the same as the event wrapper txn field, but it is not guaranteed to be identical.

When you establish a stream, you can opt-in to get the following fields in the event object:

  • prev: Provides the previous data for the event.

  • diff: Provides the difference between the prev field and the document field.

The event response method differs for each driver:

Set streaming events

The notification events from a set stream are generated by changes in the set to which you have subscribed. Events coming from a set stream differ from events coming from a document stream in that set streaming events include references to the document that changed instead of what changed in the collection document. You can use the document references returned in the set streaming event to query for the changed document if needed.

This set streaming event example has the JSON formatted for easy structure identification:

{
  type: "set",
  txn: 1646743802330000,
  event: {
    action: "add", (1)
    document: { (2)
     ref: ref(id = "325567069374382153", collection = ref( id = "Scores", collection = ref(id = "collections"))),
     ts: 1646743802330000
    },
    index: { (3)
      terms: [
        ref(id = "Scores", collection = ref(id = "collections"))
      ],
      values: []
    }
  }
}

Annotations:

1 The action field shows that the addition of a new document to the set triggered the event.
2 The document field shows the document definition for the event.
3 The index field shows the set whose change caused this notification to be sent.

A set streaming event includes the following fields:

Field Description

type

Type of event payload. One of:

  • start: Event marking the start of the stream.

    Use the txn field as the stream starting timestamp. The txn values for other events in the same stream are guaranteed to come after the start event txn.

    Two streams for the same document are not guaranteed to have the same txn timestamp.

  • error: Event in response to an error with the stream.

  • set: Event with information about a change to set membership.

txn

Timestamp of the transaction emitting the event.

event

Value describing the event.

For the start event type, the value is a timestamp.

For other event types, the value is an object with the following fields:

  • action: Type of event. One of:

    • add: Occurs when a document is added to a set.

    • remove: Occurs when a document is removed from a set.

  • document: Object with the details of a member document in the set.

    The document ts field is the document timestamp expressed as a Long. It is often the same as the event wrapper txn field but is not guaranteed to be the same.

When you establish a stream, you can opt-in to get an index field in the event object. The index field provides the definition of the involved index. Not all sets are index-based.

The methods to respond to events is different for each driver:

For the JavaScript driver, you can use the document helper, which takes care of this problem for you.

Limitations

The following limitations exist for streaming:

  • Active stream count:

    • 100 simultaneous streams per browser.

    • Using a driver, you can have more than 100 streams active simultaneously by creating more connection objects. Each connection supports up to 100 streams.

    • There can be other limits based on the host language HTTP2 implementation.

  • A document stream only reports events for the fields and values in the document data field.

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!