Check out v4 of the Fauna CLI

v4 of the Fauna CLI is now in beta.

The new version introduces enhancements to the developer experience, including an improved authentication workflow. To get started, check out the CLI v4 quick start.

Snapshot Export

Fauna’s Snapshot Export feature lets you create a point-in-time snapshot of a database or specified user-defined collections and store their document data as JSON files in an AWS S3 bucket. You can use exports to:

  • Load Fauna data into third-party systems.

  • Store Fauna data externally for compliance or data control.

You can combine exports with Event Feeds to track database changes in other systems for change data capture (CDC).

We don’t recommend using exports to restore data in Fauna. To back up and restore databases, see Database backup and restore.

Requirements

To use exports, you must:

  • Have an Enterprise plan.

  • Be an owner, admin, or developer for your Fauna account. See Team management .

  • Have an AWS S3 bucket to store exported data. The bucket must grant permissions to Fauna’s IAM role. See S3 bucket permissions.

S3 bucket permissions

To create an export, you must specify an AWS S3 bucket where the exported data will be stored. The S3 bucket must grant the following permissions to Fauna’s IAM Role ARN (arn:aws:iam::209479277009:role/FaunaExport):

  • s3:GetBucketLocation

  • s3:PutObject

  • s3:PutObjectAcl (if bucket ACLs are enabled)

Example bucket policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Allow Fauna export operations",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::209479277009:role/FaunaExport"
      },
      "Action": [
        "s3:GetBucketLocation",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::your-bucket",
        "arn:aws:s3:::your-bucket/*"
      ]
    },
    {
      "Sid": "Allow Fauna to set bucket owner full control (if ACLs enabled)",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::209479277009:role/FaunaExport"
      },
      "Action": "s3:PutObjectAcl",
      "Resource": "arn:aws:s3:::your-bucket/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    }
  ]
}

All file uploads to the bucket use S3 standard encryption.

Create and manage exports

You create and manage exports using any of the following:

Fauna CLI

The following examples create and manage exports using the Fauna CLI’s fauna export commands.

Create an export

To start an export, use the fauna export create s3 command. In the command, specify:

  • The --database, including its Region Group identifier and hierarchy. Shorthand Region Group identifiers are supported. Child databases are not included in the export.

  • One or more user-defined collections to export, passed using the --collection flag. Pass multiple collections as a space-separated list, such as --collection Product Category. To export all user-defined collections, omit the --collection flag. You can’t export system collections.

  • An optional data --format used to convert exported document data to JSON. Defaults to the simple format.

  • The S3 --bucket and --path used to store export files.

    Double-check your bucket name before submitting a request. Once requested, you can’t cancel an export. Fauna can’t delete exported data in an S3 bucket.

    We recommend using a unique path for each export. If you submit multiple export requests with the same bucket path, the files from the most recent request will overwrite existing conflicting files. Non-conflicting files will not be overwritten, which may result in a mix of files from multiple exports in the same path.

    Exports from databases in Fauna’s Europe (eu-std) Region Group must be stored in an S3 bucket located in an EU AWS region.

    		 
# Export the the 'Product' and 'Category' collections in
# the 'us/parent_db/child_db' database. Store the export
# in the 'fauna_exports/parent_db/child_db/2099-12-31'
# path of the 'doc-example-bucket' S3 bucket. Format FQL data
# using the 'simple' data format.
fauna export create s3 \
  --database us/parent_db/child_db \
  --collection Product Category \
  --bucket doc-example-bucket \ # Replace with your bucket.
  --path fauna_exports/parent_db/child_db/2099-12-31 \
  --format simple

The command starts an asynchronous export operation that runs in the background. Requests aren’t idempotent.

The command output includes a unique ID for the export:

123456789

Get exports

You can pass the export ID to the fauna export get command:

# Get an export with an ID of '123456789'.
fauna export get 123456789

The response includes the current state of the export:

id: "123456789"
state: Complete
database: us-std/parent_db/child_db
format: simple
destination:
  s3:
    bucket: doc-example-bucket
    path: fauna_exports/parent_db/child_db/2099-12-31
created_at: 2099-12-31T19:07:25.642703Z
updated_at: 2099-12-31T19:07:25.642703Z
collections:
  - Product
  - Category
destination_uri: s3://doc-example-bucket/fauna_exports/parent_db/child_db/2099-12-31

Possible state values:

State Description

Pending

The export request has been received but is not yet in progress.

InProgress

The export is in progress. This includes copying collection files to the S3 bucket.

Complete

The export is complete. Collection files are available in the S3 bucket.

Failed

There was an error processing the export. You can safely retry the export.

Alternatively, you can use the fauna export list command to get a list of all exports and their states for an account:

fauna export list

You can optionally filter the export list by --state. You can pass multiple --state values as a space-separated list.

# List exports in the 'Pending' or
# 'Complete' state.
fauna export list \
  --state Pending Complete

Account API

The following examples create and manage exports using the Fauna Account HTTP API’s Export endpoints.

Create an account key

To authenticate the Fauna Account API requests, you must have an account key. You can create an account key in the Fauna Dashboard.

  1. Log in to the Fauna Dashboard and click Account in the left navigation.

  2. Click Account Keys.

  3. Click Create Key.

  4. Enter a Name and an optional TTL. The TTL is the number of days until the account key expires.

  5. Click Create.

  6. Copy the Key Secret.

You can pass the secret as a Bearer token in Fauna Account API requests.

Create an export

To start an export, make a request to the Create Export endpoint. In the request, specify:

  • The database, including its Region Group identifier and hierarchy. Shorthand Region Group identifiers are not supported. Child databases are not included in the export.

  • One or more user-defined collections to export. To export all user-defined collections, omit the collections field. You can’t export system collections.

  • The data format used to convert exported FQL document data to JSON.

  • The S3 bucket and path used to store export files.

    Double-check your bucket name before submitting a request. Once requested, you can’t cancel an export. Fauna can’t delete exported data in an S3 bucket.

    We recommend using a unique path for each export. If you submit multiple export requests with the same bucket path, the files from the most recent request will overwrite existing conflicting files. Non-conflicting files will not be overwritten, which may result in a mix of files from multiple exports in the same path.

    Exports from databases in Fauna’s Europe (eu-std) Region Group must be stored in an S3 bucket located in an EU AWS region.

    		 
# Export the 'Product' and 'Category' collections in
# the 'us-std/parent_db/child_db' database. Store the export
# in the 'fauna_exports/parent_db/child_db/2099-12-31'
# path of the 'doc-example-bucket' S3 bucket. Format FQL data
# using the 'simple' data format.
curl -X POST \
  'https://account.fauna.com/v2/exports' \
  -H 'Authorization: Bearer <ACCOUNT_KEY_SECRET>' \
  -H 'Content-Type: application/json' \
  -d '{
    "database": "us-std/parent_db/child_db",
    "collections": [
      "Product",
      "Category"
    ],
    "format": "simple",
    "destination": {
      "s3": {
        "bucket": "doc-example-bucket", # Replace with your bucket.
        "path": "fauna_exports/parent_db/child_db/2099-12-31"
      }
    }
  }'

The request starts an asynchronous export operation that runs in the background. Requests aren’t idempotent.

The response includes a unique ID for the export and the export’s current state.

{
  "response": {
    "id": 123456789,
    "state": "Pending",
    "database": "us-std/parent_db/child_db",
    "format": "simple",
    "destination": {
      "s3": {
        "bucket": "doc-example-bucket",
        "path": "fauna_exports/my_db/2099-12-31"
      }
    },
    "created_at": "2099-12-31T21:46:12.580Z",
    "updated_at": "2099-12-31T21:46:12.580Z",
    "collections": [
      "Product",
      "Category"
    ]
  }
}

Possible state values:

State Description

Pending

The export request has been received but is not yet in progress.

InProgress

The export is in progress. This includes copying collection files to the S3 bucket.

Complete

The export is complete. Collection files are available in the S3 bucket.

Failed

There was an error processing the export. You can safely retry the export.

Get exports

You can pass the export ID to the Get Export endpoint to get the current state of an export:

curl -X GET \
  'https://account.fauna.com/v2/exports/<EXPORT_ID>' \
  -H 'Authorization: Bearer <ACCOUNT_KEY_SECRET>'

Alternatively, you can use the List Exports endpoint to get a list of all exports and their states for an account:

curl -X GET \
  'https://account.fauna.com/v2/exports' \
  -H 'Authorization: Bearer <ACCOUNT_KEY_SECRET>'

You can optionally filter the export list by state and database. You can pass the state query parameter multiple times to filter on multiple states.

# Gets exports in the 'Pending' or 'Complete' state
# for the 'us-std/parent_db/child_db' database.
curl -X GET \
  --get \
  'https://account.fauna.com/v2/exports' \
  -H 'Authorization: Bearer <ACCOUNT_KEY_SECRET>' \
  --data-urlencode 'state=Pending' \
  --data-urlencode 'state=Complete' \
  --data-urlencode 'database=us-std/parent_db/child_db'

Export files

The export operation writes the following files to the specified S3 bucket path:

Collection files

Fauna exports document data for each collection as one or more JSON files. Exported collection files use the following naming convention:

collections/<COLLECTION_NAME>/<COLLECTION_NAME>_<HOST_ID>_<FILE_IDX>.json

Where:

  • <COLLECTION_NAME> is the name of the collection.

  • <HOST_ID> is an identifier, ranging from 00 to 99 for the Fauna host that uploaded the file. An export may have multiple hosts.

  • <FILE_IDX> is an index for the file, ranging from 000000 to 999999. Fauna writes collection files sequentially in ascending index order, starting with 000000 for each host.

Collection file names can vary between exports, even exports of the same database with the same data.

Each collection export file contains an array of JSON objects, where each object represents a collection document. Documents are ordered by ascending document ID. Document data is converted from FQL to JSON based on the data format specified in the request. Example using the tagged format:

[
  {
    "id": "419646277065637921",
    "coll": {
      "@mod": "Category"
    },
    "ts": {
      "@time": "2099-01-09T23:18:46.510Z"
    },
    "name": "party",
    "description": "Party Supplies"
  },
  ...
]

Manifest file

When the export is complete, Fauna uploads a manifest.json file to the S3 bucket path. The manifest.json file contains metadata about the export and describes the structure and details of the exported data.

{
  "export_id": "123456789",
  "snapshot_ts": "2099-01-10T17:35:38.097961Z",
  "datafile_compression": false,
  "datafile_format": "json",
  "document_format": "tagged",
  "object_count": 10,
  "object_keys": [
    "fauna_exports/parent_db/child_db/2099-12-31/collections/Category/Category_00_000000.json",
    "fauna_exports/parent_db/child_db/2099-12-31/collections/Product/Product_00_000000.json",
    ...
  ]
}
Field Type Description

export_id

String

Unique ID for the export. The ID is a string-encoded integer.

snapshot_ts

String

The timestamp of the export’s snapshot in ISO 8601 format. Exports capture a consistent state of the database’s document data at the snapshot_ts.

datafile_compression

Boolean

Indicates whether the export’s collection files are compressed.

datafile_format

String

Format for the export’s collection files. Must be json.

document_format

String

Data format used to convert the export’s FQL document data to JSON. Valid values are tagged and simple.

object_count

Integer

Total number of the export’s collection files.

object_keys

Array of strings

Paths to the export’s collection files.

Permissions validated file

At the start of an export, Fauna uploads a .perms_validated.txt file to verify that Fauna has appropriate permissions access for the provided S3 bucket path. Fauna doesn’t use the file after its creation. You can safely delete it.

Limitations

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!