Check out v4 of the Fauna CLI

v4 of the Fauna CLI is now in beta.

The new version introduces enhancements to the developer experience, including an improved authentication workflow. To get started, check out the CLI v4 quick start.

Progressively enforce a document type

Learn: Schema

Fauna supports progressive enforcement for document types. Early in an application’s development, you can use collections with a schemaless or permissive document type to allow ad hoc fields in documents.

As your application evolves, you can use zero-downtime migrations to add stricter field definitions and to normalize field values. This lets you iteratively move from a permissive document type to a strict document type (or the reverse) based on your application’s needs.

In this tutorial, you’ll progressively define and enforce a document type for an example e-commerce application.

This procedure uses v4 of the Fauna CLI, which is in beta. For the latest GA version, see v3 of the Fauna CLI.

Before you start

To complete this tutorial, you’ll need:

Setup

Set up a database for the application.

  1. If you haven’t already, log in to Fauna using the CLI:

    fauna login

  2. Create an ecommerce directory and navigate to it:

    mkdir ecommerce
cd ecommerce

  3. Create an ecommerce database:

    # Replace 'us' with your preferred Region Group:
# 'us' (United States), 'eu' (Europe), or `global`.
fauna database create \
  --name "ecommerce" \
  --database us

Create a schemaless collection

Create a schemaless collection. A schemaless collection has no predefined document fields. Instead, you can include ad hoc fields of any type in the collection’s documents.

  1. Create and navigate to the schema directory:

    mkdir schema
cd schema

  2. Create a collections.fsl file and add the following collection schema to it:

    // Stores product data for the e-commerce application.
collection Product {
  // Contains no field definitions.

  // The wildcard constraint accepts
  // ad hoc fields of any type.
  *: Any
  // If a collection schema has no field definitions
  // and no wildcard constraint, it has an implicit
  // wildcard constraint of `*: Any`.

  // Defines the `sortedByStock()` index.
  // Use the index to get `Product` documents sorted by
  // ascending `stock` field value.
  index sortedByStock {
    values [.stock]
  }
}

  3. Save collections.fsl. Then push the schema to Fauna:

    # Replace 'us' with your database's Region Group.
fauna schema push \
  --database us/ecommerce

    When prompted, accept and stage the schema.

  4. Check the status of the staged schema:

    fauna schema status \
  --database us/ecommerce

  5. When the status is ready, commit the staged schema to the database:

    fauna schema commit \
  --database us/ecommerce

    The commit applies the staged schema to the database. This creates the collection.

Add sample data

Add documents to the Product collection.

  1. Start a shell session with the admin role in the Fauna CLI:

    fauna shell \
  --database us/ecommerce \
  --role admin

  2. Enter editor mode to run multi-line queries:

    > .editor

  3. Run the following FQL query:

    // Creates a `Product` collection document.
// In a schemaless collection,
// documents can contain any field of any type.
Product.create({
  name: "pinata",
  description: "Original Classic Donkey Pinata",
  price: 24_99,
  stock: 40
})

// In a schemaless collection, a field
// can contain different types across documents.
Product.create({
  name: "cups",
  description: "Translucent 9 Oz, 100 ct",
  price: 6_98,
  // This document's `stock` field uses a
  // different type than the previous document.
  stock: "foo",
  // The previous document didn't include
  // the `backorder` field.
  backorder: {
    limit: 5,
    backordered: false
  }
})

    The query creates all documents but only returns the last document.

  4. Press Ctrl+D to exit the shell.

Migrate to a permissive document type

As application development continues, stricter data requirements typically evolve. In this case, the e-commerce application wants to use the name field to fetch specific Product documents.

Edit the Product collection schema to add:

  • A field definition for the name field

  • A migrations block

  • A byName() index definition

  1. Edit collections.fsl as follows:

    collection Product {
  // Field definition for the `name` field.
  // `name` only accepts `String` values.
  name: String
  // Adds the `typeConflicts` field as a catch-all field for
  // existing `name` values that aren't `String` or `null`.
  // Because `typeConflicts` is used in a `move_conflicts` statement,
  // it must have a type of `{ *: Any }?`.
  typeConflicts: { *: Any }?

  // The schema now includes field definitions.
  // The wildcard constraint is required to
  // accept documents with ad hoc fields.
  // You can only remove a wildcard constraint
  // with a migration.
  *: Any

  // Instructs Fauna on how to handle the field and
  // wildcard constraint updates.
  migrations {
    // Migration #1 (Current)
    // Adds the `typeConflicts` field.
    add .typeConflicts
    // Adds the `name` field.
    add .name
    // Nests non-conforming `name` and `typeConflicts` field
    // values in the `typeConflicts` field.
    move_conflicts .typeConflicts
    // Sets `name` to `"Default"` for existing documents
    // with a non-string `name` value.
    backfill .name = "Default"
  }

  // Defines the `byName()` index.
  // Use the index to get `Product` documents by `name` value.
  index byName {
    terms [.name]
  }

  index sortedByStock {
    values [.stock]
  }
}

  2. Save collections.fsl. Then push the schema to Fauna:

    fauna schema push \
  --database us/ecommerce

    When prompted, accept and stage the schema.

  3. Check the status of the staged schema:

    fauna schema status \
  --database us/ecommerce

  4. When the status is ready, commit the staged schema to the database:

    fauna schema commit \
  --database us/ecommerce

    This runs the migration.

    New Product collection documents must now contain a name field with a String value. The documents can also contain other ad hoc fields.

Review existing data

Run a few queries to review changes the migration made to the collection’s documents.

  1. Start a shell session:

    fauna shell \
  --database us/ecommerce
  --role admin

  2. Run the following query:

    Product.sortedByStock()

    The query returns all Product collection documents, sorted by ascending stock value:

    {
  data: [
    {
      id: "400414078704549921",
      coll: Product,
      ts: Time("2099-06-11T16:31:12.790Z"),
      name: "pinata",
      description: "Original Classic Donkey Pinata",
      price: 2499,
      stock: 40
    },
    {
      id: "400414078705598497",
      coll: Product,
      ts: Time("2099-06-11T16:31:12.790Z"),
      name: "cups",
      description: "Translucent 9 Oz, 100 ct",
      price: 698,
      stock: "foo",
      backorder: {
        limit: 5,
        backordered: false
      }
    }
  ]
}

    The migration made no changes to existing documents. The documents already contain a name field with a String value.

Add new data

New Product collection documents must contain a name field with a String value. Attempt to add some non-conforming documents to the collection.

  1. In the shell session, run the following query in editor mode:

    // Create a `Product` collection document
// without a `name` field.
Product.create({
  description: "Conventional Hass, 4ct bag",
  price: 3_99,
  stock: 95
})

    The query returns an error. Product collection documents must contain a name field.

  2. Run the following query in editor mode:

    // Create a `Product` collection document
// with a non-String `name` value
Product.create({
  name: 12345,
  description: "Conventional Hass, 4ct bag",
  price: 3_99,
  stock: 95
})

    The query returns an error. The name field must contain a String.

  3. Run the following query in editor mode:

    // Create a `Product` collection document
// with a `name` String
Product.create({
  name: "avocados",
  description: "Conventional Hass, 4ct bag",
  price: 3_99,
  stock: 95
})

    The query runs successfully and creates a new Product document.

  4. Press Ctrl+D to exit the shell.

Add more field definitions

You can perform iterative migrations to add more field definitions.

Edit the Product collection schema to:

  • Add field definitions for the description and price fields

  • Update the migrations block

  1. Edit collections.fsl as follows:

    collection Product {
  name: String
  // Adds the `description` field.
  // Only accepts `String` values.
  description: String
  // Adds the `price` field.
  // Only accepts `Int` values.
  price: Int
  typeConflicts: { *: Any }?

  *: Any

  migrations {
    // Migration #1 (Previous)
    // Already run. Fauna ignores
    // previously run migration statements.
    add .typeConflicts
    add .name
    move_conflicts .typeConflicts
    backfill .name = "Default"

    // Migration #2 (Current)
    // New migration statements. Fauna
    // only runs these statements.
    add .description
    add .price
    // Uses the existing `typeConflicts` field
    // as a catch-all field.
    move_conflicts .typeConflicts
    backfill .description = "Default"
    backfill .price = 1
  }

  index byName {
    terms [.name]
  }

  index sortedByStock {
    values [.stock]
  }
}

  2. Save collections.fsl. Then push the schema to Fauna:

    fauna schema push \
  --database us/ecommerce

    When prompted, accept and stage the schema.

  3. Check the status of the staged schema:

    fauna schema status \
  --database us/ecommerce

  4. When the status is ready, commit the staged schema to the database:

    fauna schema commit \
  --database us/ecommerce

    This runs the migration.

    New Product collection documents must now contain:

    • A name field with a String value

    • A description field with a String value

    • A price field with an Int value

    The documents can also contain other ad hoc fields.

Review existing data

Run a few queries to review changes the migration made to the collection’s documents.

  1. Start a shell session:

    fauna shell \
  --database us/ecommerce \
  --role admin

  2. Run the following query:

    Product.sortedByStock()

    The query returns all Product collection documents, sorted by ascending stock value:

    {
  data: [
    {
      id: "400414078704549921",
      coll: Product,
      ts: Time("2099-06-11T16:31:12.790Z"),
      name: "pinata",
      description: "Original Classic Donkey Pinata",
      price: 2499,
      stock: 40
    },
    ...
  ]
}

    The migration made no changes to the existing documents.

  3. Press Ctrl+D to exit the shell.

Handle non-conforming values

Ad hoc fields may contain different types across documents in a collection. You can add a field definition to normalize an existing ad hoc field and narrow its accepted types.

A move_conflicts migration statement lets you assign non-conforming field values to a catch-all field.

Edit the Product collection schema to:

  • Add a field definition for the stock field

  • Update migrations block

  1. Edit collections.fsl as follows:

    collection Product {
  name: String
  description: String
  price: Int
  // Adds the `stock` field.
  // Only accepts `Int` values.
  stock: Int
  typeConflicts: { *: Any }?

  *: Any

  migrations {
    // Migration #1 (Previous)
    // Already run. Ignored.
    add .typeConflicts
    add .name
    move_conflicts .typeConflicts
    backfill .name = "Default"

    // Migration #2 (Previous)
    // Already run. Ignored.
    add .description
    add .price
    move_conflicts .typeConflicts
    backfill .description = "Default"
    backfill .price = 1

    // Migration #3 (Current)
    // New migration statements.
    add .stock
    // Uses the existing `typeConflicts` field
    // as a catch-all field.
    move_conflicts .typeConflicts
    backfill .stock = 0
  }

  index byName {
    terms [.name]
  }

  index sortedByStock {
    values [.stock]
  }
}

  2. Save collections.fsl. Then push the schema to Fauna:

    fauna schema push \
  --database us/ecommerce

    When prompted, accept and stage the schema.

  3. Check the status of the staged schema:

    fauna schema status \
  --database us/ecommerce

  4. When the status is ready, commit the staged schema to the database:

    fauna schema commit \
  --database us/ecommerce

    This runs the migration.

    New Product collection documents must now contain:

    • A name field with a String value

    • A description field with a String value

    • A price field with an Int value

    • A stock field with an Int value

    The documents can also contain other ad hoc fields.

Review existing data

Run a query to see how the migration affected a document with a non-conforming stock value.

  1. Start a shell session:

    fauna shell \
  --database us/ecommerce \
  --role admin

  2. Run the following query:

    Product.byName("cups").first()

    The query returns:

    {
  id: "400414078705598497",
  coll: Product,
  ts: Time("2099-06-11T17:24:08.690Z"),
  name: "cups",
  description: "Translucent 9 Oz, 100 ct",
  price: 698,
  stock: 0,
  backorder: {
    limit: 5,
    backordered: false
  },
  typeConflicts: {
    stock: "foo"
  }
}

    During the migration, Fauna nested the document’s non-conforming stock value in the typeConflicts catch-all field, which was specified by the move_conflicts statement.

    The migration then filled in the specified backfill value of 0 so the document conforms to the field definition.

  3. Press Ctrl+D to exit the shell.

Migrate to a strict document type

As the application scales and its data model stabilizes, storage costs and data consistency become more important than flexibility. It no longer makes sense to accept and store ad hoc fields.

Edit the Product collection schema to remove the wildcard constraint.

  1. Edit collections.fsl as follows:

    collection Product {
  name: String
  description: String
  price: Int
  stock: Int
  typeConflicts: { *: Any }?

  // Removes the `*: Any` wildcard constraint.
  // The collection no longer accepts
  // documents with ad hoc fields.

  migrations {
    // Migration #1 (Previous)
    // Already run. Ignored.
    add .typeConflicts
    add .name
    move_conflicts .typeConflicts
    backfill .name = "Default"

    // Migration #2 (Previous)
    // Already run. Ignored.
    add .description
    add .price
    move_conflicts .typeConflicts
    backfill .description = "Default"
    backfill .price = 1

    // Migration #3 (Previous)
    // Already run. Ignored.
    add .stock
    move_conflicts .typeConflicts
    backfill .stock = 0

    // Migration #4 (Current)
    // New migration statements.
    // Moves any existing ad hoc fields to
    // the `typeConflicts` catch-all field.
    move_wildcard .typeConflicts
  }

  index byName {
    terms [.name]
  }

  index sortedByStock {
    values [.stock]
  }
}

  2. Save collections.fsl. Then push the schema to Fauna:

    fauna schema push \
  --database us/ecommerce

    When prompted, accept and stage the schema.

  3. Check the status of the staged schema:

    fauna schema status \
  --database us/ecommerce

  4. When the status is ready, commit the staged schema to the database:

    fauna schema commit \
  --database us/ecommerce

    This runs the migration.

    New Product collection documents must now contain:

    • A name field with a String value

    • A description field with a String value

    • A price field with an Int value

    • A stock field with an Int value

    The documents can’t contain other fields.

Review existing data

Run a query to see how the migration affected a document that contained an ad hoc field without a field definition.

  1. Start a shell session:

    fauna shell \
  --database us/ecommerce \
  --role admin

  2. Run the following query in editor mode:

    // Gets `Product` collection documents
// with a `name` of `cups`. Then get the
// first document.
Product.byName("cups").first()

    The query returns:

    {
  id: "400414078705598497",
  coll: Product,
  ts: Time("2099-06-11T17:24:08.690Z"),
  name: "cups",
  description: "Translucent 9 Oz, 100 ct",
  price: 698,
  stock: 0,
  typeConflicts: {
    stock: "foo",
    backorder: {
      limit: 5,
      backordered: false
    }
  }
}

    During the migration, Fauna nested the document’s ad hoc backorder field value in the typeConflicts catch-all field, which specified by the move_wildcard statement.

Attempt to add invalid data

Attempt to add non-conforming documents to the Product collection. New documents can’t contain ad hoc fields. Field values must conform to their field definitions.

  1. In the shell session, run the following query in editor mode:

    // Create a `Product` collection document
// with an ad hoc `stocked` field.
Product.create({
  name: "limes",
  description: "Conventional, 1 ct",
  price: 35,
  stock: 100,
  stocked: true
})

    The query returns an error. The Product collection no longer accepts documents with ad hoc fields.

  2. Run the following query in editor mode:

    // Create a `Product` collection document
// with a missing `stock` field.
Product.create({
  name: "limes",
  description: "Conventional, 1 ct",
  price: 35
})

    The query returns an error. The stock field must be present in new documents.

  3. Run the following query in editor mode:

    // Create a `Product` collection document
// with all required fields and no ad hoc fields.
Product.create({
  name: "limes",
  description: "Conventional, 1 ct",
  price: 35,
  stock: 100
})

    The query runs successfully and creates a new Product document.

  4. Press Ctrl+D to exit the shell.

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!