Progressively enforce a document type
Learn: Schema |
---|
Fauna supports progressive enforcement for document types. Early in an application’s development, you can use collections with a schemaless or permissive document type to allow ad hoc fields in documents.
As your application evolves, you can use zero-downtime migrations to add stricter field definitions and to normalize field values. This lets you iteratively move from a permissive document type to a strict document type (or the reverse) based on your application’s needs.
In this tutorial, you’ll progressively define and enforce a document type for an example e-commerce application.
Before you start
To complete this tutorial, you’ll need:
-
The Fauna CLI
-
Familiarity with Fauna schema
Setup
Set up a database for the application.
-
Log in to Fauna using the CLI:
fauna cloud-login
When prompted, enter:
-
Endpoint name:
cloud
(Press Enter)An endpoint defines the settings the CLI uses to run API requests against a Fauna account or database. See Endpoints.
-
Email address: The email address for your Fauna account.
-
Password: The password for your Fauna account.
-
Which endpoint would you like to set as default? The
cloud-*
endpoint for your preferred Region Group. For example, to use the US Region Group, usecloud-us
.
fauna cloud-login
requires an email and password login. If you log in to Fauna using GitHub or Netlify, you can enable email and password login using the Forgot Password workflow. -
-
Create an
ecommerce
directory and navigate to it:mkdir ecommerce cd ecommerce
-
Create an
ecommerce
database:fauna create-database ecommerce
-
Initialize a project directory:
fauna project init
When prompted, use:
-
schema
as the schema directory. The command creates the directory. -
dev
as the environment name. -
The default endpoint.
-
ecommerce
as the database.
-
Create a schemaless collection
Create a schemaless collection. A schemaless collection has no predefined document fields. Instead, you can include ad hoc fields of any type in the collection’s documents.
-
Navigate to the
schema
directory:cd schema
-
Create a
collections.fsl
file and add the following collection schema to it:// Stores product data for the e-commerce application. collection Product { // Contains no field definitions. // The wildcard constraint accepts // ad hoc fields of any type. *: Any // If a collection schema has no field definitions // and no wildcard constraint, it has an implicit // wildcard constraint of `*: Any`. // Defines the `sortedByStock()` index. // Use the index to get `Product` documents sorted by // ascending `stock` field value. index sortedByStock { values [.stock] } }
-
Save
collections.fsl
. Then push the schema to Fauna:fauna schema push
When prompted, accept and stage the schema.
-
Check the status of the staged schema:
fauna schema status
-
When the status is
ready
, commit the staged schema to the database:fauna schema commit
The commit applies the staged schema to the database. This creates the collection.
Add sample data
Add documents to the Product
collection.
-
Start a shell session in the Fauna CLI:
fauna shell
The shell runs with the default
admin
key you created when you logged in usingfauna cloud-login
. -
Enter editor mode to run multi-line queries:
> .editor
-
Run the following FQL query:
// Creates a `Product` collection document. // In a schemaless collection, // documents can contain any field of any type. Product.create({ name: "pinata", description: "Original Classic Donkey Pinata", price: 24_99, stock: 40 }) // In a schemaless collection, a field // can contain different types across documents. Product.create({ name: "cups", description: "Translucent 9 Oz, 100 ct", price: 6_98, // This document's `stock` field uses a // different type than the previous document. stock: "foo", // The previous document didn't include // the `backorder` field. backorder: { limit: 5, backordered: false } })
The query creates all documents but only returns the last document.
-
Press Ctrl+D to exit the shell.
Migrate to a permissive document type
As application development continues, stricter data requirements typically
evolve. In this case, the e-commerce application wants to use the name
field
to fetch specific Product
documents.
Edit the Product
collection schema to add:
-
A field definition for the
name
field -
A migrations block
-
A
byName()
index definition
-
Edit
collections.fsl
as follows:collection Product { // Field definition for the `name` field. // `name` only accepts `String` values. name: String // Adds the `typeConflicts` field as a catch-all field for // existing `name` values that aren't `String` or `null`. // Because `typeConflicts` is used in a `move_conflicts` statement, // it must have a type of `{ *: Any }?`. typeConflicts: { *: Any }? // The schema now includes field definitions. // The wildcard constraint is required to // accept documents with ad hoc fields. // You can only remove a wildcard constraint // with a migration. *: Any // Instructs Fauna on how to handle the field and // wildcard constraint updates. migrations { // Migration #1 (Current) // Adds the `typeConflicts` field. add .typeConflicts // Adds the `name` field. add .name // Nests non-conforming `name` and `typeConflicts` field // values in the `typeConflicts` field. move_conflicts .typeConflicts // Sets `name` to `"Default"` for existing documents // with a non-string `name` value. backfill .name = "Default" } // Defines the `byName()` index. // Use the index to get `Product` documents by `name` value. index byName { terms [.name] } index sortedByStock { values [.stock] } }
-
Save
collections.fsl
. Then push the schema to Fauna:fauna schema push
When prompted, accept and stage the schema.
-
Check the status of the staged schema:
fauna schema status
-
When the status is
ready
, commit the staged schema to the database:fauna schema commit
This runs the migration.
New
Product
collection documents must now contain aname
field with a String value. The documents can also contain other ad hoc fields.
Review existing data
Run a few queries to review changes the migration made to the collection’s documents.
-
Start a shell session:
fauna shell
-
Run the following query:
Product.sortedByStock()
The query returns all
Product
collection documents, sorted by ascendingstock
value:{ data: [ { id: "400414078704549921", coll: Product, ts: Time("2099-06-11T16:31:12.790Z"), name: "pinata", description: "Original Classic Donkey Pinata", price: 2499, stock: 40 }, { id: "400414078705598497", coll: Product, ts: Time("2099-06-11T16:31:12.790Z"), name: "cups", description: "Translucent 9 Oz, 100 ct", price: 698, stock: "foo", backorder: { limit: 5, backordered: false } } ] }
The migration made no changes to existing documents. The documents already contain a
name
field with a String value.
Add new data
New Product
collection documents must contain a name
field with a
String value. Attempt to add some non-conforming documents to the
collection.
-
In the shell session, run the following query in editor mode:
// Create a `Product` collection document // without a `name` field. Product.create({ description: "Conventional Hass, 4ct bag", price: 3_99, stock: 95 })
The query returns an error.
Product
collection documents must contain aname
field. -
Run the following query in editor mode:
// Create a `Product` collection document // with a non-String `name` value Product.create({ name: 12345, description: "Conventional Hass, 4ct bag", price: 3_99, stock: 95 })
The query returns an error. The
name
field must contain a String. -
Run the following query in editor mode:
// Create a `Product` collection document // with a `name` String Product.create({ name: "avocados", description: "Conventional Hass, 4ct bag", price: 3_99, stock: 95 })
The query runs successfully and creates a new
Product
document. -
Press Ctrl+D to exit the shell.
Add more field definitions
You can perform iterative migrations to add more field definitions.
Edit the Product
collection schema to:
-
Add field definitions for the
description
andprice
fields -
Update the migrations block
-
Edit
collections.fsl
as follows:collection Product { name: String // Adds the `description` field. // Only accepts `String` values. description: String // Adds the `price` field. // Only accepts `Int` values. price: Int typeConflicts: { *: Any }? *: Any migrations { // Migration #1 (Previous) // Already run. Fauna ignores // previously run migration statements. add .typeConflicts add .name move_conflicts .typeConflicts backfill .name = "Default" // Migration #2 (Current) // New migration statements. Fauna // only runs these statements. add .description add .price // Uses the existing `typeConflicts` field // as a catch-all field. move_conflicts .typeConflicts backfill .description = "Default" backfill .price = 1 } index byName { terms [.name] } index sortedByStock { values [.stock] } }
-
Save
collections.fsl
. Then push the schema to Fauna:fauna schema push
When prompted, accept and stage the schema.
-
Check the status of the staged schema:
fauna schema status
-
When the status is
ready
, commit the staged schema to the database:fauna schema commit
This runs the migration.
Review existing data
Run a few queries to review changes the migration made to the collection’s documents.
-
Start a shell session:
fauna shell
-
Run the following query:
Product.sortedByStock()
The query returns all
Product
collection documents, sorted by ascendingstock
value:{ data: [ { id: "400414078704549921", coll: Product, ts: Time("2099-06-11T16:31:12.790Z"), name: "pinata", description: "Original Classic Donkey Pinata", price: 2499, stock: 40 }, ... ] }
The migration made no changes to the existing documents.
-
Press Ctrl+D to exit the shell.
Handle non-conforming values
Ad hoc fields may contain different types across documents in a collection. You can add a field definition to normalize an existing ad hoc field and narrow its accepted types.
A move_conflicts
migration statement lets you assign non-conforming field
values to a catch-all field.
Edit the Product
collection schema to:
-
Add a field definition for the
stock
field -
Update migrations block
-
Edit
collections.fsl
as follows:collection Product { name: String description: String price: Int // Adds the `stock` field. // Only accepts `Int` values. stock: Int typeConflicts: { *: Any }? *: Any migrations { // Migration #1 (Previous) // Already run. Ignored. add .typeConflicts add .name move_conflicts .typeConflicts backfill .name = "Default" // Migration #2 (Previous) // Already run. Ignored. add .description add .price move_conflicts .typeConflicts backfill .description = "Default" backfill .price = 1 // Migration #3 (Current) // New migration statements. add .stock // Uses the existing `typeConflicts` field // as a catch-all field. move_conflicts .typeConflicts backfill .stock = 0 } index byName { terms [.name] } index sortedByStock { values [.stock] } }
-
Save
collections.fsl
. Then push the schema to Fauna:fauna schema push
When prompted, accept and stage the schema.
-
Check the status of the staged schema:
fauna schema status
-
When the status is
ready
, commit the staged schema to the database:fauna schema commit
This runs the migration.
Review existing data
Run a query to see how the migration affected a document with a non-conforming
stock
value.
-
Start a shell session:
fauna shell
-
Run the following query:
Product.byName("cups").first()
The query returns:
{ id: "400414078705598497", coll: Product, ts: Time("2099-06-11T17:24:08.690Z"), name: "cups", description: "Translucent 9 Oz, 100 ct", price: 698, stock: 0, backorder: { limit: 5, backordered: false }, typeConflicts: { stock: "foo" } }
During the migration, Fauna nested the document’s non-conforming
stock
value in thetypeConflicts
catch-all field, which was specified by themove_conflicts
statement.The migration then filled in the specified
backfill
value of0
so the document conforms to the field definition. -
Press Ctrl+D to exit the shell.
Migrate to a strict document type
As the application scales and its data model stabilizes, storage costs and data consistency become more important than flexibility. It no longer makes sense to accept and store ad hoc fields.
Edit the Product
collection schema to remove the wildcard constraint.
-
Edit
collections.fsl
as follows:collection Product { name: String description: String price: Int stock: Int typeConflicts: { *: Any }? // Removes the `*: Any` wildcard constraint. // The collection no longer accepts // documents with ad hoc fields. migrations { // Migration #1 (Previous) // Already run. Ignored. add .typeConflicts add .name move_conflicts .typeConflicts backfill .name = "Default" // Migration #2 (Previous) // Already run. Ignored. add .description add .price move_conflicts .typeConflicts backfill .description = "Default" backfill .price = 1 // Migration #3 (Previous) // Already run. Ignored. add .stock move_conflicts .typeConflicts backfill .stock = 0 // Migration #4 (Current) // New migration statements. // Moves any existing ad hoc fields to // the `typeConflicts` catch-all field. move_wildcard .typeConflicts } index byName { terms [.name] } index sortedByStock { values [.stock] } }
-
Save
collections.fsl
. Then push the schema to Fauna:fauna schema push
When prompted, accept and stage the schema.
-
Check the status of the staged schema:
fauna schema status
-
When the status is
ready
, commit the staged schema to the database:fauna schema commit
This runs the migration.
Review existing data
Run a query to see how the migration affected a document that contained an ad hoc field without a field definition.
-
Start a shell session:
fauna shell
-
Run the following query in editor mode:
// Gets `Product` collection documents // with a `name` of `cups`. Then get the // first document. Product.byName("cups").first()
The query returns:
{ id: "400414078705598497", coll: Product, ts: Time("2099-06-11T17:24:08.690Z"), name: "cups", description: "Translucent 9 Oz, 100 ct", price: 698, stock: 0, typeConflicts: { stock: "foo", backorder: { limit: 5, backordered: false } } }
During the migration, Fauna nested the document’s ad hoc
backorder
field value in thetypeConflicts
catch-all field, which specified by themove_wildcard
statement.
Attempt to add invalid data
Attempt to add non-conforming documents to the Product
collection. New
documents can’t contain ad hoc fields. Field values must conform to
their field definitions.
-
In the shell session, run the following query in editor mode:
// Create a `Product` collection document // with an ad hoc `stocked` field. Product.create({ name: "limes", description: "Conventional, 1 ct", price: 35, stock: 100, stocked: true })
The query returns an error. The
Product
collection no longer accepts documents with ad hoc fields. -
Run the following query in editor mode:
// Create a `Product` collection document // with a missing `stock` field. Product.create({ name: "limes", description: "Conventional, 1 ct", price: 35 })
The query returns an error. The
stock
field must be present in new documents. -
Run the following query in editor mode:
// Create a `Product` collection document // with all required fields and no ad hoc fields. Product.create({ name: "limes", description: "Conventional, 1 ct", price: 35, stock: 100 })
The query runs successfully and creates a new
Product
document. -
Press Ctrl+D to exit the shell.
Is this article helpful?
Tell Fauna how the article can be improved:
Visit Fauna's forums
or email docs@fauna.com
Thank you for your feedback!