Migrations block

Learn: Schema

Schema migrations are in beta. By performing a schema migration, you automatically opt-in to the beta.

During beta, schema migrations triggers a rebuild of all of a collection’s indexes. This can make large indexes unusable for extended periods.

To register for the beta and sign up for production support, go to https://go.fauna.com/schemabetaproductionsupport.

A migrations block instructs Fauna how to handle updates to a collection’s field definitions or wildcard constraint.

This process, called a schema migration, lets you change the structure of a collection’s documents. For a tutorial, see Progressively enforce a document type.

You include a migrations block in a collection schema. A collection schema can only contain one migration block. The block must include one or more migration statements:

collection Product {
  ...
  migrations {
    // Applied 2099-05-06
    add .typeConflicts
    add .quantity
    move_conflicts .typeConflicts
    backfill .quantity = 0
    drop .internalDesc
    move .desc -> .description
    split .creationTime -> .creationTime, .creationTimeEpoch

    // Applied 2099-05-20
    // Make `price` a required field.
    split .price -> .price, .tempPrice
    drop .tempPrice
    backfill .price = 10.00
  }
}

Syntax

migrations {
  [add <field> . . .]
  [backfill <field> = <value> . . .]
  [drop <field> . . .]
  [move <origField> -> <newField> . . .]
  [move_conflicts <field> . . .]
  [move_wildcard <field> . . .]
  [split <origField> -> <splitField>, <splitField>[, <splitField> . . .] . . .]
}

Migration statements

Keyword Required Description

add

Adds a field definition. For examples, see Add a nullable field and Add and backfill a non-nullable field.

Requires a <field> accessor. Supports dot notation and bracket notation.

If the schema accepted ad hoc fields before migration, a move_conflicts statement must follow the add statement. If the field is present in existing documents, Fauna assigns non-conforming values to the move_conflicts statement’s catch-all field.

backfill

Backfills a new field with a value. For examples, see Add and backfill a non-nullable field.

Requires a <field> accessor and a field <value>. The accessor supports dot notation and bracket notation.

A backfill statement is required for any migration that could result in an empty non-nullable field.

The backfill operation only affects existing documents where the field is missing. It does not affect documents added after the migration.

The field value can be an FQL expression. The expression can have no effect other than to:

Fauna evaluates the expression at schema update time.

You can use a document as a backfill value. See Backfill using a document.

drop

Removes an existing field and its values. For an example, see Drop a field.

Requires a <field> accessor. Supports dot notation and bracket notation.

move

Renames an existing field. For an example, see Rename a field.

Requires <origField> and <newField> accessors. The accessors support dot notation and bracket notation.

move_conflicts

Assigns non-conforming values for fields in previous add migration statements to a catch-all field. For examples, see:

The move_conflicts statement only affects existing documents. It does not affect documents added after the migration.

Requires a <field> accessor for the catch-all field. The accessor supports dot notation and bracket notation.

The catch-all field’s definition must be <fieldName>: { *: Any }?. The statement nests non-conforming values in the catch-all field using the original field name as a property key.

If the catch-all field already contains a nested field with the same key, Fauna prepends the new key with an underscore (_).

move_wildcard

Assigns fields without a field definition to a catch-all field. Required to remove a wildcard constraint. For an example, see Remove a wildcard constraint.

Requires a <field> accessor for the catch-all field. The accessor supports dot notation and bracket notation.

The catch-all field’s definition must be <fieldName>: { *: Any }?. The statement nests values in the catch-all field using the original field name as a property key.

If the catch-all field already contains a nested field with the same key, Fauna prepends the new key with an underscore (_).

split

Splits an existing field into multiple fields based on data type. For examples, see Split a field.

Requires an <origField> accessor and two or more <splitField> accessors. The <origField> can be one of the <splitField> accessors.

The <origField> field’s values are assigned to the first <splitField> field with a matching type. Fields are checked from left to right. For an example, see Match values to split fields.

Run a schema migration

A typical schema migration involves three steps:

  1. Update the field definitions and wildcard constraint in the collection schema.

  2. Add one or more related migration statements to the collection schema’s migrations block. Include comments to group and annotate statements related to the same migration.

  3. Submit the updated collection schema to Fauna using the Fauna CLI or the Fauna Dashboard.

Fauna runs each new migration statement sequentially from top to bottom. Fauna ignores unchanged migration statements from previous migrations.

Changes from the migration are immediately visible in any subsequent queries.

Migration errors

When you submit a collection schema, Fauna checks the schema’s field definitions and migration statements for potential conflicts.

If a change could conflict with the collection’s data, Fauna rejects the schema with an error message. The check doesn’t require a read or scan of the collection’s documents.

Previous migration statements

For documentation purposes, you can retain migration statements from previous schema migrations in a collection schema. This lets you apply the same changes to other databases. For example, you could copy migration statements used for a staging database to run a similar migration on a production database.

Migrations for empty collections

If a collection contains no documents, you can change its field definitions and wildcard constraint without a migrations block. If the collection schema includes a migrations block, Fauna ignores it.

Limitations

A migration statement’s field accessors can’t reference nested fields, including:

  • Properties of an object field

  • Elements of an array field

Examples

Add a nullable field

Starting with the following collection schema:

collection Product {
  // Contains no field definitions.
  // Accepts ad hoc fields of any type.
  // Has an implicit wildcard constraint of
  // `*: Any`.
}

The following migration adds a nullable field to the collection. Nullable fields aren’t required in new collection documents.

collection Product {
  // Adds the `description` field.
  // Accepts `String` or `null` values.
  description: String?
  // Adds the `typeConflicts` field as a catch-all field for
  // existing `description` values that aren't `String` or `null`.
  // Because `typeConflicts` is used in a `move_conflicts`statement,
  // it must have a type of `{ *: Any }?`.
  // If the schema didn't accept ad hoc field before
  // the migration, a catch-all field isn't needed.
  typeConflicts: { *: Any }?

  // The schema now includes field definitions.
  // Adds an explicit wildcard constraint to continue
  // accepting documents with ad hoc fields.
  *: Any

  migrations {
    // Adds the `typeConflicts` field.
    add .typeConflicts
    // Adds the `description` field.
    add .description
    // Nests non-conforming `description` and `typeConflicts`
    // field values in the `typeConflicts` catch-all field.
    // If the schema didn't accept ad hoc fields before the
    // migration, a `move_conflicts` statement isn't needed.
    move_conflicts .typeConflicts
  }
}

How a catch-all field works

The previous migration uses a move_conflicts statement to reassign non-conforming description field values to the typeConflicts catch-all field.

The following examples show how the migration would affect existing documents that contain a description field.

The catch-all field for a move_wildcard statement works similarly.

Migrate a document with no changes

The migration does not affect existing documents that contain a field value of an accepted type.

{
  ...
  // `description` contains an accepted data type.
  // The field stays the same throughout the migration.
  description: "Conventional Hass, 4ct bag",
  ...
}

Similarly, a move_wildcard statement does not affect existing fields that conform to a field definition.

Migrate a non-conforming field value

If an existing document contains a description field with a non-conforming value, the migration nests the value in the typeConflicts catch-all field.

// Before migration:
{
  ...
  // `description` contains an unaccepted type.
  description: 5,
  ...
}

// After migration:
{
  ...
  // The `description` field is nested in
  // the `typeConflicts` catch-all field.
  typeConflicts: {
    description: 5
  }
  ...
}
The catch-all field already exists as an object

If the document already contains the catch-all field as an object, the migration uses the existing field.

// Before migration:
{
  ...
  // `description` contains an unaccepted type.
  description: 5,
  // The `typeConflicts` catch-all field already exists as an object.
  typeConflicts: {
    backordered: "yes"
  }
  ...
}

// After migration:
{
  ...
  // The `description` field is nested in
  // the existing `typeConflicts` catch-all field.
  typeConflicts: {
    description: 5,
    backordered: "yes"
  }
  ...
}
The catch-all field already exists with non-conforming values

If you add the catch-all field in the same migration, Fauna nests any existing, non-conforming values for the field in itself.

// Before migration:
{
  ...
  // `description` contains an unaccepted type.
  description: 5,
  // The `typeConflicts` catch-all field already exists but isn't an object.
  // The field contains an unaccepted type.
  typeConflicts: true
  ...
}

// After migration:
{
  ...
  // The existing `typeConflicts` field value doesn't conform
  // to the new `typeConflicts` field definition. The migration
  // nests the existing, non-conforming `typeConflicts` field
  // value in itself.
  typeConflicts: {
    description: 5,
    typeConflicts: true
  }
  ...
}
The catch-all field already contains the field key

If the catch-all field already contains a nested field with the same key, Fauna prepends the new key with an underscore (_).

// Before migration:
{
  ...
  // `description` contains an unaccepted type.
  description: 5,
  // The `typeConflicts` catch-all field already contains a nested
  // `description` field.
  typeConflicts: {
    description: "Conventional Hass, 4ct bag"
  }
  ...
}

// After migration:
{
  ...
  typeConflicts: {
    description: "Conventional Hass, 4ct bag",
    // The new key is prepended with an underscore.
    _description: 5
  }
  ...
}

Add and backfill a non-nullable field

Starting with the following collection schema:

collection Product {
  // Contains no field definitions.
  // Accepts ad hoc fields of any type.
  // Has an implicit wildcard constraint of
  // `*: Any`.
}

The following migration adds a non-nullable field to the collection. Non-nullable fields must include a backfill statement for existing documents.

collection Product {
  // Adds the `quantity` field.
  quantity: Int
  // Adds the `typeConflicts` field as a catch-all field for
  // existing `quantity` values that aren't `Int`.
  // Because `typeConflicts` is used in a `move_conflicts`statement,
  // it must have a type of `{ *: Any }?`.
  // If the schema didn't accept ad hoc field before
  // the migration, a catch-all field isn't needed.
  typeConflicts: { *: Any }?

  *: Any

  migrations {
    // Adds the `typeConflicts` field.
    add .typeConflicts
    // Adds the `quantity` field.
    add .quantity
    // Nests non-conforming `quantity` and `typeConflicts`
    // field values in the `typeConflicts` catch-all field.
    // If the schema didn't accept ad hoc fields before the
    // migration, a `move_conflicts` statement isn't needed.
    move_conflicts .typeConflicts
    // Set `quantity` to `0` for existing documents
    // with a `null` (missing) or non-conforming `quantity` value.
    backfill .quantity = 0
  }
}

For examples of how the migration’s move_conflicts statement reassigns non-conforming field values, see How a catch-all field works.

Backfill using today’s date

Use Date.today() to use today’s date as a backfill value:

collection Product {
  // Adds the `creationDate` field.
  creationDate: Date
  typeConflicts: { *: Any }?

  *: Any

  migrations {
    add .typeConflicts
    add .creationDate
    move_conflicts .typeConflicts
    // Set `creationDate` to today for existing documents.
    backfill .creationDate = Date.today()
  }
}

Fauna evaluates the expression at schema update time.

Backfill using the current time

Use Time.now() to use the current time as a backfill value:

collection Product {
  // Adds the `creationTime` field.
  creationTime: Time
  typeConflicts: { *: Any }?

  *: Any

  migrations {
    add .typeConflicts
    add .creationTime
    move_conflicts .typeConflicts
    // Set `creationTime` to now for existing documents.
    backfill .creationTime = Time.now()
  }
}

Fauna evaluates the expression at schema update time.

Backfill using an ID

Use newId() to use a unique ID as a backfill value. You must cast the ID to a String using toString():

collection Product {
  // Adds the `productId` field.
  productId: String = newId().toString()
  typeConflicts: { *: Any }?

  *: Any

  migrations {
    add .typeConflicts
    add .productId
    move_conflicts .typeConflicts
    // Set `productId` to an ID for existing documents.
    backfill .productId = newId().toString()
  }
}

Fauna uses the same ID value to backfill existing documents. The backfilled ID is not unique among documents.

Backfill using a document

You can use a document as a backfill value:

collection Product {
  // Adds the `store` field.
  store: Ref<Store>
  typeConflicts: { *: Any }?

  *: Any

  migrations {
    add .typeConflicts
    add .store
    move_conflicts .typeConflicts
    // Set `store` to a `Store` collection document.
    // Replace `400684606016192545` with a `Store` document ID.
    backfill .store = Store("400684606016192545")
  }
}

Fauna doesn’t guarantee the document exists. You can’t fetch the document using an FQL expression.

Add multiple fields with the same catch-all field

Multiple fields can use the same move_conflicts statement during a migration.

For example, starting with the following collection schema:

collection Product {
  // Contains no field definitions.
  // Accepts ad hoc fields of any type.
  // Has an implicit wildcard constraint of
  // `*: Any`.
}

The following migration adds multiple fields. If the fields are present in existing documents, they nest the values in the field specified by the next move_conflicts statement:

collection Product {
  description: String?
  price: Double

  typeConflicts: { *: Any }?
  *: Any

  migrations {
    add .typeConflicts
    add .description
    add .price
    // Nests non-conforming `description`, `price`, and `typeConflicts`
    // field values in the `typeConflicts` catch-all field.
    move_conflicts .typeConflicts
    backfill .price = 0.00
  }
}

Add multiple fields with different catch-all fields

A migration can include multiple move_conflicts statements. This lets you use different catch-all fields for different fields.

For example, starting with the following collection schema:

collection Product {
  // Contains no field definitions.
  // Accepts ad hoc fields of any type.
  // Has an implicit wildcard constraint of
  // `*: Any`.
}

The following migration includes multiple move_conflicts statements:

collection Product {
  description: String?
  price: Double
  quantity: Int?


  typeConflicts: { *: Any }?
  quantityTypeConflicts: { *: Any }?
  *: Any

  migrations {
    add .typeConflicts
    add .description
    add .price
    // Nests non-conforming `description`, `price`, and `typeConflicts`
    // field values in the `typeConflicts` catch-all field.
    move_conflicts .typeConflicts
    backfill .price = 0.00

    add .quantityTypeConflicts
    add .quantity
    // Nests non-conforming `quantity` and `quantityTypeConflicts`
    // field values in the `quantityTypeConflicts` catch-all field.
    move_conflicts .quantityTypeConflicts
  }
}

Drop a field

Starting with the following collection schema:

collection Product {
  price: Double = 0.00
  internalDesc: String?
}

The following migration removes the internalDesc field and its values from the collection’s documents:

collection Product {
  price: Double = 0
  // Removed the `internalDesc` field.

  migrations {
    drop .internalDesc
  }
}

Rename a field

Starting with the following collection schema:

collection Product {
  desc: String?
}

The following migration renames the desc field to description:

collection Product {
  // Renamed `desc` to `description`.
  description: String?

  migrations {
    move .desc -> .description
  }
}

Split a field

Starting with the following collection schema:

collection Product {
  creationTime: Time | Number?
}

The following migration reassigns creationTime field values in the following order:

  1. Time values to creationTime

  2. Number values to creationTimeEpoch

collection Product {
  // `creationTime` accepts `Time` values.
  creationTime: Time?
  // `creationTimeEpoch` accepts `Number` values.
  creationTimeEpoch: Number?

  migrations {
    split .creationTime -> .creationTime,  .creationTimeEpoch
  }
}

Match values to split fields

split assigns field values to the first field with a matching type. Keep this in mind when using superset types, such as Number or Any.

For example, starting with the following collection schema:

collection Product {
  creationTime: Time | Number?
}

The following migration would reassign creationTime field values in the following order:

  1. Time values to creationTime

  2. Number values, including Int values, to creationTimeNum

  3. Int values to creationTimeInt

collection Product {
  // `creationTime` accepts `Time` values.
  creationTime: Time?
  // `creationTimeNum` accepts any `Number` value, including `Int` values.
  creationTimeNum: Number?
  // `creationTimeInt` accepts `Int` values.
  creationTimeInt: Int?

  migrations {
    split .creationTime -> .creationTime, .creationTimeNum, .creationTimeInt
  }
}

Because creationTimeNum precedes creationTimeInt, split would never assign a value to the creationTimeInt field.

Instead, you can reorder the split statement as follows:

collection Product {
  // `creationTime` accepts `Time` values.
  creationTime: Time?
  // `creationTimeInt` accepts `Int` values.
  creationTimeInt: Int?
  // `creationTimeNum` accepts any other `Number` value.
  creationTimeNum: Number?

  migrations {
    split .creationTime -> .creationTime, .creationTimeInt, .creationTimeNum
  }
}

Now, split assigns any creationTime values with an Int type to creationTimeInt. Any remaining Number values are assigned to creationTimeNum.

Narrow a field’s accepted types

You can use migration statements to narrow a field’s accepted data types, including Null. For example, you can convert a nullable field to a non-nullable field.

Starting with the following collection schema:

collection Product {
  // Accepts `String` and `null` values.
  description: String?
  price: Double?
}

The following migration:

  • Uses split to reassign null values for the description field to a temporary tmp field.

  • Drops the tmp field and its values.

  • Backfills any description values that were previously null with the "default" string.

collection Product {
  // Accepts `String` values only.
  description: String
  price: Double?

  migrations {
    split .description -> .description, .tmp
    drop .tmp
    backfill .description = "default"
  }
}

Because it follows a split statement, backfill only affects documents where the description field value was null and split to tmp.

Add a wildcard constraint

Adding a wildcard constraint to a collection schema doesn’t require any related migration statements.

Starting with the following collection schema:

collection Product {
  name: String?
  description: String?
  price: Double?
  quantity: Int?
}

The following migration adds a wildcard constraint. Once added, the collection accepts documents with ad hoc fields.

collection Product {
  name: String?
  description: String?
  price: Double?
  quantity: Int?

  *: Any
}

Remove a wildcard constraint

Starting with the following collection schema:

collection Product {
  name: String?
  description: String?
  price: Double?
  quantity: Int?

  *: Any
}

The following migration removes the collection’s wildcard constraint. Once removed, the collection no longer accepts documents with ad hoc fields.

collection Product {
  name: String?
  description: String?
  price: Double?
  quantity: Int?

  // Removes the `*: Any` wildcard constraint.

  // Adds the `typeConflicts` field as a catch-all field for
  // existing ad hoc fields that don't
  // have a field definition.
  typeConflicts: { *: Any }?

  migrations {
    add .typeConflicts
    move_conflicts .typeConflicts
    // Nests existing ad hoc field values without a field definition
    // in the `typeConflicts` catch-all field.
    move_wildcard .typeConflicts
  }
}

The move_wildcard statement’s catch-all field works similarly to a move_conflict statement’s catch-all field. See How a catch-all field works.

See also

Collection schema
Field definitions

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!