Indexes
Reference: Index definitions |
---|
An index stores, or covers, specific document field values for quick retrieval. You can use indexes to filter and sort a collection’s documents in a performant way.
Using indexes can significantly improve query performance and reduce costs, especially for large datasets. Unindexed queries should be avoided.
Define an index
You create and manage indexes at the collection level as part of an FSL collection schema. An index definition can include:
-
Terms: Document fields for exact match searches
-
Values: Document fields for sorting and range searches
You can only index persistable field values.
An index definition must include at least one term or value. A collection can include multiple index definitions:
collection Product {
...
// Defines the `byName()` index.
index byName {
// `terms` are document fields for exact match searches.
// In this example, you get `Product` collection documents
// by their `name` field value.
terms [.name]
// `values` are document fields for sorting and range searches.
// In this example, you sort or filter index results by their
// descending `stock` field value.
values [desc(.stock)]
}
...
}
You manage and submit schema to Fauna using the Fauna Dashboard or Fauna CLI.
FQL index definitions
Fauna stores schema for user-defined collections as documents in the
Collection
system collection.
Collection
documents include an FQL version of the collection’s index
definitions:
{
name: "Customer",
coll: Collection,
ts: Time("2099-10-03T20:45:53.780Z"),
...
indexes: {
byEmail: {
terms: [
{
field: ".email",
mva: false
}
],
values: [
{
field: ".email",
order: "desc",
mva: false
},
{
field: ".name",
order: "asc",
mva: false
}
],
queryable: true,
status: "complete"
},
},
...
}
You can use Collection methods to access and manage index definitions in FQL.
Index builds
When you submit a new or updated collection schema, Fauna may need to build (or rebuild) the collection’s indexes. Fauna builds an index if:
-
The index definition is new.
-
The index definition is updated, including:
-
Adding a new index term or index value
-
Changing the order of existing index terms or index values
-
-
The definition, or function body, of a computed field covered by the index changes. Changing the name of a computed field does not trigger a rebuild.
-
A field that has a field definition and is covered by the index is migrated in a way that affects the field’s values. Other migrations, such as renaming a field, don’t require a rebuild.
If the collection contains more than 128 documents, Fauna uses a background task to build the index. During a build, the index may not be queryable.
If the collection contains 128 or fewer documents, Fauna builds the index in the same transaction as the collection update. The index is immediately queryable.
Monitor index builds
The FQL version of the collection schema, stored as a
Collection
document, includes status
and queryable
properties for indexes
objects.
You can use these properties to monitor the availability of an index during a
build:
// Gets the FQL definition of the `Product` collection schema.
// Projects the `indexes` object from the schema.
Collection.byName("Product") { indexes }
{
indexes: {
// FQL version of the `byCategory` index definition.
byEmail: {
terms: [
{
field: ".email",
mva: false
}
],
values: [
{
field: ".email",
order: "desc",
mva: false
},
{
field: ".name",
order: "asc",
mva: false
}
],
queryable: true, // Indicates the index is queryable.
status: "complete" // Indicates the index build is complete.
},
...
}
}
Call an index
In an FQL query, you call an index as a method on its collection:
// Call the `byName()` index to fet `Product` collection
// documents with a `name` value of `limes`. Values must
// match exactly.
Product.byName("limes")
The call returns a Set of matching collection documents.
Reference: FQL index method docs |
---|
Terms
You can use index terms to run exact match searches on document field values.
Use index terms for exact match search
The following index definition includes name
as an index term:
collection Product {
...
index byName {
terms [.name]
}
...
}
When you call the index, you must pass an argument for each term in the index definition.
// Get products named "limes"
Product.byName("limes")
The call returns a Set of Product
collection documents with a name
of
limes
.
Pass multiple index terms
The following index definition includes two index terms:
collection Customer {
...
index byName {
terms [.firstName, .lastName]
}
}
In an index call, use a comma to separate term arguments. Provide arguments in the same field order used in the index definition.
// Get customers named "Alice Appleseed"
Customer.byName("Alice", "Appleseed")
The call returns a Set of matching collection documents.
Best practices for index terms
Avoid using frequently updated fields as index terms.
Internally, Fauna partitions indexes based on its terms, if present. Frequent updates to term field values trigger updates to these partitions.
If you need to filter or run an exact match search on a frequently updated field, consider adding the field as an index value instead:
collection Product {
...
// Defines the `sortedByName()` index.
// The index includes the `name` field as an index value.
// `name` is a frequently updated field.
index sortedByName {
values [.name, .description, .price]
}
}
Then use the index to run a range search on the index value:
// Uses the `sortedByName()` index to run a range search
// on `name` field values. The query only retrieves `Product`
// collection documents with a `name` of `limes`. The query
// is covered and avoids document reads.
Product.sortedByName({ from: "limes", to: "limes" }) {
name,
description,
price
}
Values
You can use index values to sort a collection’s documents. You can also use index values for range searches.
Sort collection documents
The following index definition includes several index values:
collection Product {
...
index sortedByPriceLowToHigh {
values [.price, .name, .description]
}
}
Call the sortedByPriceLowToHigh()
index with no arguments to return Product
documents sorted by:
-
Ascending
price
, then … -
Ascending
name
, then … -
Ascending
description
, then … -
Ascending
id
(default)
// Get products by ascending price, name, and description
Product.sortedByPriceLowToHigh()
Descending order
By default, index values sort results in ascending order. To use descending
order, use desc()
in the index definition:
collection Product {
...
index sortedByPriceHighToLow {
values [desc(.price), .name, .description]
}
...
}
Call the index with no arguments to return Product
documents sorted by:
-
Descending
price
, then … -
Ascending
name
, then … -
Ascending
description
, then … -
Ascending
id
(default)
// Get products by descending price,
// ascending name, and ascending description
Product.sortedByPriceHighToLow()
Run a range search
You can also use index values for range searches.
The following index definition includes several index values:
collection Product {
...
index sortedByPriceLowToHigh {
values [.price, .name, .description]
}
}
The index specifies price
as its first value. The following query passes an
argument to run a range search on price
:
// Get products with a price between
// 20_00 (inclusive) and 30_00 (inclusive)
Product.sortedByPriceLowToHigh({ from: 20_00, to: 30_00 })
If an index value uses descending order, pass the higher value in from
:
// Get products with a price between
// 20_00 (inclusive) and 30_00 (inclusive) in desc order
Product.sortedByPriceHighToLow({ from: 30_00, to: 20_00 })
Omit from
or to
to run unbounded range searches:
// Get products with a price greater than or equal to 20_00
Product.sortedByPriceLowToHigh({ from: 20_00 })
// Get products with a price less than or equal to 30_00
Product.sortedByPriceLowToHigh({ to: 30_00 })
Pass multiple index values
Use an Array to pass multiple value arguments. Pass the arguments in the same field order used in the index definition.
Product.sortedByPriceLowToHigh({ from: [ 20_00, "l" ], to: [ 30_00, "z" ] })
The index returns any document that matches the first value in the from
and
to
Arrays. If matching documents have the same values, they are compared
against the next Array element value, and so on.
For example, the Product
collection’s sortedByPriceLowToHigh()
index covers
the price
and name
fields as index values. The Product
collection contains
two documents:
Document |
|
|
Doc1 |
|
|
Doc2 |
|
|
The following query returns both Doc1 and Doc2, in addition to other matching documents:
Product.sortedByPriceLowToHigh({ from: [4_99, "p"] })
The first value (4_99
and 6_98
) of each document matches the first value
(4_99
) of the from
Array.
Later, you update the document values to:
Document | price |
name |
---|---|---|
Doc1 |
|
|
Doc2 |
|
|
The following query no longer returns Doc2:
Product.sortedByPriceLowToHigh({ from: [4_99, "p"] })
Although the first value (4_99
) in both documents matches the first value in
the from
Array, the second value (cups
) in Doc2 doesn’t match the second
value (p
) of the from
Array.
Run a range search on id
All indexes implicitly include an ascending document id
as the index’s last
value.
If you intend to run range searches on id
, we recommend you explicitly include
an ascending id
as the last index value in the index definition, even if you
have an otherwise identical index.
For example, the following sortByStock()
and sortByStockandId()
indexes have the same values:
collection Product {
...
index sortByStock {
values [.stock]
}
index sortByStockandId {
values [.stock, .id]
}
...
}
Although it’s not explicitly listed, sortByStock()
implicitly includes an
ascending id
as its last value.
To reduce your costs, Fauna only builds the sortByStock()
index. When a
query calls the sortByStockandId()
index, Fauna uses the
sortByStock()
index behind the scenes. sortByStockandId()
only acts as
a virtual index and isn’t
materialized.
Pass terms and values
If an index has both terms and values, you can run an exact match search on documents in a provided range.
The following index definition includes name
as an index term and
stock
as an index value:
collection Product {
...
index byName {
terms [.name]
values [.stock]
}
...
}
When you call the index, you must provide a term and can specify an optional range:
// Get products named "donkeypinata"
// with a stock between 10 (inclusive) and 50 (inclusive)
Product.byName("donkey pinata", { from: 10, to: 50 })
Index an Array field
By default, Fauna assumes index term and value field contain scalar values.
Use mva()
to index an Array field’s values:
collection Product {
...
index byCategory {
// `categories` is an Array field.
terms [mva(.categories)]
}
index sortedByCategory {
// `categories` is an Array field.
values [mva(.categories)]
// You can combine `mva()` with
// `desc()` and `asc()`. Ex:
// values [desc(mva(.categories))]
}
...
}
mva()
only works on the last item in the provided field accessor. For more
complex nested Arrays, such as an object Array, use a
computed field:
collection Order {
...
// `Order` collection documents
// have the following structure:
// {
// customer: Customer("<CUSTOMER_DOC_ID>"),
// items: [
// {
// product: Product("PRODUCT_DOC_ID"),
// quantity: 10
// },
// ...
// ],
// ...
// }
// Defines the `quantities` computed field.
// The field uses `map()` to extract `price` values
// from the `cart` Array's object to a flat Array.
compute quantities = (.items.map(item => item.quantity))
// Uses `mva()` to index the computed `quantities` field.
index byQuantities {
terms [mva(.quantities)]
}
...
}
Covered queries
If you project an index’s covered term or value fields, Fauna gets the field values from the index.
The following index definition includes several index values:
collection Product {
...
index sortedByPriceLowToHigh {
values [.price, .name, .description]
}
}
The following query is a covered query.
// This is a covered query.
// `name`, `description`, and `prices` are values
// in the `sortedByPriceLowToHigh()` index definition.
Product.sortedByPriceLowToHigh() {
name,
description,
price
}
If the projection contains an uncovered field, Fauna must retrieve the field values from the documents. This is an uncovered query.
// This is an uncovered query.
// `stock` is not one of the terms or values
// in the `sortedByPriceLowToHigh()` index definition.
Product.sortedByPriceLowToHigh() {
stock,
name
}
Similarly, a query without a projection is uncovered:
// This is an uncovered query.
// Queries without a projection
// require a document read.
Product.sortedByPriceLowToHigh()
Covered queries are typically faster and less expensive than uncovered queries, which require document reads. If you frequently run uncovered queries, consider adding the uncovered fields to the index definition.
Index document relationships
When you index a field that contains a document, you index a document reference. The reference consists of the document’s collection and document ID:
collection Product {
...
// The `category` field contains a reference to
// a `Category` collection document.
category: Ref<Category>
...
// Indexes the `category` field as an index term.
// The index stores `Category` document references.
// Example reference: Category("123")
index byCategory {
terms [.category]
}
}
An index can’t store a referenced document’s fields. An index also can’t store a computed field that references another document. See Patterns to avoid.
Missing or null values
-
Terms: If an index definition contains terms, Fauna doesn’t index a document if all its index terms are missing or otherwise evaluate to null. This applies even if the document contains index values.
-
Values: If an index definition contains only values, Fauna indexes all documents in the collection, regardless of whether the document’s index values are missing or otherwise null.
Partitions
When an index has one or more terms, the index is internally partitioned by its terms. Partitioning lets Fauna scale indexes efficiently.
Virtual indexes
To reduce your costs, Fauna doesn’t build duplicate indexes that have the same terms and values. Instead, Fauna only builds a single index and internally points any duplicates to the single index.
For example, in the following collection, the byDescription()
and byDesc()
indexes are duplicates:
collection Product {
...
index byDescription {
terms [.description]
}
index byDesc {
terms [.description]
}
}
When a query calls the byDesc()
index, Fauna uses the existing
byDescription()
index internally. byDesc()
is considered a virtual index and
is never materialized.
Document history
To support temporal queries, indexes cover field values from both current documents and their historical document snapshots.
To enable quicker sorting and
range searches, current and
historical index entries are stored together, sorted by index values
. All
indexes implicitly include an ascending
document id
as the
index’s last value.
When you read data from an index, including the
collection.all()
index,
Fauna must read from both current and historical index entries to determine if
they apply to the query. Fauna then filters out any data not returned by the
query.
You are charged for any Transactional Read Operations (TROs) used to read current or historical index data, including data not returned by the query.
You are not charged for any historical data older than the retention period set
by the history_days
setting.
Is this article helpful?
Tell Fauna how the article can be improved:
Visit Fauna's forums
or email docs@fauna.com
Thank you for your feedback!