Indexes

Indexes allow for the organization and retrieval of documents by attributes other than their Reference. They act as a lookup table that improves the performance of finding documents: instead of reading every single document to find the one(s) that you are interested in, you query an index to find those documents.

See Limits for details on concurrent index builds and transaction limits. See the CreateIndex reference page for limitations on what an index may be named.

Source

When you create an index, you specify its source, which is one or more collections of documents. Once the index is active, any query that creates, updates, or deletes a document in the source collection(s) causes the index to be updated.

Terms

An index can specify terms: these are zero or more values from indexed documents that help you to find specific documents. terms are comparable to column=value predicates in an SQL WHERE clause. For example, if your documents contain a name field, you can define terms to include that field, and then you can find all of the documents that match a specific name.

When an index has one or more terms, the index is partitioned by the terms, allowing Fauna to efficiently scale indexes.

When a document is indexed, and all of the index’s defined terms evaluate to null, no index entry is stored for the document.

Values

An index can specify values: these are zero or more values returned for each index entry that matches the terms when you query the index. values are comparable to the SQL SELECT clause.

values are also how indexes are sorted: each field value in values is sorted lexically according to the field’s type, and the order can be inverted by specifying reverse.

Each index entry records the Reference of each document involved in the index. When no values are specified, the index returns the Reference for each matching index entry. When one or more values are specified, only those values are returned.

When a document is indexed, and all of the index’s defined values evaluate to null, no index entry is stored for the document.

Collection index

An index with no terms and values specified is known as a collection index: searching for specific documents is not possible, and all documents within the collection are included in the result set, and are sorted by their reference in ascending order.

Unique

You can specify that an index is unique. This means that, for the defined terms and values, the index contains only one entry for a document having those specific terms and values. As a result, creating or updating a document to have the same terms and values as an existing document would cause an error.

Bindings

Fauna indexes can also specify bindings, which are Lambda functions that you can use to compute fields in the terms or values. For example, if your documents store a timestamp, and you want to search for documents by year, you could write a binding that converts the timestamp to a year and include the computed year as one of the terms. Similarly, if you want to report the month of a document, you could write a binding that converts the timestamp to a month, and include the computed month as one of the values.

For background, a Set is a sorted group of immutable data from a collection. An Index is a group of sets within a collection. Indexes are defined as documents within the system indexes collection.

Index details

This section covers the following topics:

Example index

The simplest index is called a "collection" index; it has no terms or values defined. This means that the index includes all documents with no search terms, and that the index returns the Reference to each indexed document. Such an index can be created with just a name and a source collection:

try
{
    Value result = await client.Query(
        CreateIndex(
            Obj("name", "new-index", "source", Collection("spells"))
        )
    );
    Console.WriteLine(result);
}
catch (Exception e)
{
    Console.WriteLine($"ERROR: {e.Message}");
}
ObjectV(ref: RefV(id = "new-index", collection = RefV(id = "indexes")),ts: LongV(1603756181200000),active: BooleanV(True),serialized: BooleanV(True),name: StringV(new-index),source: RefV(id = "spells", collection = RefV(id = "collections")),partitions: LongV(8))
result, err := client.Query(
	f.CreateIndex(
		f.Obj{
			"name": "new-index",
			"source": f.Collection("decrepit_huts"),
		}))

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[active:true name:new-index partitions:8 ref:{new-index 0xc00016a0c0 0xc00016a0c0 <nil>} serialized:true source:{decrepit_huts 0xc00016a1b0 0xc00016a1b0 <nil>} ts:1603747135990000]
System.out.println(
    client.query(
        CreateIndex(
            Obj(
                "name", Value("new-index"),
                "source", Collection("spells")
            )
        )
    ).get());
{ref: ref(id = "new-index", collection = ref(id = "indexes")), ts: 1593464646510000, active: true, serialized: true, name: "new-index", source: ref(id = "spells", collection = ref(id = "collections")), partitions: 8}
client.query(
  q.CreateIndex({
    name: 'new-index',
    source: q.Collection('spells'),
  })
)
.then((ret) => console.log(ret))
.catch((err) => console.error('Error: %s', err))
{
  ref: Index("new-index"),
  ts: 1591996190530000,
  active: true,
  serialized: true,
  name: 'new-index',
  source: Collection("spells"),
  partitions: 8
}
result = client.query(
  q.create_index({
    "name": "new-index",
    "source": q.collection("spells")
  })
)
print(result)
{'ref': Ref(id=new-index, collection=Ref(id=indexes)), 'ts': 1592856422060000, 'active': True, 'serialized': True, 'name': 'new-index', 'source': Ref(id=spells, collection=Ref(id=collections)), 'partitions': 8}
try {
  println(Await.result(
    client.query(
  CreateIndex(
    Obj("name" -> "new-index", "source" -> Collection("spells")))),
    5.seconds
  ))
} catch {
  case unknown: Throwable => println("Error: " + unknown.getMessage())
}
{name: "new-index", source: ref(id = "spells", collection = ref(id = "collections")), ts: 1594485038640000, ref: ref(id = "new-index", collection = ref(id = "indexes")), partitions: 8, serialized: true, active: true}
CreateIndex({
  name: 'new-index',
  source: Collection('spells'),
})
{
  ref: Index("new-index"),
  ts: 1624310362210000,
  active: true,
  serialized: true,
  name: 'new-index',
  source: Collection("spells"),
  partitions: 8
}

Index fields

Field Name Field Type Definition and Requirements

name

The logical name of the index.

Cannot be events, sets, self, documents, or _. Cannot contain the % character.

source

A Collection reference, or an array of one or more source objects describing source collections and (optional) binding fields.

terms

Optional - An array of Term objects describing the fields that should be searchable. Indexed terms can be used to search for field values, via the Match function. The default is an empty Array.

values

Optional - An array of Value objects describing the fields that should be reported in search results. The default is an empty Array. When no values fields are defined, search results report the indexed document’s Reference.

unique

Optional - If true, maintains a unique constraint on combined terms and values. The default is false.

serialized

Optional - If true, writes to this index are serialized with concurrent reads and writes. The default is true.

permissions

Optional - Indicates who is allowed to read the index. The default is everyone can read the index.

data

Optional - This is user-defined metadata for the index. It is provided for the developer to store information at the index level. The default is an empty object having no data.

The maximum size of an index entry, which is comprised of the terms and values content (and some overhead to distinguish multiple fields), must not exceed 64k bytes. If an index entry is too large, the query that created/updated the index entry fails.

Source objects

Source objects describe the source collection of index entries and, optionally, bindings. A binding must be a pure Lambda function (it must not create side effects, such as additional reads or writes) that emits values to be used as a term and/or value.

An index cannot be created in the same transaction that creates its source collection(s).

The collection field can be a single collection reference or set of references. Documents within collections matching the collection field apply the associated bindings to be used in the index’s terms or values. A collection reference can only exist in one source object. If the collection field is a wildcard (_), the index evaluates all collections. Bindings associated with a wildcard are only used if the collection is not matched by any other source object.

Field Type Definition and Requirements

collection

The collection or collections to be indexed, or a wildcard (_).

fields

An object mapping a binding’s name to a Lambda function.

The following examples demonstrates the structure of a source object, which includes an example binding object:

try
{
    Value result = await client.Query(
        Obj(
            "source", Obj(
                "collection", Collection("collection"),
                "fields", Obj(
                    "binding1", Query(
                        Lambda(
                            "document",
                            Select(Arr("data", "field"), Var("document"))
                        )
                    )
                )
            )
        )
    );
    Console.WriteLine(result);
}
catch (Exception e)
{
    Console.WriteLine($"ERROR: {e.Message}");
}
ObjectV(source: ObjectV(collection: RefV(id = "collection", collection = RefV(id = "collections")),fields: ObjectV(binding1: QueryV(System.Collections.Generic.Dictionary`2[System.String,FaunaDB.Query.Expr]))))
result, err := client.Query(
	f.Obj{
		"source": f.Obj{
			"collection": f.Collection("collection"),
			"fields": f.Obj{
				"binding1": f.Query(
					f.Lambda(
						"document",
						f.Select(f.Arr{"data", "field"}, f.Var("document"))))}}})

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[source:map[collection:{collection 0xc00007fe30 0xc00007fe30 <nil>} fields:map[binding1:{[123 34 97 112 105 95 118 101 114 115 105 111 110 34 58 34 52 34 44 34 108 97 109 98 100 97 34 58 34 100 111 99 117 109 101 110 116 34 44 34 101 120 112 114 34 58 123 34 115 101 108 101 99 116 34 58 91 34 100 97 116 97 34 44 34 102 105 101 108 100 34 93 44 34 102 114 111 109 34 58 123 34 118 97 114 34 58 34 100 111 99 117 109 101 110 116 34 125 125 125]}]]]
System.out.println(
    client.query(
        Obj(
            "source", Obj(
                "collection", Collection("collection"),
                "fields", Obj(
                    "binding1", Query(
                        Lambda(
                            "document",
                            Select(
                                Arr(Value("data"), Value("field")),
                                Var("document")
                            )
                        )
                    )
                )
            )
        )
    ).get());
{source: {collection: ref(id = "collection", collection = ref(id = "collections")), fields: {binding1: QueryV({api_version=4, lambda=document, expr={select=[data, field], from={var=document}}})}}}
client.query({
  source: {
    collection: q.Collection('collection'),
    fields: {
      binding1: q.Query(
        q.Lambda(
          'document',
          q.Select(['data', 'field'], q.Var('document'))
        )
      ),
    },
  },
})
.then((ret) => console.log(ret))
.catch((err) => console.error('Error: %s', err))
{
  source: {
    collection: Collection("collection"),
    fields: {
      binding1: Query(Lambda("document", Select(["data", "field"], Var("document"))))
    }
  }
}
result = client.query(
  {
    "source": {
      "collection": q.collection("collection"),
      "fields": {
        "binding1": q.query(
          q.lambda_(
            "document",
            q.select(["data", "field"], q.var("document"))
          )
        )
      }
    }
  }
)
print(result)
{'source': {'collection': Ref(id=collection, collection=Ref(id=collections)), 'fields': {'binding1': Query({'api_version': '4', 'lambda': 'document', 'expr': {'select': ['data', 'field'], 'from': {'var': 'document'}}})}}}
try {
  println(Await.result(
    client.query(
      Obj(
        "source" -> Obj(
          "collection" -> Collection("collection"),
          "fields" -> Obj(
            "binding1" -> Query(
              Lambda(
                "document",
                Select(Arr("data", "field"), Var("document"))
              )
            )
          )
        )
      )
    ),
    5.seconds
  ))
} catch {
  case unknown: Throwable => println("Error: " + unknown.getMessage())
}
{source: {collection: ref(id = "collection", collection = ref(id = "collections")), fields: {binding1: {api_version: "4", lambda: "document", expr: {select: ["data", "field"], from: {var: "document"}}}}}}
{
  source: {
    collection: Collection("collection"),
    fields: {
      binding1: Query(
        Lambda(
          "document",
          Select(["data", "field"], Var("document")),
        )
      ),
    },
  },
}
{
  source: {
    collection: Collection("collection"),
    fields: {
      binding1: Query(Lambda("document", Select(["data", "field"], Var("document"))))
    }
  }
}

Binding objects

A binding object binds a field name to a pure, single-argument Lambda function (it must not create side effects, such as additional reads or writes). The function must take the document to be indexed and emit either a single scalar value or an array of scalar values. Binding functions are not permitted to perform reads or writes.

Once defined, bindings can be used in an index’s terms or values field definitions, as if they were document fields.

Functions that cannot be used in bindings include:

try
{
    Value result = await client.Query(
        Obj(
            "binding1", Query(
                Lambda(
                    "document",
                    Select(Arr("data", "field"), Var("document"))
                )
            )
        )
    );
    Console.WriteLine(result);
}
catch (Exception e)
{
    Console.WriteLine($"ERROR: {e.Message}");
}
ObjectV(binding1: QueryV(System.Collections.Generic.Dictionary`2[System.String,FaunaDB.Query.Expr]))
result, err := client.Query(
	f.Obj{
		"binding1": f.Query(
			f.Lambda(
				"document",
				f.Select(f.Arr{"data", "field"}, f.Var("document")),
			))})

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[binding1:{[123 34 97 112 105 95 118 101 114 115 105 111 110 34 58 34 52 34 44 34 108 97 109 98 100 97 34 58 34 100 111 99 117 109 101 110 116 34 44 34 101 120 112 114 34 58 123 34 115 101 108 101 99 116 34 58 91 34 100 97 116 97 34 44 34 102 105 101 108 100 34 93 44 34 102 114 111 109 34 58 123 34 118 97 114 34 58 34 100 111 99 117 109 101 110 116 34 125 125 125]}]
System.out.println(
    client.query(
        Obj(
            "binding1", Query(
                Lambda(
                    "document",
                    Select(
                        Arr(Value("data"), Value("field")),
                        Var("document")
                    )
                )
            )
        )
    ).get());
{binding1: QueryV({api_version=4, lambda=document, expr={select=[data, field], from={var=document}}})}
client.query({
  binding1: q.Query(
    q.Lambda('document', q.Select(['data', 'field'], q.Var('document')))
  ),
})
.then((ret) => console.log(ret))
.catch((err) => console.error('Error: %s', err))
{ binding1:
   Query(Lambda("document", Select(["data", "field"], Var("document")))) }
result = client.query(
  {
    "binding1": q.query(
      q.lambda_( "document", q.select(["data", "field"], q.var("document")))
    )
  }
)
print(result)
{'binding1': Query({'api_version': '4', 'lambda': 'document', 'expr': {'select': ['data', 'field'], 'from': {'var': 'document'}}})}
println(Await.result(
  client.query(
    Obj(
      "binding1" -> Query(
        Lambda(
          "document",
          Select(Arr("data", "field"), Var("document"))
        )
      )
    )
  ),
  5.seconds
))
{binding1: {api_version: "4", lambda: "document", expr: {select: ["data", "field"], from: {var: "document"}}}}
{
  binding1: Query(
    Lambda('document', Select(['data', 'field'], Var('document')))
  )
}
Query(Lambda("document", Select(["data", "field"], Var("document"))))

Term objects

Term objects describe the fields whose values are used to search for entries in the index.

When a terms field is missing from an indexed document, the field’s value in the index is null. If all defined terms fields evaluate to null, no index entry is stored for the document.

If no term objects are defined, passing term values to Match is not required. The resulting set contains all documents within the source collection.

When a term field is an array, one index entry per array item is created, which makes it easy to search for any one of the array’s items.

A value can be from a field in the document or a binding defined by the source object.

Field Type Definition

field

Array or String

The field name path (the list of field names required to access a specific field nested within the document structure) or field name within the document to be indexed.

binding

The name of a binding from a source object.

The following example demonstrates an index’s terms field definition with two term objects, the first specifies a binding, the second specifies a document field:

try
{
    Value result = await client.Query(
        Obj(
            "terms", Arr(
                Obj("binding", "binding1" ),
                Obj("field", Arr("data", "field"))
            )
        )
    );
    Console.WriteLine(result);
}
catch (Exception e)
{
    Console.WriteLine($"ERROR: {e.Message}");
}
ObjectV(terms: Arr(ObjectV(binding: StringV(binding1)), ObjectV(field: Arr(StringV(data), StringV(field)))))
result, err := client.Query(
	f.Obj{
		"terms": f.Arr{
			f.Obj{"binding": "binding1"},
			f.Obj{"field": f.Arr{"data", "field"}},
		},
	})

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[terms:[map[binding:binding1] map[field:[data field]]]]
System.out.println(
    client.query(
        Obj(
            "terms", Arr(
                Obj("binding", Value("binding1")),
                Obj("field", Arr(Value("data"), Value("field")))
            )
        )
    ).get());
{terms: [{binding: "binding1"}, {field: ["data", "field"]}]}
client.query({
  terms: [
    { binding: 'binding1' },
    { field: ['data', 'field'] },
  ],
})
.then((ret) => console.log(ret))
.catch((err) => console.error('Error: %s', err))
{
  terms: [ { binding: 'binding1' }, { field: [ 'data', 'field' ] } ]
}
result = client.query(
  {
    "terms": [
      { "binding": "binding1" },
      { "field": ["data", "field"] }
    ]
  }
)
print(result)
{'terms': [{'binding': 'binding1'}, {'field': ['data', 'field']}]}
try {
  println(Await.result(
    client.query(
      Obj(
        "terms" -> Arr(
          Obj("binding" -> "binding1"),
          Obj("field" -> Arr("data", "field"))
        )
      )
    ),
    5.seconds
  ))
} catch {
  case unknown: Throwable => println("Error: " + unknown.getMessage())
}
{terms: [{binding: "binding1"}, {field: ["data", "field"]}]}
{
  terms: [
    { binding: 'binding1' },
    { field: ['data', 'field'] },
  ]
}
[ { binding: 'binding1' }, { field: [ 'data', 'field' ] } ]

Value objects

Value objects describe the fields whose values should be used to sort the index, and whose values should be reported in query results. By default, indexes have no values defined, and return the references of indexed documents.

When a values field is missing from an indexed document, the field’s value in the index is null. If all defined values fields evaluate to null, no index entry is stored for the document.

A value can be from a field in the document, or a binding function defined in a Source objects.

Field Type Definition

field

Array or String

The field name path (the list of field names required to access a specific field nested within the document structure) or field name within the document to be indexed.

binding

The name of a binding from a Source objects.

reverse

Whether this field’s value should sort reversed. Defaults to false.

The document’s reference also appears in before and after cursors when paging through an index with the Paginate function, even if the reference is not included as a values field. Pagination always uses the indexed document’s reference to stabilize pagination.

Any of a document’s fields may be indexed. The value of field in a Term or Value object indicates the position within a document for a field. For example, the field ref refers to the top-level ref field. The field ["data", "address", "street"] refers to the street field contained in an address object within the document’s data object.

The following example demonstrates an index’s values field definition with two term objects, the first specifies a binding, the second specifies a document field that should be sorted in reverse:

try
{
    Value result = await client.Query(
        Obj(
            "values", Arr(
                Obj("binding", "binding1" ),
                Obj("field", Arr("data", "field"), "reverse", true)
            )
        )
    );
    Console.WriteLine(result);
}
catch (Exception e)
{
    Console.WriteLine($"ERROR: {e.Message}");
}
ObjectV(values: Arr(ObjectV(binding: StringV(binding1)), ObjectV(field: Arr(StringV(data), StringV(field)),reverse: BooleanV(True))))
result, err := client.Query(
	f.Obj{
		"values": f.Arr{
			f.Obj{"binding": "binding1"},
			f.Obj{"field": f.Arr{"data", "field"}, "reverse": true},
		},
	})

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[values:[map[binding:binding1] map[field:[data field] reverse:true]]]
System.out.println(
    client.query(
        Obj(
            "values", Arr(
                Obj("binding", Value("binding1")),
                Obj(
                    "field", Arr(Value("data"), Value("field")),
                    "reverse", Value(true)
                )
            )
        )
    ).get());
{values: [{binding: "binding1"}, {field: ["data", "field"], reverse: true}]}
client.query({
  values: [
    { binding: 'binding1' },
    { field: ['data', 'field'], reverse: true },
  ],
})
.then((ret) => console.log(ret))
.catch((err) => console.error('Error: %s', err))
{
  values: [
    { binding: 'binding1' },
    { field: [ 'data', 'field' ], reverse: true }
  ]
}
result = client.query(
  {
    "values": [
      {"binding": "binding1"},
      {"field": ["data", "field"], "reverse": True}
    ]
  }
)
print(result)
{'values': [{'binding': 'binding1'}, {'field': ['data', 'field'], 'reverse': True}]}
try {
  println(Await.result(
    client.query(
      Obj(
        "values" -> Arr(
          Obj("binding" -> "binding1"),
          Obj(
            "field" -> Arr("data", "field"),
            "reverse" -> true
          )
        )
      )
    ),
    5.seconds
  ))
} catch {
  case unknown: Throwable => println("Error: " + unknown.getMessage())
}
{values: [{binding: "binding1"}, {field: ["data", "field"], reverse: true}]}
{
  values: [
    { binding: 'binding1' },
    { field: ['data', 'field'], reverse: true },
  ]
}
[
  { binding: 'binding1' },
  { field: [ 'data', 'field' ], reverse: true }
]

To learn more about common index operations, see Index recipes.

Was this article helpful?

We're sorry to hear that.
Tell us how we can improve!
Visit Fauna's Discourse forums or email docs@fauna.com

Thank you for your feedback!