CreateIndex

CreateIndex( param_object )
CreateIndex( param_object )
CreateIndex( param_object )
CreateIndex( param_object )
create_index( param_object )
CreateIndex( param_object )

Description

The CreateIndex function adds a new index to the database with the specified parameters. After the transaction containing the CreateIndex is completed, the index is immediately available for reads. (The index may not be used in the transaction it was created, and it may not be created in the same transaction as its source collection(s).) The index may return incomplete results until it is fully built and marked as active. Fauna builds the index asynchronously by scanning over relevant documents of the source collection(s).

When an index is created, documents are immediately indexed if the associated collection contains up to 128 events (which include document creation, updates, and deletions). When an index contains all of a collection’s documents, including each document’s events, it is an "active" index.

For collections with more than 128 events (which include document creation, updates, and deletions), indexing is handled by a background task, and you may have to wait a short period before the index returns values. Until the indexing task is complete, the index is an "inactive" index.

To check whether an index is "active", run the following query (replacing index_name with the name of the index that you want to check):

Select("active", Get(Index("index_name")))

If you see true in the output, the index is "active" and is ready for your queries. Otherwise, you should wait and check again later, until true appears in the output.
The maximum size of an index entry, which is comprised of the terms and values content (and some overhead to distinguish multiple fields), must not exceed 64k bytes. If an index entry is too large, the query that created/updated the index entry fails.

Updating index fields

  • It is possible to rename an index by updating its name field. Renaming an index changes its reference, but preserves inbound references to the index. Index data is not rebuilt.

  • If you update the unique field, it does not remove existing duplicated items from the index, but does prevent future duplicates from being added.

  • The fields of an index which may not be modified are terms and values.

Unique name required

The index name must be unique within the database. If you try to lookup an index by name and only create it if it does not exist, the operation might fail if another transaction created the index in the meantime.

Deleting an index

When an index is deleted, it becomes inaccessible, and its data is deleted asynchronously.

Parameters

Parameter Type Definition and Requirements

param_object

Object

The param_object fields are described below.

param_object

Field Name Field Type Definition and Requirements

name

String

The logical name of the index.

Cannot be events, sets, self, documents, or _. Cannot have the % character.

source

Reference or Array

A Collection reference, or an array of one or more source objects describing source collections and (optional) binding fields.

terms

Array

Optional - An array of Term objects describing the fields that should be searchable. Indexed terms can be used to search for field values, via the Match function. The default is an empty Array.

values

Array

Optional - An array of Value objects describing the fields that should be reported in search results. The default is an empty Array. When no values fields are defined, search results report the indexed document’s Reference.

unique

Boolean

Optional - If true, maintains a unique constraint on combined terms and values. The default is false.

serialized

Boolean

Optional - If true, writes to this index are serialized with concurrent reads and writes. Subsequent reads or writes must wait until this index’s writes for covered documents have completed.

If false, writes to this index occur asynchronously and do not block subsequent reads or writes. This is better performance-wise, but the lack of serialization makes it notably more difficult to use this index to read your own writes.

The default is true.

See Isolation levels for details.

permissions

Object

Optional - Indicates who is allowed to read the index. The default is everyone can read the index.

data

Object

Optional - This is user-defined metadata for the index. It is provided for the developer to store information at the index level. The default is an empty object having no data.

ttl

Timestamp

Optional - A timestamp indicating the time-to-live for a document, which is when the document should be removed. When a document is removed, its existence ceases (as if it never existed), and temporal queries cannot recover the document.

ttl is valid for documents in user-created collections, and for Keys and Tokens. ttl is ignored for documents in other system collections.

Removal is handled by a background task, so once a document (including collections, databases, indexes, keys, roles, and tokens) "expires" due to the setting in the ttl field, it could be some time (hours or days) before the removal occurs. There is no guarantee that removal occurs.

As of version 3.0.0, the ttl field is honored on read — an expired but not yet removed document behaves as if removal is complete. But, until removal is completed by background task processing, you can continue to access the history of the document, provided you have its reference, using the Events function.

On January 23, 2023, the behavior of the document ttl field is changing. See Temporality for details.

Returns

An object containing the metadata about the CreateIndex operations.

Field Name Field Type Definition and Requirements

ref

Reference

The Reference to the index that was created in the database.

name

String

The name of the index created.

source

Array

The source and binding of the newly created index.

ts

Long

The timestamp, with microsecond resolution, associated with the creation of the index.

active

Boolean

Indicates if the index has completed building and is fully operational.

partitions

Integer

The number of sub-partitions used by this index. This value can be 8 or 1:

  • 1 when unique is true.

  • 8 when the index has no terms.

  • 1 in all other cases.

Examples

Create an index with terms and values

The following example creates an index called spells_by_element_with_name for the collection spells.

The index includes the element field in its terms definition, which means that you can use this index to search for documents in the spells collection by values in the documents' element field.

The index also includes the name field in its values definition, meaning that matching entries from this index are sorted by the values of the indexed documents' name field.

try
{
    Value result = await client.Query(
        CreateIndex(
            Obj(
                "name", "spells_by_element_with_name",
                "source", Collection("spells"),
                "terms", Arr(
                    Obj("field", Arr("data", "element"))
                ),
                "values", Arr(
                    Obj("field", Arr("data", "name"))
                )
            )
        )
    );
    Console.WriteLine(result);
}
catch (Exception e)
{
    Console.WriteLine($"ERROR: {e.Message}");
}
ObjectV(ref: RefV(id = "spells_by_element_with_name", collection = RefV(id = "indexes")),ts: LongV(1655934674090000),active: BooleanV(True),serialized: BooleanV(True),name: StringV(spells_by_element_with_name),source: RefV(id = "spells", collection = RefV(id = "collections")),terms: Arr(ObjectV(field: Arr(StringV(data), StringV(element)))),values: Arr(ObjectV(field: Arr(StringV(data), StringV(name)))),partitions: LongV(1))
result, err := client.Query(
	f.CreateIndex(
		f.Obj{
			"name":   "spells_by_element_with_name",
			"source": f.Collection("spells"),
			"terms": f.Arr{
				f.Obj{"field": f.Arr{"data", "element"}},
			},
			"values": f.Arr{
				f.Obj{"field": f.Arr{"data", "name"}},
			},
		}))

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[active:true name:spells_by_element_with_name partitions:1 ref:{spells_by_element_with_name 0x140000924b0 0x140000924b0 <nil>} serialized:true source:{spells 0x140000925a0 0x140000925a0 <nil>} terms:[map[field:[data element]]] ts:1657754535190000 values:[map[field:[data name]]]]
System.out.println(
    client.query(
        CreateIndex(
            Obj(
                "name", Value("spells_by_element_with_name"),
                "source", Collection("spells"),
                "terms", Arr(
                    Obj("field", Arr(Value("data"), Value("element")))
                ),
                "values", Arr(
                    Obj("field", Arr(Value("data"), Value("name")))
                )
            )
        )
    ).get());
{ref: ref(id = "spells_by_element_with_name", collection = ref(id = "indexes")), ts: 1655935684020000, active: true, serialized: true, name: "spells_by_element_with_name", source: ref(id = "spells", collection = ref(id = "collections")), terms: [{field: ["data", "element"]}], values: [{field: ["data", "name"]}], partitions: 1}
client.query(
  q.CreateIndex({
    name: 'spells_by_element_with_name',
    source: q.Collection('spells'),
    terms: [
      { field: ['data', 'element'] },
    ],
    values: [
      { field: ['data', 'name'] },
    ],
  })
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  ref: Index("spells_by_element_with_name"),
  ts: 1655934099560000,
  active: true,
  serialized: true,
  name: 'spells_by_element_with_name',
  source: Collection("spells"),
  terms: [ { field: [ 'data', 'element' ] } ],
  values: [ { field: [ 'data', 'name' ] } ],
  partitions: 1
}
result = client.query(
  q.create_index({
    "name": "spells_by_element_with_name",
    "source": q.collection("spells"),
    "terms": [
      { "field": ["data", "element"] }
    ],
    "values": [
      { "field": ["data", "name"] }
    ],
  })
)
print(result)
{'ref': Ref(id=spells_by_element_with_name, collection=Ref(id=indexes)), 'ts': 1655935888700000, 'active': True, 'serialized': True, 'name': 'spells_by_element_with_name', 'source': Ref(id=spells, collection=Ref(id=collections)), 'terms': [{'field': ['data', 'element']}], 'values': [{'field': ['data', 'name']}], 'partitions': 1}
CreateIndex({
  name: 'spells_by_element_with_name',
  source: Collection('spells'),
  terms: [
    { field: ['data', 'element'] }
  ],
  values: [
    { field: ['data', 'name'] }
  ]
})
{
  ref: Index("spells_by_element_with_name"),
  ts: 1655934099560000,
  active: true,
  serialized: true,
  name: 'spells_by_element_with_name',
  source: Collection("spells"),
  terms: [ { field: [ 'data', 'element' ] } ],
  values: [ { field: [ 'data', 'name' ] } ],
  partitions: 1
}
Query metrics:

  •    bytesIn:   197

  •   bytesOut:   364

  • computeOps:     1

  •    readOps:     0

  •   writeOps:     5

  •  readBytes: 1,905

  • writeBytes: 1,596

  •  queryTime:   9ms

  •    retries:     0

To use this index to search for all spells with the water element:

try
{
    Value result = await client.Query(
        Paginate(
            Match(Index("spells_by_element_with_name"), "water")
        )
    );
    Console.WriteLine(result);
}
catch (Exception e)
{
    Console.WriteLine($"ERROR: {e.Message}");
}
ObjectV(data: Arr(StringV(Hippo's Wallow), StringV(Water Dragon's Claw)))
result, err := client.Query(
	f.Paginate(
		f.MatchTerm(f.Index("spells_by_element_with_name"), "water"),
	))

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[data:[Hippo's Wallow Water Dragon's Claw]]
System.out.println(
    client.query(
        Paginate(
            Match(Index("spells_by_element_with_name"), Value("water"))
        )
    ).get());
{data: ["Hippo's Wallow", "Water Dragon's Claw"]}
client.query(
  q.Paginate(q.Match(q.Index('spells_by_element_with_name'), 'water'))
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{ data: [ "Hippo's Wallow", "Water Dragon's Claw" ] }
result = client.query(
  q.paginate(q.match(q.index("spells_by_element_with_name"), "water"))
)
print(result)
{'data': ["Hippo's Wallow", "Water Dragon's Claw"]}
Paginate(Match(Index("spells_by_element_with_name"), "water"))
{ data: [ "Hippo's Wallow", "Water Dragon's Claw" ] }
Query metrics:

  •    bytesIn:  78

  •   bytesOut:  62

  • computeOps:   1

  •    readOps:   1

  •   writeOps:   0

  •  readBytes: 141

  • writeBytes:   0

  •  queryTime: 2ms

  •    retries:   0

Create a collection index

The following example creates a "collection" index — an index with no terms or values defined — called new-index for the collection spells:

try
{
    Value result = await client.Query(
        CreateIndex(
            Obj(
                "name", "new-index",
                "source", Collection("spells")
            )
        )
    );
    Console.WriteLine(result);
}
catch (Exception e)
{
    Console.WriteLine($"ERROR: {e.Message}");
}
ObjectV(ref: RefV(id = "new-index", collection = RefV(id = "indexes")),ts: LongV(1603756181200000),active: BooleanV(True),serialized: BooleanV(True),name: StringV(new-index),source: RefV(id = "spells", collection = RefV(id = "collections")),partitions: LongV(8))
result, err := client.Query(
	f.CreateIndex(
		f.Obj{
			"name":   "new-index",
			"source": f.Collection("spells"),
		},
	))

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[active:true name:new-index partitions:8 ref:{new-index 0x1400011d950 0x1400011d950 <nil>} serialized:true source:{spells 0x1400011da40 0x1400011da40 <nil>} ts:1657754502810000]
System.out.println(
    client.query(
        CreateIndex(
            Obj(
                "name", Value("new-index"),
                "source", Collection("spells")
            )
        )
    ).get());
{ref: ref(id = "new-index", collection = ref(id = "indexes")), ts: 1593464646510000, active: true, serialized: true, name: "new-index", source: ref(id = "spells", collection = ref(id = "collections")), partitions: 8}
client.query(
  q.CreateIndex({
    name: 'new-index',
    source: q.Collection('spells'),
  })
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  ref: Index("new-index"),
  ts: 1591996190530000,
  active: true,
  serialized: true,
  name: 'new-index',
  source: Collection("spells"),
  partitions: 8
}
result = client.query(
  q.create_index({
    "name": "new-index",
    "source": q.collection("spells")
  })
)
print(result)
{'ref': Ref(id=new-index, collection=Ref(id=indexes)), 'ts': 1592856422060000, 'active': True, 'serialized': True, 'name': 'new-index', 'source': Ref(id=spells, collection=Ref(id=collections)), 'partitions': 8}
CreateIndex({
  name: 'new-index',
  source: Collection('spells'),
})
{
  ref: Index("new-index"),
  ts: 1624310362210000,
  active: true,
  serialized: true,
  name: 'new-index',
  source: Collection("spells"),
  partitions: 8
}
Query metrics:

  •    bytesIn:    81

  •   bytesOut:   252

  • computeOps:     1

  •    readOps:     0

  •   writeOps:     5

  •  readBytes: 1,825

  • writeBytes:   855

  •  queryTime:   9ms

  •    retries:     0

Since the introduction of the Documents function, we no longer recommend the creation of collection indexes: they provide no advantage, yet incur write ops.

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!