Paginate

Paginate( input, [ts], [after], [before], [size], [events], [sources] )
Paginate( input, { ["ts": timestamp], ["before": before], ["after": after], ["size": size], ["events": events], ["sources": sources] })
paginate( input, [ts], [after], [before], [size], [events], [sources] )
Paginate( input, { ["ts": timestamp], ["before": before], ["after": after], ["size": size], ["events": events], ["sources": sources] })

Description

The Paginate function takes a Set or Reference, and returns a page of results.

Pagination refers to the process of breaking a large set of results into smaller chunks, or pages, that are easier to process. This means that processing a whole result set can involve multiple queries.

A page is a list of results from input (which has an implicit sort order) that has a limited size. By default, a page has up to 64 results, but you can request up to 100,000 results. The limited page size prevents queries involving many documents from consuming enough server or client resources that overall performance is impacted.

When input is larger than the current page size, only the page size number of results are returned, but the page has one or two cursors that can be used in subsequent queries to access the following or previous pages in the result set. If there are results that follow the current page, the page has the after cursor, which points to the first entry in the next page. If there are results that come before the current page, the page has the before cursor, which points to the first entry in the current page.

The other optional parameters influence the pagination in various ways. ts can be used to specify a point in time for the results. Results newer than ts are not included. size lets you specify the number of results returned. events causes the history of the result set to be reported, rather than the entries in the set. sources causes the result set to include the source for each entry, which is useful when multiple indexes and/or collections are used.

Parameters

Parameter Type Definition and Requirements

input

Set or Reference

A Set, or Reference, to paginate.

.ts

Long or Timestamp

Optional, default current. Return the result Set at the point in time (number of UNIX microseconds or a Timestamp).

For large values of ts (positive or negative), you may need to express the value in a String to maintain numeric precision (Fauna uses 64-bit signed integers, whereas JavaScript uses 53-bit signed integers).

.before

Cursor

Optional - Return the previous Page of results before this cursor (exclusive).

.after

Cursor

Optional - Return the next Page of results after this cursor (inclusive).

.size

Integer

Optional - default 64. The maximum number of results to return in a single Page. The maximum Page size is 100,000 results.

.events

Boolean

Optional - default false. If true, return a Page from the event history of the input.

.sources

Boolean

Optional - default false. If true, includes the source of truth providing why this object is included in the result set.

The parameters ts, before, after, size, events, and sources are all optional. If provided, they must be expressed in a JavaScript object, identified here as .

Other FQL functions accept a param_object, which could be composed with functions that return Objects, such as Merge. Unlike those functions, Paginate requires that these parameters be expressed as a JavaScript object in the driver code (they are not sent to the server in an object).

This means that you cannot compose an object, assign it to a variable, and then use the Var function to pass those options to Paginate. For example, the following query does not work:

client.query(
  q.Let(
    {
      collection: 'foo',
      paginate_options: { size: 3, events: true }
    },
    q.Paginate(
      q.Documents(q.Collection(q.Var('collection'))),
      q.Var('paginate_options')
    )
  )
)
Field name Field type Definition and requirements

input

Set or Reference

A Set, or Reference, to paginate.
Field name Field type Definition and requirements

input

Set or Reference

A Set, or Reference, to paginate.

cursor

Cursor

Optional - An Object that describes the Page of results to return. The Object should have a single field, named before or after, and the value should point to the item representing the start of the previous or next Page of results.

The cursor object by using the Before or After functions:

cursor = Before(Ref(Collection("Letters"), 104))
cursor = After(Ref(Collection("Letters"), 104))

ts

Long or Timestamp

Optional - default current. Return the result set at the point in time (number of UNIX microseconds or a Timestamp).

size

Integer

Optional - default 64. Maximum number of results to return in a single Page. The maximum Page size is 100,000 results.

sources

Boolean

Optional - default false. If true, includes the source of truth providing why this object is included in the results.

events

Boolean

Optional - default false. If true, return a Page from the event history of the input.
Field name Field type Definition and requirements

input

Set or Reference

A Set, or Reference, to paginate.

ts

Long or Timestamp

Optional - default current. Return the results at the point in time (number of UNIX microseconds or a Timestamp).

after

Cursor

Optional - Return the next Page of results after this cursor (inclusive).

before

Cursor

Optional - Return the previous Page of results before this cursor (exclusive).

size

Integer

Optional - default 64. Maximum number of results to return in a single Page. The maximum Page size is 100,000 results.

events

Boolean

Optional - default false. If true, return a Page from the event history of the input.

sources

Boolean

Optional - default false. If true, includes the source of truth providing why this object is included in the results.

Returns

A "builder" object with methods to set more options. Each "builder" object method accepts one parameter which sets one specific pagination option:

Method name Parameter type Definition and requirements

ts

Long or Timestamp

Optional - default current. Return the results at the point in time (number of UNIX microseconds or a Timestamp).

after

Cursor

Optional - Return the next Page of results after this cursor (inclusive).

before

Cursor

Optional - Return the previous Page of results before this cursor (exclusive).

cursor

Cursor

Optional - An object that describes the Page of results to return. The object should have a single field, named before or after, and the value should point to the item representing the start of the previous or next Page of results.

size

Integer

Optional - default 64. The maximum number of results to return in a single Page. The maximum Page size is 100,000 results.

events

Boolean

Optional - default false. If true, return a Page from the event history of the input.

sources

Boolean

Optional - default false. If true, includes the source of truth providing why this object is included in the results.

Each method returns an updated "builder" object after setting its specific option.

When the "builder" object is "resolved," by the end of the query or by using it in a function that accepts a set or array, the pagination results are returned as a Page object.

A Page object.

Page object

A Page has an array of results and other decorated elements. In some cases the whole result set may not fit into the array, so other fields (the cursor fields) allow you to walk the results set in blocks (like pages in a book). The cursor fields retrieve blocks of results before or after the current page of results. When Pages are passed to functions that accept arrays, only the array element of the Page is examined or transformed. Other elements of the Page, such as the cursor, remain unaffected and are passed directly through to the output.

Field Type Description

data

Array

The elements in the page.

after

Cursor

The cursor for the next page, inclusive. Optional.

before

Cursor

The cursor for the previous page, exclusive. Optional.

Cursor

A cursor is an object used as a marker to indicate a position in a set. The cursor has a structure identical to the values definition of the associated index, and uses the same fields with the same types, in the same order.

Cursor matching is prefix-based. That means that if your index has two or more fields, you only need to specify the first field to achieve a match. You may have to specify more fields if there are multiple index entries that would match the first field.

A before cursor indicates the first item in a page of results. It is used to compute the page of results prior to the before cursor. The page before the current page excludes the before cursor.

An after cursor indicates the item after the last item in a page of results. It is used to compute the page of results beginning with the after cursor. The page after the current pages includes the after cursor.

The type of pagination cursor depends on whether the page is from a set, or from an events timeline. Set cursors may be a single scalar value, or an Array of values.

Events timeline cursors may be one of:

  • An Integer representing a timestamp.

  • A Timestamp.

  • A Date. Dates are interpreted as midnight on that date, in UTC.

  • A partial Event object: ts, ts and action, or all of ts, action, and resource must be provided.

It is possible to fetch the last page of a set without knowing how many pages are available by specifying a synthetic before cursor:

  • { before: null }: for indexes that have no values fields defined, or indexes where the first values field uses the default order.

  • { before: -9223372036854775808 }: for indexes where the first values field has reverse: true set.

    -9223372036854775808 is the minimum integer value (for a 64-bit signed integer). Using it guarantees that you get the last page for an index whose first values field has reverse: true set. When you are not indexing signed integers, 0 should work in most cases.

When you are using the JavaScript driver, which only supports 53-bit signed integers (instead of 64-bit), the minimum integer to use is -9007199254740991.

These cursor values work due to the sort precedence.

You can fetch the first page by not specifying a cursor.

Examples

The following query paginates the set of letters (established in the Index tutorials), but sets the size option to 3 to reduce the size of the result set:

result, err := client.Query(
	f.Paginate(
		f.Documents(f.Collection("Letters")),
		f.Size(3)))

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[after:[{104 0xc00016e180 0xc00016e180 <nil>}] data:[{101 0xc00016e330 0xc00016e330 <nil>} {102 0xc00016e4e0 0xc00016e4e0 <nil>} {103 0xc00016e690 0xc00016e690 <nil>}]]
client.query(
  q.Paginate(
    q.Documents(q.Collection('Letters')),
    { size: 3 }
  )
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{ after: [ Ref(Collection("Letters"), "104") ],
  data:
   [ Ref(Collection("Letters"), "101"),
     Ref(Collection("Letters"), "102"),
     Ref(Collection("Letters"), "103") ] }
result = client.query(
  q.paginate(
    q.documents(q.collection("Letters")),
    size=3
  )
)
print(result)
{'after': [Ref(id=104, collection=Ref(id=Letters, collection=Ref(id=collections)))], 'data': [Ref(id=101, collection=Ref(id=Letters, collection=Ref(id=collections))), Ref(id=102, collection=Ref(id=Letters, collection=Ref(id=collections))), Ref(id=103, collection=Ref(id=Letters, collection=Ref(id=collections)))]}
Paginate(
  Documents(Collection('Letters')),
  { size: 3 }
)
{
  after: [ Ref(Collection("Letters"), "104") ],
  data: [
    Ref(Collection("Letters"), "101"),
    Ref(Collection("Letters"), "102"),
    Ref(Collection("Letters"), "103")
  ]
}
Query metrics:

  •    bytesIn:    60

  •   bytesOut:   449

  • computeOps:     1

  •    readOps:     8

  •   writeOps:     0

  •  readBytes: 1,216

  • writeBytes:     0

  •  queryTime:  13ms

  •    retries:     0

The following query repeats the previous query, with the addition of applying the after cursor from the previous result to return the next page of results:

result, err := client.Query(
	f.Paginate(
		f.Documents(f.Collection("Letters")),
		f.After(f.Ref(f.Collection("Letters"), 104)),
		f.Size(3)))

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[after:[{107 0xc00009a390 0xc00009a390 <nil>}] before:[{104 0xc00009a1e0 0xc00009a1e0 <nil>}] data:[{104 0xc00009a540 0xc00009a540 <nil>} {105 0xc00009a6f0 0xc00009a6f0 <nil>} {106 0xc00009a960 0xc00009a960 <nil>}]]
client.query(
  q.Paginate(
    q.Documents(q.Collection('Letters')),
    {
      size: 3,
      after: [ q.Ref(q.Collection('Letters'), '104') ],
    }
  )
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{ before: [ Ref(Collection("Letters"), "104") ],
  after: [ Ref(Collection("Letters"), "107") ],
  data:
   [ Ref(Collection("Letters"), "104"),
     Ref(Collection("Letters"), "105"),
     Ref(Collection("Letters"), "106") ] }
result = client.query(
  q.paginate(
    q.documents(q.collection("Letters")),
    after = q.ref(q.collection("Letters"), 104),
    size=3
  )
)
print(result)
{'before': [Ref(id=104, collection=Ref(id=Letters, collection=Ref(id=collections)))], 'after': [Ref(id=107, collection=Ref(id=Letters, collection=Ref(id=collections)))], 'data': [Ref(id=104, collection=Ref(id=Letters, collection=Ref(id=collections))), Ref(id=105, collection=Ref(id=Letters, collection=Ref(id=collections))), Ref(id=106, collection=Ref(id=Letters, collection=Ref(id=collections)))]}
Paginate(
  Documents(Collection('Letters')),
  {
    size: 3,
    after: [ Ref(Collection('Letters'), '104') ],
  }
)
{
  before: [ Ref(Collection("Letters"), "104") ],
  after: [ Ref(Collection("Letters"), "107") ],
  data: [
    Ref(Collection("Letters"), "104"),
    Ref(Collection("Letters"), "105"),
    Ref(Collection("Letters"), "106")
  ]
}
Query metrics:

  •    bytesIn:   114

  •   bytesOut:   564

  • computeOps:     1

  •    readOps:     8

  •   writeOps:     0

  •  readBytes: 1,174

  • writeBytes:     0

  •  queryTime:  14ms

  •    retries:     0

The following query paginates the set of distinct "spells" documents with a fire or water element:

result, err := client.Query(
	f.Paginate(
		f.Union(
			f.MatchTerm(f.Index("spells_by_element"), "fire"),
			f.MatchTerm(f.Index("spells_by_element"), "water"))))

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[data:[{181388642046968320 0xc00008e300 0xc00008e300 <nil>} {181388642071085568 0xc00008e4b0 0xc00008e4b0 <nil>} {181388642088911360 0xc00008e660 0xc00008e660 <nil>}]]
client.query(
  q.Paginate(
    q.Union(
      q.Match(q.Index('spells_by_element'), 'fire'),
      q.Match(q.Index('spells_by_element'), 'water'),
    )
  )
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  data: [
    Ref(Collection("spells"), "181388642046968320"),
    Ref(Collection("spells"), "181388642071085568"),
    Ref(Collection("spells"), "181388642088911360")
  ]
}
result = client.query(
  q.paginate(
    q.union(
      q.match(q.index("spells_by_element"), "fire"),
      q.match(q.index("spells_by_element"), "water")
    )
  )
)
print(result)
{'data': [Ref(id=181388642046968320, collection=Ref(id=spells, collection=Ref(id=collections))), Ref(id=181388642071085568, collection=Ref(id=spells, collection=Ref(id=collections))), Ref(id=181388642088911360, collection=Ref(id=spells, collection=Ref(id=collections)))]}
Paginate(
  Union(
    Match(Index('spells_by_element'), 'fire'),
    Match(Index('spells_by_element'), 'water'),
  )
)
{
  data: [
    Ref(Collection("spells"), "181388642046968320"),
    Ref(Collection("spells"), "181388642071085568"),
    Ref(Collection("spells"), "181388642088911360")
  ]
}
Query metrics:

  •    bytesIn:  135

  •   bytesOut:  377

  • computeOps:    1

  •    readOps:    2

  •   writeOps:    0

  •  readBytes:  211

  • writeBytes:    0

  •  queryTime: 12ms

  •    retries:    0

The following query returns the same set of results as the previous query, but by setting the sources option to true, the source of the match that included the entry in the set is included in the result:

result, err := client.Query(
	f.Paginate(
		f.Union(
			f.MatchTerm(f.Index("spells_by_element"), "fire"),
			f.MatchTerm(f.Index("spells_by_element"), "water")),
		f.Sources(true)))

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[data:[map[sources:[{map[match:{spells_by_element 0xc00011a330 0xc00011a330 <nil>} terms:fire]}] value:{181388642046968320 0xc00011a210 0xc00011a210 <nil>}] map[sources:[{map[match:{spells_by_element 0xc00011a630 0xc00011a630 <nil>} terms:fire]} {map[match:{spells_by_element 0xc000164090 0xc000164090 <nil>} terms:water]}] value:{181388642071085568 0xc00011a510 0xc00011a510 <nil>}] map[sources:[{map[match:{spells_by_element 0xc000164390 0xc000164390 <nil>} terms:water]}] value:{181388642088911360 0xc000164270 0xc000164270 <nil>}]]]
client.query(
  q.Paginate(
    q.Union(
      q.Match(q.Index('spells_by_element'), 'fire'),
      q.Match(q.Index('spells_by_element'), 'water'),
    ),
    { sources: true },
  )
)
.then((ret) => console.log(util.inspect(ret, { depth: null })))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  data: [
    {
      value: Ref(Collection("spells"), "181388642046968320"),
      sources: [ Match(Index("spells_by_element"), "fire") ]
    },
    {
      value: Ref(Collection("spells"), "181388642071085568"),
      sources: [
        Match(Index("spells_by_element"), "fire"),
        Match(Index("spells_by_element"), "water")
      ]
    },
    {
      value: Ref(Collection("spells"), "181388642088911360"),
      sources: [ Match(Index("spells_by_element"), "water") ]
    }
  ]
}
result = client.query(
  q.paginate(
    q.union(
      q.match(q.index("spells_by_element"), "fire"),
      q.match(q.index("spells_by_element"), "water")
    ),
    sources=True
  )
)
print(result)
{'data': [{'value': Ref(id=181388642046968320, collection=Ref(id=spells, collection=Ref(id=collections))), 'sources': [SetRef({'match': Ref(id=spells_by_element, collection=Ref(id=indexes)), 'terms': 'fire'})]}, {'value': Ref(id=181388642071085568, collection=Ref(id=spells, collection=Ref(id=collections))), 'sources': [SetRef({'match': Ref(id=spells_by_element, collection=Ref(id=indexes)), 'terms': 'fire'}), SetRef({'match': Ref(id=spells_by_element, collection=Ref(id=indexes)), 'terms': 'water'})]}, {'value': Ref(id=181388642088911360, collection=Ref(id=spells, collection=Ref(id=collections))), 'sources': [SetRef({'match': Ref(id=spells_by_element, collection=Ref(id=indexes)), 'terms': 'water'})]}]}
Paginate(
  Union(
    Match(Index('spells_by_element'), 'fire'),
    Match(Index('spells_by_element'), 'water'),
  ),
  { sources: true },
)
{
  data: [
    {
      value: Ref(Collection("spells"), "181388642046968320"),
      sources: [ Match(Index("spells_by_element"), "fire") ]
    },
    {
      value: Ref(Collection("spells"), "181388642071085568"),
      sources: [
        Match(Index("spells_by_element"), "fire"),
        Match(Index("spells_by_element"), "water")
      ]
    },
    {
      value: Ref(Collection("spells"), "181388642088911360"),
      sources: [ Match(Index("spells_by_element"), "water") ]
    }
  ]
}
Query metrics:

  •    bytesIn:  150

  •   bytesOut:  881

  • computeOps:    1

  •    readOps:    2

  •   writeOps:    0

  •  readBytes:  211

  • writeBytes:    0

  •  queryTime: 18ms

  •    retries:    0

The following query returns the same set of results as the first "spells" query, but by setting the events option to true, the events associated with each item in the set are included in the result:

result, err := client.Query(
	f.Paginate(
		f.Events(
			f.Union(
				f.MatchTerm(f.Index("spells_by_element"), "fire"),
				f.MatchTerm(f.Index("spells_by_element"), "water")))))

if err != nil {
	fmt.Fprintln(os.Stderr, err)
} else {
	fmt.Println(result)
}
map[data:[map[action:add document:{181388642046968320 0xc00008f5f0 0xc00008f5f0 <nil>} ts:1603747233890000] map[action:add document:{181388642071085568 0xc00008f7d0 0xc00008f7d0 <nil>} ts:1603747233890000] map[action:add document:{181388642088911360 0xc0001340f0 0xc0001340f0 <nil>} ts:1603747233890000]]]
client.query(
  q.Paginate(
    q.Union(
      q.Match(q.Index('spells_by_element'), 'fire'),
      q.Match(q.Index('spells_by_element'), 'water'),
    ),
    { events: true },
  )
)
.then((ret) => console.log(ret))
.catch((err) => console.error(
  'Error: [%s] %s: %s',
  err.name,
  err.message,
  err.errors()[0].description,
))
{
  data: [
    {
      ts: 1592112564110000,
      action: 'add',
      document: Ref(Collection("spells"), "181388642046968320")
    },
    {
      ts: 1592112564110000,
      action: 'add',
      document: Ref(Collection("spells"), "181388642071085568")
    },
    {
      ts: 1592112564110000,
      action: 'add',
      document: Ref(Collection("spells"), "181388642088911360")
    }
  ]
}
result = client.query(
  q.paginate(
    q.union(
      q.match(q.index("spells_by_element"), "fire"),
      q.match(q.index("spells_by_element"), "water")
    ),
    events=True
  )
)
print(result)
{'data': [{'ts': 1592874755080000, 'action': 'add', 'document': Ref(id=181388642046968320, collection=Ref(id=spells, collection=Ref(id=collections)))}, {'ts': 1592874755080000, 'action': 'add', 'document': Ref(id=181388642071085568, collection=Ref(id=spells, collection=Ref(id=collections)))}, {'ts': 1592874755080000, 'action': 'add', 'document': Ref(id=181388642088911360, collection=Ref(id=spells, collection=Ref(id=collections)))}]}
Paginate(
  Union(
    Match(Index('spells_by_element'), 'fire'),
    Match(Index('spells_by_element'), 'water'),
  ),
  { events: true },
)
{
  data: [
    {
      ts: 1624310483590000,
      action: 'add',
      document: Ref(Collection("spells"), "181388642046968320")
    },
    {
      ts: 1624310483590000,
      action: 'add',
      document: Ref(Collection("spells"), "181388642071085568")
    },
    {
      ts: 1624310483590000,
      action: 'add',
      document: Ref(Collection("spells"), "181388642088911360")
    }
  ]
}
Query metrics:

  •    bytesIn:  149

  •   bytesOut:  527

  • computeOps:    1

  •    readOps:    2

  •   writeOps:    0

  •  readBytes:  211

  • writeBytes:    0

  •  queryTime: 10ms

  •    retries:    0

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!