Check out v4 of the Fauna CLI

v4 of the Fauna CLI is now in beta.

The new version introduces enhancements to the developer experience, including an improved authentication workflow. To get started, check out the CLI v4 quick start.

Pagination

Pagination lets you iterate through large Sets returned by a query.

This guide covers default pagination, customizing page size, and accessing paginated results within FQL queries.

Default pagination

Fauna automatically paginates result Sets with 16 or more elements.

When a query returns paginated results, Fauna materializes a subset of the Set with an after pagination cursor:

// Uses the `Product` collection's `sortedByPriceLowToHigh()` index to
// return all `Product` collection documents.
// The collection contains more than 16 documents.
Product.sortedByPriceLowToHigh()
{
  // The result Set contains 16 elements.
  data: [
    {
      id: "555",
      coll: Product,
      ts: Time("2099-07-30T15:57:03.730Z"),
      name: "single lime",
      description: "Conventional, 1 ct",
      price: 35,
      stock: 1000,
      category: Category("789")
    },
    {
      id: "888",
      coll: Product,
      ts: Time("2099-07-30T15:57:03.730Z"),
      name: "cilantro",
      description: "Organic, 1 bunch",
      price: 149,
      stock: 100,
      category: Category("789")
    },
    ...
  ],
 // Use the `after` cursor to get the next page of results.
  after: "hdW..."
}

Customize page size

Reference: set.pageSize()

Use set.pageSize() to change the maximum number of elements per page:

// Calls `pageSize()` with a size of `2`.
Product.sortedByPriceLowToHigh().pageSize(2)
{
  // The result Set contains two elements or fewer.
  data: [
    {
      id: "555",
      coll: Product,
      ts: Time("2099-07-30T15:57:03.730Z"),
      name: "single lime",
      description: "Conventional, 1 ct",
      price: 35,
      stock: 1000,
      category: Category("789")
    },
    {
      id: "888",
      coll: Product,
      ts: Time("2099-07-30T15:57:03.730Z"),
      name: "cilantro",
      description: "Organic, 1 bunch",
      price: 149,
      stock: 100,
      category: Category("789")
    }
  ],
  after: "hdaExad..."
}

You should typically place pageSize() last in a method chain. pageSize() only affects the rendering of a Set, not subsequent operations. Methods chained to pageSize() access the entire calling Set, not a page of results.

Iterate through pages

Reference: Set.paginate()

To iterate through paginated results, pass the after cursor to Set.paginate():

Set.paginate("hdW...")

Example implementation

The following example shows how you can iterate through paginated results using the JavaScript driver:

import { Client, fql } from "fauna";

const client = new Client({
    secret: '<FAUNA_SECRET>'
});

// Defines a function that accepts a Fauna `after` cursor as
// an optional argument.
async function getProducts(afterCursor) {
  // Initial FQL query.
  const query = fql`Product.sortedByPriceLowToHigh().pageSize(2)`;

  const response = await client.query(
    // If an `after` cursor is provided, use `Set.paginate()`
    // to get the next page of results.
    // Otherwise, use the initial FQL query.
    afterCursor ? fql`Set.paginate(${afterCursor})` : query
  );

  const data = response.data.data;
  const nextCursor = response.data.after;

  // Print the results and the after cursor.
  console.log("Data:", data);
  console.log("Next cursor:", nextCursor);

  return { data, nextCursor };
}

// Defines a function to loop through paginated results.
async function getAllProducts() {
  let afterCursor;

  do {
    const { data, nextCursor } = await getProducts(afterCursor);
    afterCursor = nextCursor;
  } while (afterCursor);
}

// Call the function to loop through the results.
getAllProducts();

Driver pagination methods

The Fauna client drivers also include methods for automatically iterating through pages. See:

Access pages and cursors within a query

Reference: set.paginate()

If you need to access an after cursor or paginated results within an FQL query, use set.paginate():

Product.sortedByPriceLowToHigh().pageSize(2).paginate()

For example, you can use paginate() to return the after cursor for use as a URL in a client application.

Alternatively, you can use paginate() to iteratively update a large Set of collection documents over several queries. For an example, see the paginate() reference docs.

Considerations for paginate()

paginate() accepts an optional argument to control page size. In most cases, you should not use paginate() in place of pageSize().

The following table outlines differences between set.pageSize() and set.paginate():

Difference set.pageSize() set.paginate()

Use case

Use in most cases.

Use when needing to access an 'after' cursor or paginated results within an FQL query.

Return type

Returns a set.

Returns an object.

Loading strategy

Lazy loading. Only fetches results as needed.

Eager loading. Fetches results instantly, even if the results aren’t returned or used.

Client driver methods

Compatible with driver pagination methods.

Incompatible with driver pagination methods.

Projection

Supports projections.

Doesn’t support projections.

Set instance methods

Supports set instance methods.

Doesn’t support set instance methods.

Cursor state and expiration

If a paginated Set contains documents, the after cursor fetches historical snapshots of the documents at the time of the original query.

You can control the retention of document snapshots using the collection schema’s history_days setting. An after cursor is valid for history_days plus 15 minutes. If history_days is 0 or unset, the cursor is valid for 15 minutes.

If a document’s snapshot is no longer available, a NullDoc is returned instead:

{
  data: [
    {
      id: "393605620096303168",
      coll: Product,
      ts: Time("2099-03-28T12:53:40.750Z"),
      name: "limes",
      ...
    },
    Product("401942927818883138") /* not found */
  ],
  after: "hdWCxmd..."
}
See Document history

Invalid cursor

If you pass an invalid or expired after cursor to Set.paginate(), the Fauna Core HTTP API’s Query endpoint returns a query runtime error with an invalid_cursor error code and a 400 HTTP status code:

{
  "error": {
    "code": "invalid_cursor",
    "message": "Cursor cursor is invalid or expired."
  },
  ...
}

Fauna’s client drivers include classes for query runtime errors:

Cursor handling and best practices

When working with after cursors, keep the following in mind:

  • Encoding: after cursors can contain special characters, such as +.

    If you’re using an after cursor in a URL or as a query parameter, make sure to URL-encode the cursor value. This ensures that the cursor is properly transmitted and interpreted.

    Generally, we don’t recommend you use after cursors for user-facing pagination. See UI pagination.

  • Cursor integrity: You shouldn’t directly change or manipulate after cursor values. The cursor is a string-encoded hash that contains all information required to get the next page in the Set.

    Any modification to the cursor could result in unexpected behavior or errors when retrieving the next page.

Paginate in reverse

Paginated queries don’t include a before cursor. Instead, you can use a range search and document IDs or other unique field values to paginate in reverse. For example:

  1. Run an initial paginated query:

    Product.all().pageSize(2)
    {
  data: [
    {
      id: "111",
      coll: Product,
      ts: Time("2099-08-16T14:00:59.075Z"),
      name: "cups",
      description: "Translucent 9 Oz, 100 ct",
      price: 698,
      stock: 100,
      category: Category("123")
    },
    {
      id: "222",
      coll: Product,
      ts: Time("2099-08-16T14:00:59.075Z"),
      name: "donkey pinata",
      description: "Original Classic Donkey Pinata",
      price: 2499,
      stock: 50,
      category: Category("123")
    }
  ],
  after: "hdW..."
}

  2. Page forward until you find the document you want to start reversing from:

    Set.paginate("hdW...")

    Copy the ID of the document:

    {
  data: [
    {
      id: "333",
      coll: Product,
      ts: Time("2099-08-16T14:00:59.075Z"),
      name: "pizza",
      description: "Frozen Cheese",
      price: 499,
      stock: 100,
      category: Category("456")
    },
    {
      // Begin reverse pagination from this doc ID.
      id: "444",
      coll: Product,
      ts: Time("2099-08-16T14:00:59.075Z"),
      name: "avocados",
      description: "Conventional Hass, 4ct bag",
      price: 399,
      stock: 1000,
      category: Category("789")
    }
  ],
  after: "hdW..."
}

  3. To reverse paginate, run the original query with:

    // "444" is the ID of the document to reverse from.
Product.all({ to: "444" }).reverse().pageSize(2)
    {
  data: [
    {
      // The results of the previous query are reversed.
      id: "444",
      coll: Product,
      ts: Time("2099-08-16T14:00:59.075Z"),
      name: "avocados",
      description: "Conventional Hass, 4ct bag",
      price: 399,
      stock: 1000,
      category: Category("789")
    },
    {
      id: "333",
      coll: Product,
      ts: Time("2099-08-16T14:00:59.075Z"),
      name: "pizza",
      description: "Frozen Cheese",
      price: 499,
      stock: 100,
      category: Category("456")
    }
  ],
  after: "hdW..."
}

    To get historical snapshots of documents at the time of the original query, use an at expression:

    // Time of the original query.
let originalQueryTime = Time.fromString("2099-08-16T14:30:00.000Z")
at (originalQueryTime) {
  // "444" is the ID of the document to reverse from.
  Product.all({ to: "444" }).reverse().pageSize(2)
}

  4. Repeat the previous step to continue paginating in reverse:

    Product.all({ to: "333" }).reverse().pageSize(2)

UI pagination

Fauna’s pagination functionality is designed to help client apps consume large result Sets at a controlled pace. It isn’t intended for use in user-facing pagination.

To implement UI pagination in your app, we recommend you use range searches instead. Range searches give you precise control over returned data and consistent results.

Build UI pagination with ranged searches

For example, an e-commerce application must present users with a paginated list of products. Products are sorted by ascending price with up to five products per page.

To implement UI pagination using ranged searches:

  1. Define an index in the Product collection schema:

    collection Product {
  ...
  index inOrderOfPrice {
    // `values` are document fields for sorting and range searches.
    // In this example, you sort and filter index results by their
    // ascending `price` field value. Include the document `id`
    // to handle cases where multiple products have the same price.
    values [ .price, .id ]
  }
}

    Submit the updated schema to Fauna using the Fauna Dashboard or the Fauna CLI's fauna schema push command.

  2. To get the first page of results, the app runs the following FQL query:

    // Gets the first page of products,
// starting with a price of `0`.
Product.inOrderOfPrice({ from: [0] })!
  .take(5 + 1)  // PAGE_SIZE + 1
  .toArray() {
    id,
    name,
    price
  }

    The result array includes six items, but the app UI only displays the first five. The presence of a sixth item indicates a "next" page is available.

    [
  {
    id: "555",
    name: "single lime",
    price: 35
  },
  {
    id: "888",
    name: "cilantro",
    price: 149
  },
  {
    id: "777",
    name: "limes",
    price: 299
  },
  {
    id: "666",
    name: "organic limes",
    price: 349
  },
  {
    id: "444",
    name: "avocados",
    price: 399
  },
  {
    id: "333",
    name: "pizza",
    price: 499
  }
]

  3. To page forward:

    • Use the price and id of the last item from the previous results.

    • Take PAGE_SIZE + 1 items.

    // Gets the next page of products, starting with
// the `price` (499) and document `id` of last item
// from the previous results.
Product.inOrderOfPrice({ from: [499, "333"] })!
  .take(5 + 1)  // PAGE_SIZE + 1
  .toArray() {
    id,
    name,
    price
  }

    Like the previous query, the app UI only displays the first five items in the results. The presence of a sixth item indicates a "next" page is available.

    [
  {
    id: "333",
    name: "pizza",
    price: 499
  },
  {
    id: "111",
    name: "cups",
    price: 698
  },
  {
    id: "999",
    name: "taco pinata",
    price: 2399
  },
  {
    id: "222",
    name: "donkey pinata",
    price: 2499
  },
  {
    id: "123",
    name: "gorilla pinata",
    price: 2599
  },
  {
    id: "456",
    name: "giraffe pinata",
    price: 2799
  }
]

  4. To page backward:

    • Pass the price and id of the first item from the previous results to to in the ranged search.

    • Call set.reverse() on the Set.

    • Take PAGE_SIZE + 2 items.

    • Drop the first item, which is a duplicate from the last page. Range searches are inclusive.

    // Gets the previous page of products, starting with
// the `price` (499) and document `id` of the first
// item from the previous results.
Product.inOrderOfPrice({ to: [499, "333"] })!
  .reverse()
  .take(5 + 2)  // PAGE_SIZE + 2
  .drop(1)
  .toArray() {
    id,
    name,
    price
  }

    Similar to the previous queries, the app UI only displays the first five items in the results. The presence of a sixth item indicates a "previous" page is available. In this example, no "previous" page is available.

    [
  {
    id: "444",
    name: "avocados",
    price: 399
  },
  {
    id: "666",
    name: "organic limes",
    price: 349
  },
  {
    id: "777",
    name: "limes",
    price: 299
  },
  {
    id: "888",
    name: "cilantro",
    price: 149
  },
  {
    id: "555",
    name: "single lime",
    price: 35
  }
]

Jump to a page using an offset

Using an offset to jump to a specific page can be expensive for large datasets. The offset can consume many Transactional Read Operations (TROs), which will be discarded.

This is a common issue with offset-based pagination across all databases and is not unique to Fauna.

For large datasets, it may be more efficient to use a driver pagination method to fetch all results from Fauna and handle pagination in your app instead.

To jump to a specific page, you can specify an offset using set.take() and set.drop():

// Jumps to page 3 (PAGE_NUM = 3) with each
// page containing 5 items (PAGE_SIZE = 5).
Product.inOrderOfPrice()!
  .take((5 * 3) + 1)  // (PAGE_SIZE * PAGE_NUM) + 1
  .drop(5 * (3 - 1))  // PAGE_SIZE * (PAGE_NUM - 1)
  .toArray() {
    id,
    name,
    price
  }
[
  {
    id: "456",
    name: "giraffe pinata",
    price: 2799
  },
  {
    id: "789",
    name: "whale pinata",
    price: 3299
  },
  {
    id: "234",
    name: "tablet",
    price: 3599
  },
  {
    id: "567",
    name: "drone",
    price: 3799
  },
  {
    id: "890",
    name: "smartphone",
    price: 15999
  },
  {
    id: "345",
    name: "laptop",
    price: 24999
  }
]

Consistent results across pages

By default, queries run against the most recent version of documents. If the documents change during pagination — for example, if you create or delete documents — results may not be consistent across queries.

To ensure consistent results, use at expression to get historical snapshots of documents at the time of the original query:

// Time of the original query.
let originalQueryTime = Time.fromString("2099-08-16T14:30:00.000Z")
at (originalQueryTime) {
  Product.inOrderOfPrice({ to: [499, "333"] })!
    .reverse()
    .take(5 + 2)  // PAGE_SIZE + 2
    .drop(1)
    .toArray() {
      id,
      name,
      price
    }
}

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!