Fauna Logs

You can use Fauna Logs to monitor the performance of your Fauna databases and diagnose issues.

Requirements

To use Fauna Logs, you must have a Fauna Pro or Enterprise plan.

Log format

Fauna outputs Fauna Logs in a JSON+L format. Each line contains a JSON object that represents a log entry for a single query. An example Fauna Log object formatted for readability:

{
  "API_VERSION": "10",
  "BYTES_IN": 43,
  "BYTES_OUT": 2692,
  "BYTE_READ_OPS": 21,
  "BYTE_WRITE_OPS": 0,
  "COMPUTE_OPS": 1,
  "DATABASE": ["ECommerce"],
  "FQL_SHAPE":"Product.sortedByPriceLowToHigh()\n",
  "QUERY_TIME_MS": 64,
  "REGION_GROUP": "us-std",
  "REQUEST_HEADERS": "{\"user_agent\":\"curl/8.4.0\",\"x_forwarded_for\":\"108.83.200.91, 10.1.1.124\"}",
  "REQUEST_METHOD": "POST",
  "RESPONSE_CODE": "200",
  "TAGS": { "request_type": "product_search", "sort_by": "price" },
  "TRACEPARENT": "00-00000000000000000992df31c81cc1c8-481003b57ce0897f-00",
  "TS": "2099-05-10 18:05:42.871 Z",
  "TXN_RETRIES": 0
}

The Datadog integration uses a different format for Fauna Log entries. See Log record format in the Datadog integration docs.
Reference: Log record format

Instrument queries for Fauna Logs

You can instrument your queries for Fauna Logs by including query tags and traces in query requests.

Query tags

You can instrument your queries with query tags to better identify their source and context in Fauna Logs.

A query tag is an arbitrary key-value pair. Fauna Log entries include query tags in the TAGS property:

{
  ...
  // Query tags
  "TAGS": { "request_type": "product_search", "sort_by": "price" },
  ...
}

Add query tags in API requests

If you use the Query HTTP API endpoint, you can pass query tags for a request using the x-query-tags request header:

curl -X POST \
  'https://db.fauna.com/query/1' \
  -H 'Authorization: Bearer <FAUNA_SECRET>' \
  -H 'Content-Type: application/json' \
  -H 'x-query-tags: request_type=product_search,sort_by=price' \
  -d '{"query": "Product.sortedByPriceLowToHigh()"}'

Query API responses include query tags in the query_tags property:

{
  "data": {
    ...
  },
  "static_type": "Set<Product>",
  "summary": "",
  "txn_ts": 1720713675611973,
  "stats": {
    ...
  },
  // Query tags are returned in the response.
  "query_tags": "sort_by=price,request_type=product_search",
  "schema_version": 1720712455226000
}

Add query tags using a client driver

The Fauna client drivers include query options for passing query tags:

For example, using the JavaScript client driver:

import { Client, fql } from "fauna";

const client = new Client({
  secret: '<FAUNA_SECRET>'
});

const query = fql`Product.sortedByPriceLowToHigh() { name, description, price }`;

// Define the query options.
const options = {
  // Define the query tags.
  query_tags: {
    request_type: "product_search",
    sort_by: "price"
  }
};

// Pass the query options to the query.
const response = await client.query(query, options);
// Print the query tags from the response.
console.log(response.query_tags);

client.close();

You can use your programming language’s native variables to dynamically populate the keys and values of query tags. For example, you can include an end user ID as a query tag:

import { Client, fql } from "fauna";

const client = new Client({
  secret: '<FAUNA_SECRET>'
});

// Uses the FQL `Query.identity()` method to get the
// document ID for the token secret used to authenticate queries.
const userIdQuery = fql`Query.identity() { id }`;
const userIdRes = await client.query(userIdQuery);
// Store the identity document ID as the `userId`.
// If the query is authenticated using a key or JWT,
// set `userId` to "None."
const userId = userIdRes.data?.id ?? "None";

const query = fql`Product.sortedByPriceLowToHigh() { name, description, price }`;

// Define the query options.
const options = {
  // Define the query tags.
  query_tags: {
    // Set the `user_id` query tag to the `userId`.
    user_id: userId
  }
};

// Pass the query options to the query.
const response = await client.query(query, options);
// Print the query tags from the response.
console.log(response.query_tags);

client.close();

Query tag constraints

  • A query can include up to 25 query tags.

  • Query tag keys and values must be alphanumeric strings. The strings can include the underscore (_) character. Empty strings are not supported.

  • A tag key can be up to 40 characters. A tag value can be up to 80 characters.

A query with invalid query tags fails and returns an error with an invalid_request error code and a 400 HTTP status code.

Traces

You can identify queries triggered by a specific service, component, or process using a traceparent header.

Fauna Log entries include the header’s traceparent identifier in the TRACEPARENT property. If you don’t include a traceparent identifier in a query request or use an invalid identifier, Fauna generates a valid identifier.

Add a traceparent in API requests

If you use the Query HTTP API endpoint, you can include a traceparent header in the request. For example:

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

Add a traceparent using a client driver

The Fauna client drivers include query options for passing a traceparent:

For example, using the JavaScript client driver:

import { Client, fql } from "fauna";

const client = new Client({
  secret: '<FAUNA_SECRET>'
});

const query = fql`Product.sortedByPriceLowToHigh() { name, description, price }`;

// Define the query options.
const options = {
  // Define the traceparent.
  traceparent: "00-750efa5fb6a131eb2cf4db39f28366cb-000000000000000b-00"
};

// Pass the query options to the query.
const response = await client.query(query, options);

console.log(response);

client.close();

Access Fauna Logs

You can access Fauna Logs using the:

Datadog integration

You can use Fauna’s Datadog integration to send Fauna Logs to Datadog in near real time. The integration includes a Datadog dashboard you can use to monitor query volume, performance, costs, responses, and failures.

Datadog dashboard

See Datadog integration

Fauna Logs API endpoints

You can use the Fauna Logs endpoints of the Fauna Account HTTP API to request and download Fauna Logs programmatically.

See Access Fauna Logs using the HTTP API

Fauna dashboard

You can manually request and download raw logs using the Fauna Dashboard. If wanted, you can then upload the logs to a third-party service for visualization or analysis.

See Access logs in the Fauna Dashboard

Aggregate Fauna Logs

You can aggregate Fauna Logs on entry properties using a script or JSON-processing tool, such as jq.

Example: Aggregate logs by query tags

The following example uses jq to aggregate query log entries by the user_id query tag in the TAGS property:

jq -s '
  # Filter out log entries without a `user_id` query tag.
  map(select(.TAGS.user_id != null)) |

  # Group all entries by `user_id`.
  group_by(.TAGS.user_id) |

  # For each group, create an object with aggregated data
  map({
    user_id: .[0].TAGS.user_id,
    query_count: length,
    total_bytes_in: map(.BYTES_IN // 0) | add,
    total_bytes_out: map(.BYTES_OUT // 0) | add,
    total_query_time_ms: map(.QUERY_TIME_MS // 0) | add,
    databases_accessed: map(.DATABASE[]) | unique
  }) |

  # Transform the Array into an object with `user_id`s as keys.
  map({key: .user_id, value: .}) |
  from_entries

  # Replace `input.jsonl` with the path to your log file.
' input.jsonl > output.json
{
  "403221241075335241": {
    "user_id": "403221241075335241",
    "query_count": 25,
    "total_bytes_in": 12194,
    "total_bytes_out": 43789,
    "total_query_time_ms": 1157,
    "databases_accessed": [
      "ECommerce"
    ]
  },
  ...
}

Example: Aggregate logs by query shape

The following example aggregates Fauna Logs that include the Product collection name in the FQL_SHAPE property:

jq -s '
  # Filter entries where `FQL_SHAPE` references the "Product" collection.
  map(select(.FQL_SHAPE | contains("Product"))) |

  # Group all entries. In this case, it will be a single group.
  group_by(true) |

  # Create a single object with aggregated data
  map({
    query_count: length,
    total_bytes_in: map(.BYTES_IN // 0) | add,
    total_bytes_out: map(.BYTES_OUT // 0) | add,
    total_query_time_ms: map(.QUERY_TIME_MS // 0) | add,
    databases_accessed: map(.DATABASE[]) | unique,
    unique_shapes: map(.FQL_SHAPE) | unique,
    response_codes: map(.RESPONSE_CODE) | group_by(.) | map({key: .[0], value: length}) | from_entries
  }) |

  # Extract the single group.
  .[0]

  # Replace `input.jsonl` with the path to your log file.
' input.jsonl > output.json
{
  "query_count": 6,
  "total_bytes_in": 4663,
  "total_bytes_out": 8981,
  "total_query_time_ms": 340,
  "databases_accessed": [
    "ECommerce"
  ],
  "unique_shapes": [
    "Product.sortedByPriceLowToHigh()\n",
    ...
  ],
  "response_codes": {
    "200": 4,
    "400": 2
  }
}

Limitations

  • A log entry is available approximately 30 seconds to 10 minutes after a query’s execution.

  • You can request up to 90 days of Fauna Logs. You can specify future times in the request.

  • You can’t get Fauna Logs for queries run more than one year ago.

  • You can’t request Fauna Logs for a database nested 10 or more levels below a top-level parent database.

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!