fauna import

Import JSON files, CSV files, or directories into a collection.

Syntax

fauna import --path <VALUE> [--allow-short-rows] [--append]
  [--collection <value>] [--[no-]color ]
  [--db <value>] [--dry-run] [--endpoint <value>]
  [--environment <value>] [--secret <value>]
  [--timeout <value>] [--treat-empty-csv-cells-as empty|null]
  [--type <value>] [--url <value>]

Description

The import command imports the contents of a JSON or CSV file, or a directory of JSON or CSV files into a Fauna collection. If you import a directory of source files, each file is imported into a separate collection.

Import considerations

  • JSON source files must be valid JSON. Each JSON object in the file is a document in the target collection.

  • CSV source files must be comma-delimited. Each line in the CSV file becomes a document in the target collection. A CSV file must have a header line with the field names for the target collection.

  • If your CSV file has rows with fewer fields than the number of fields in the header line, you can use the --allow-short-rows option to allow the import to continue. Otherwise, the import fails with an error. If you use the --allow-short-rows option, the documents imported from short rows don’t include the missing fields.

  • If the CSV file has empty columns, you can use the treat-empty-csv-cells-as option to choose:

    • null: (default) The field doesn’t exist in the imported document.

    • empty: The field exists in the imported document as an empty string.

  • The target collection can be an existing Fauna collection or a new one. If the target collection exists and isn’t empty, you must use the --append option. The new documents are added to the existing collection. If you don’t specify a target collection, the file name of the source file is used as the name of the target collection.

  • The --path option is the required path to the source file or directory of source files.

  • Floating-point numbers that end with .0 are converted by JavaScript to integers.

Document references

You can’t use the import command to import documents that contain document references. Instead, you can use fauna eval to import a .fql file that creates the documents.

For example, you can run:

fauna eval --file products.fql

Where products.fql contains:

// Create an Array of objects that contain document data.
let products = [
  {
    id: 123,
    name: 'key limes',
    description: 'Conventional, 16 oz bag',
    price: 2_99,
    stock: 30,
    category: Category.byName('produce')!.first()
  },
  {
    id: 456,
    name: 'peaches',
    description: 'Organic, 2 ct',
    price: 3_49,
    stock: 50,
    category: Category.byName('produce')!.first()
  }
]

// Use `forEach()` to create a `Product` collection document for each
// element of the previous Array.
products.forEach(doc => Product.create({ doc }))
// `forEach()` returns `null`.

Billing considerations

Importing data with the import command involves billable write and compute operations.

See Plans and billing for detailed information.

To ensure data import integrity, each source file record should include a unique identifying field. You can create a unique index on that field to ensure that records are not imported multiple times and that you can query against the unique field to verify import completeness.

Options

Option Description

--allow-short-rows

Allow CSV files in which some rows have fewer fields than the number of fields in the header row. If the import finds a row that has more fields than the number of fields in the header row, the import fails with an error.

--append

Append documents to an existing collection.

--collection=<collection>

Create a collection to be created. Defaults to the file name of the source file.

--[no-]color

Enables or disables color formatting for the output. Color formatting is enabled by default if the terminal supports it (determined using chalk/supports-color). Use --no-color to disable.

--db=<db>

Name the database in which to create a new target collection or append to an existing target collection. The parent database is the database associated with the secret you specify for the command.

--dry-run

Do all import operations, including record processing and type conversions, except creating documents. This allows you to detect issues that might impact an import before writing documents to your collections.

--endpoint

Connection endpoint from .fauna-shell.

--environment

Environment to use, from a Fauna project.

--help

Help for create-database command.

--path=<path>

Path to a source file or directory of source files.

The directories can have only files, not subdirectories.

--secret

Authentication secret. Overrides the secret in .fauna-shell.

Use a scoped key to interact with a child database using a parent database’s admin key.

For example, with a parent database’s admin key secret of fn123, you can access a child database by appending the child database name and role: fn123:childDB:admin.

--timeout

Connection timeout (milliseconds).

--treat-empty-csv-cells-as=<empty|null>

Define how empty fields in a record should be handled:
       empty = Empty fields should occur in the imported document with an empty string.
       null = (default) Empty fields shouldn’t occur in the imported document.

--type=<column>::<type>

Describe a <column> data type. This feature is available only when importing CSV files. Type conversion is ignored for JSON and JSONL files. See the following CSV column types.

--url

Database URL. Overrides the URL in .fauna-shell.

 

CSV column types:

A <type> is one of the following data types:

Type Action

bool

Convert true, t, yes, and 1 to true. Convert all other values to false, except null, which remains null.

number

Convert strings to numbers.

dateString

Convert an ISO 8601 or RFC 2022 date string into a Time value. A best effort is made for other date string formats, and warnings are emitted when such date conversions are made.

dateEpochMillis

Convert milliseconds since Unix epoch to a Time value.

dateEpochSeconds

Convert seconds since Unix epoch to a Time value.

Examples

These are some common examples of how to use the import command.

Import a JSON file

A file named zipcodes.json has the following:

{ "zipcode" : "01001", "city" : "AGAWAM", "pop" : 15338, "state" : "MA" }
{ "zipcode" : "01002", "city" : "CUSHMAN", "pop" : 36963, "state" : "MA" }
{ "zipcode" : "01005", "city" : "BARRE", "pop" : 4546, "state" : "MA" }
{ "zipcode" : "01007", "city" : "BELCHERTOWN", "pop" : 10579, "state" : "MA" }
{ "zipcode" : "01008", "city" : "BLANDFORD", "pop" : 1240, "state" : "MA" }

The following terminal command imports zipcodes.json:

fauna import --path=./zipcodes.json

In the preceding command, no --collection option is given, so the Fauna CLI creates a new collection called zipcodes.

Import a JSON file and append to an existing collection

A file named zipcodes2.json has the following:

{ "zipcode" : "01010", "city" : "BRIMFIELD", "pop" : 3706, "state" : "MA" }
{ "zipcode" : "01011", "city" : "CHESTER", "pop" : 1688, "state" : "MA" }
{ "zipcode" : "01012", "city" : "CHESTERFIELD", "pop" : 177, "state" : "MA" }

The following terminal command imports zipcodes2.json and appends the documents to the existing collection zipcodes:

fauna import --path=./zipcodes2.json --collection=zipcodes --append

Import a JSON file with configuration options

The following terminal command overrides any configuration options in the configuration file:

fauna import --path=./zipcodes.json --endpointURL=https/db.us.fauna.com:8443 --secret=secret

Import a CSV file with type conversions

The following CSV document has three string values:

myDate,myBool,myNumber
"May 3, 2021",true,15338

To convert those string values to other types, you can use the --type option.

fauna import --path=./myFile.csv --type=myDate::date --type=myBool::bool --type=myNumber::number

Import a directory of source files

A directory named source_files has the following files:

myJSONfile1.json
myJSONfile2.json
myCSVfile1.csv
myCSVfile2.csv

The following command imports four files and creates four new collections:

fauna import --path=./source_files

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!