import

Import JSON files, CSV files, or directories into a collection.

Syntax

fauna import --path <value> [--url <value>] [--timeout <value>] [--secret <value>] [--endpoint <value>] [--environment <value>] [--db <value>] [--collection <value>] [--type <value>] [--append] [--allow-short-rows] [--dry-run] [--treat-empty-csv-cells-as empty|null]

Description

The import command imports the contents of a JSON or CSV file, or a directory of JSON or CSV files into a Fauna collection. If you import a directory of source files, each file is imported into a separate collection.

File and directory import has the following considerations:

  • JSON source files must be valid JSON, and each JSON object in the file is a document in the target collection.

  • CSV source files must be comma-delimited, and each line in the CSV file becomes a document in the target collection. A CSV file must have a header line with the field names to be used in the target collection.

  • If your CSV file has rows with fewer fields than the number of fields in the header line, you can use the --allow-short-rows option to allow the import to continue. Otherwise, the import fails with an error. If you use the --allow-short-rows option, the documents imported from short rows don’t include the missing fields.

  • If the CSV file has empty columns, you can use the treat-empty-csv-cells-as option to choose:

    • null: (default) The field doesn’t exist in the imported document.

    • empty: The field exists in the imported document as an empty string.

  • The target collection can be an existing Fauna collection or a new one. If the target collection exists and isn’t empty, you must use the --append option, and the new documents are added to the existing collection. If you don’t specify a target collection, the file name of the source file is used as the name of the target collection.

  • The --path option is the required path to the source file or directory of source files. If you don’t include other options, Fauna Shell uses the parameters in the configuration file.

  • Floating-point values that end with .0 are converted by JavaScript to integers.

Billing considerations

Importing data with the import command involves billable write and compute operations.

See Plans and billing for detailed information.

To ensure data import integrity, each source file record should include a unique identifying field. You can create a unique index on that field to ensure that records are not imported multiple times and that you can query against the unique field to verify import completeness.

Options

Option Description

--allow-short-rows

Allow CSV files in which some rows have fewer fields than the number of fields in the header row. If the import finds a row that has more fields than the number of fields in the header row, the import fails with an error.

--append

Append documents to an existing collection.

--collection=<collection>

Create a collection to be created. Defaults to the file name of the source file.

--db=<db>

Name the database in which to create a new target collection or append to an existing target collection. The parent database is the database associated with the secret you specify for the command.

--dry-run

Do all import operations, including record processing and type conversions, except creating documents. This allows you to detect issues that might impact an import before writing documents to your collections.

--endpoint

Connection endpoint, from the ~/.fauna-shell file.

--environment

Environment to use, from a Fauna project.

--help

Help for create-database command.

--path=<path>

Path to a source file or directory of source files.

The directories can have only files, not subdirectories.

--secret

Secret key. Overrides the secret in the ~/.fauna-shell file.

--timeout

Connection timeout (milliseconds).

--treat-empty-csv-cells-as=<empty|null>

Define how empty fields in a record should be handled:
       empty = Empty fields should occur in the imported document with an empty string.
       null = (default) Empty fields shouldn’t occur in the imported document.

--type=<column>::<type>

Describe a <column> data type. This feature is available only when importing CSV files. Type conversion is ignored for JSON and JSONL files. See the following CSV column types.

--url

Database URL. Overrides the URL in the \~/.fauna-shell file.

 

CSV column types:

A <type> is one of the following data types:

Type Action

bool

Convert true, t, yes, and 1 to true. Convert all other values to false, except null, which remains null.

number

Convert strings to numbers.

dateString

Convert an ISO-8601 or RFC-2022 date string into a Time value. A best effort is made for other date string formats, and warnings are emitted when such date conversions are made.

dateEpochMillis

Convert milliseconds since Unix epoch to a Time value.

dateEpochSeconds

Convert seconds since Unix epoch to a Time value.

Examples

These are some common examples of how to use the import command.

Import a JSON file

A file named zipcodes.json has the following:

{ "zipcode" : "01001", "city" : "AGAWAM", "pop" : 15338, "state" : "MA" }
{ "zipcode" : "01002", "city" : "CUSHMAN", "pop" : 36963, "state" : "MA" }
{ "zipcode" : "01005", "city" : "BARRE", "pop" : 4546, "state" : "MA" }
{ "zipcode" : "01007", "city" : "BELCHERTOWN", "pop" : 10579, "state" : "MA" }
{ "zipcode" : "01008", "city" : "BLANDFORD", "pop" : 1240, "state" : "MA" }

The following terminal command imports zipcodes.json:

fauna import --path=./zipcodes.json
Database connection established
Starting Import!
 ›   Success: Import from ./zipcodes.json to zipcodes completed. 5 rows/object imported.

In the preceding command, no --collection option is given, so Fauna Shell creates a new collection called zipcodes.

Import a JSON file and append to an existing collection

A file named zipcodes2.json has the following:

{ "zipcode" : "01010", "city" : "BRIMFIELD", "pop" : 3706, "state" : "MA" }
{ "zipcode" : "01011", "city" : "CHESTER", "pop" : 1688, "state" : "MA" }
{ "zipcode" : "01012", "city" : "CHESTERFIELD", "pop" : 177, "state" : "MA" }

The following terminal command imports zipcodes2.json and appends the documents to the existing collection zipcodes:

fauna import --path=./zipcodes2.json --collection=zipcodes --append
Database connection established
Starting Import!
 ›   Success: Import from ./zipcodes2.json to zipcodes completed. 3 rows/object imported.

Import a JSON file with configuration options

The following terminal command overrides any configuration options in the configuration file:

fauna import --path=./zipcodes.json --endpointURL=https/db.us.fauna.com:8443 --secret=secret
Database connection established
Starting Import!
 ›   Success: Import from ./zipcodes.json to zipcodes completed. 5 rows/object imported.

Import a CSV file with type conversions

The following CSV document has three string values:

myDate,myBool,myNumber
"May 3, 2021",true,15338

To convert those string values to other types, you can use the --type option.

fauna import --path=./myFile.json --type=myDate::date --type=myBool::bool --type=myNumber::number
Database connection established
Starting Import!
Warning: the string 'May 3, 2021' isn't valid ISO-8601 nor RFC_2822 date. Making a best-effort translation to 'Mon May 03 2021 00:00:00 GMT-0700 (Pacific Daylight Saving Time)'
 ›   Success: Import from ./myFile.csv to myFile completed. 1 rows/object imported.

Import a directory of source files

A directory named source_files has the following files:

myJSONfile1.json
myJSONfile2.json
myCSVfile1.csv
myCSVfile2.csv

The following command imports four files and creates four new collections:

fauna import --path=./source_files
Database connection established
Starting Import!
 ›   Success: Import from ./source_files/myCSVfile1.csv to myCSVfile1 completed. 8 rows/object imported.
Starting Import!
 ›   Success: Import from ./source_files/myCSVfile2.csv to myCSVfile2 completed. 14 rows/object imported.
Starting Import!
 ›   Success: Import from ./source_files/myJSONfile1.json to myJSONfile1 completed. 10 rows/object imported.
Starting Import!
 ›   Success: Import from ./source_files/myJSONfile2.json to myJSONfile2 completed. 10 rows/object imported.

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!