Read JSON files

Description

The Read JSON files activity reads JSON files using DuckDB’s read_json_auto function and returns the result as workflow data. It can read multiple JSON files, merge their rows, and align columns found across files.

Supported Features

• Multiple file reading: Reads each input .json file and appends the rows.
• DuckDB JSON auto-detection: Uses read_json_auto.
• Configurable DuckDB parameters: Pass parameters such as format, records, columns, or other supported DuckDB JSON reader options.
• Default parameters: Adds auto_detect = true and maximum_object_size = uint.MaxValue when they are not supplied.
• Column alignment: Merges files and includes all columns discovered across successful reads.
• Preview limit support: Stops reading more files when the preview limit is reached.

---

Input

Type	Required	Description
Files	Yes	One or more .json files. Non-JSON files are skipped with an error.

Input Scenarios

1. Single JSON File

{
  "Files": [
    { "FileName": "customers.json", "FullPath": "C:/Work/customers.json" }
  ]
}

2. Multiple JSON Files With Different Columns

{
  "Files": [
    { "FileName": "customers-a.json", "FullPath": "C:/Work/customers-a.json" },
    { "FileName": "customers-b.json", "FullPath": "C:/Work/customers-b.json" }
  ]
}

3. Mixed File Types

Non-JSON files are not read and produce an error such as orders.csv is not of type json.

---

Output

Field	Type	Description
Data	Array	Rows returned by DuckDB from all successfully read JSON files.
Errors	Array	Errors for invalid file types or failed reads.

Example Output

{
  "Data": [
    {
      "CustomerId": 1,
      "Name": "Alice",
      "City": "Boston"
    },
    {
      "CustomerId": 2,
      "Name": "Bob",
      "City": "Chicago"
    }
  ],
  "Errors": []
}

---

Configuration Fields

Field Name	Type	Required	Description
Parameters	Dictionary	No	Additional parameters passed directly to DuckDB read_json_auto. If auto_detect is not present, the activity adds auto_detect = true. If maximum_object_size is not present, the activity adds maximum_object_size = uint.MaxValue.

Conditional Field Rendering Rules

No conditional configuration fields are defined for this activity.

---

Sample Input

customers.json

[
  { "CustomerId": 1, "Name": "Alice", "City": "Boston" },
  { "CustomerId": 2, "Name": "Bob", "City": "Chicago" }
]

orders.json

[
  { "OrderId": "O-001", "CustomerId": 1, "Amount": 120.5 },
  { "OrderId": "O-002", "CustomerId": 2, "Amount": 89.0 }
]

---

Sample Configuration

Field	Value
Parameters	auto_detect = true, maximum_object_size = 4294967295

---

Sample Output

{
  "Data": [
    {
      "CustomerId": 1,
      "Name": "Alice",
      "City": "Boston"
    },
    {
      "CustomerId": 2,
      "Name": "Bob",
      "City": "Chicago"
    }
  ],
  "Errors": []
}