Description
The Filter on Format activity validates values in a selected column against a configured format and applies the selected filter action. It can keep matching rows, remove matching rows, clear matching or non-matching cells, or add a boolean flag column.
Supported Features
- Format validation: Validate values as
Text, Integer, Decimal, Boolean, Date, Url, EmailAddress, IpV4Address, IpV6Address, or Currency.
- Row filtering: Keep only matching rows or remove matching rows.
- Cell clearing: Clear matching cells or non-matching cells.
- Row flagging: Add a flag column showing whether the value matched the selected format.
- Type-aware checks: Text, integer, decimal, boolean, and date use cast-based checks; URL, email, IP, and currency use regex checks.
| Type | Required | Description |
|---|
| Data | Yes | Input rows containing the column to validate. |
1. Keep Valid Values
Use KeepMatchingRowsOnly to keep rows where the selected column matches the chosen format.
2. Remove Valid Values
Use RemoveMatchingRows to remove rows where the selected column matches the chosen format.
3. Clear Cells
Use ClearContentOfMatchingCells or ClearContentOfNonMatchingCells to set selected column values to null without removing rows.
4. Flag Rows
Use FlagRows to add a boolean output column indicating whether the selected column matches the format.
Output
| Field | Type | Description |
|---|
| Data | Array | Transformed rows after the selected filter action is applied. |
Configuration Fields
| Field Name | Type | Required | Description |
|---|
| Column | Previous Data Column | Yes | Column to validate against the selected format. |
| Format | Dropdown | Yes | Expected format. Options: Text, Integer, Url, Boolean, Currency, IpV4Address, IpV6Address, Date, EmailAddress, Decimal. |
| Action | Dropdown | Yes | Action to apply. Options: KeepMatchingRowsOnly, RemoveMatchingRows, ClearContentOfMatchingCells, ClearContentOfNonMatchingCells, FlagRows. |
| FlagRowsColumnName | Text | Conditional | Name of the flag column. Used only when Action is FlagRows. If empty at execution, the transformer uses <FuseDataRowFlag>. |
Conditional Field Rendering Rules
- FlagRowsColumnName is shown when Action is
FlagRows.
Sample Configurations
Scenario 1: Flag Valid Email Addresses
| Field | Value |
|---|
| Column | Email |
| Format | EmailAddress |
| Action | FlagRows |
| FlagRowsColumnName | IsValidEmail |
Scenario 2: Keep Only Valid Email Addresses
| Field | Value |
|---|
| Column | Email |
| Format | EmailAddress |
| Action | KeepMatchingRowsOnly |
Sample Output
Output for Scenario 1
Output for Scenario 2
