Creating Data Quality

Creating Data Quality

This section provides you with detailed guidance and support for navigating and effectively using the Infoveave Data Quality creation features.

Creating Data Quality Manually

1. Start by clicking the New Data Quality button.
2. A dialog box opens where you can select a specific data quality type such as Data Quality using AI or Data Quality using the toggle button before moving to the next step.
3. After selecting the data quality type, choose the connection and table to be used for the data quality checks and click Next.
   
4. In the Infoboard Setup tab, enter the name and description for the data quality.
   
5. Drag a column from the Columns tab into the designer.
   
6. Click the Add rule button to add and configure the rules.
   
7. In the rule setup screen, select rules for data quality such as accuracy, completeness, validity, and more.
   
8. After adding the rules, click Save. Once saved, configure each rule and click the Validate button.
   
9. After all rules are configured and validated, click the Validate rules icon to view the validation results.
10. Once validation is complete, you see the results displayed in a table showing the success percentage and other details of the rule configurations. If necessary, you can tweak the rules and validate them again.
11. After validating the rules, click Save to save the configuration.
12. Click Execute Data Quality to run the configured data quality checks.
    
13. The execution results appear in a table showing success rates and other details for each rule. You can monitor progress and ensure the checks are completed successfully.
    
14. You can also schedule your Data Quality execution from the Schedules tab. After adding a schedule, click Save.

Creating Data Quality Using AI

1. Start by clicking the New Data Quality button.
2. In the dialog box that appears, enable the AI enabled toggle.
3. After selecting the data quality type, choose the connection and table to be used for the data quality checks and click Next.
   
4. An AI-generated description is automatically filled in the pop-up. You can choose to share catalogue information with AI by checking the checkbox. Click the Generate Data Quality Rules button.
   
5. AI generates data quality rules for the selected dataset to ensure that the data meets the required standards and is ready for analysis.
   • Each rule includes a relevant Column name to ensure accurate validation
   • A Citation and Rule Group explains the rule’s purpose, such as checking for uniqueness, non-null values, or correct formats for columns like IDs or dates
   • Each rule has a unique Rule name and Rule type
     After reviewing the rules, click Generate Data Quality to apply the specified checks across the dataset.
     
6. The data quality name and description are generated automatically by AI.
   
7. Click the Save button. After saving, you can Validate each rule.
   
8. After validating each rule, click the Validate rules icon to view the results.
9. Once validations are complete, you see the results in a green table showing success percentages and other rule details. If needed, you can adjust the rules and validate again.
10. After validating the rules, click Save to save the configuration.
11. Click Execute Data Quality to run the configured data quality checks.
    
12. The execution results are displayed in a table with success rates and details for each rule. You can track progress and confirm that the data quality checks are successfully completed.
    
13. You can also schedule your Data Quality execution from the Schedules tab. After adding the schedule, click Save.