Additional operations
In this section, you will learn how to improve the quality of your data in the Data Sources module in CluedIn.
Although normalizing, transforming, and improving the quality of records before processing is optional, we recommend that you do it for several reasons:
-
Ensure alignment with your data normalization polices.
-
Get better matches in deduplication projects.
-
Reduce the number of records to clean.
-
Optimize the streaming of records.
CluedIn provides the following tools that you can use to enhance the quality of your data before processing:
-
Preview – analyze the uploaded records and improve their quality before processing.
-
Validations – check the records for errors, inconsistencies, and missing values and fix these issues to improve the quality of the records.
-
Property rules – normalize and transform property values of mapped records.
-
Pre-process rules – improve the overall quality of mapped records.
-
Advanced mapping code – modify clues programmatically by applying complex conditions.
-
Quarantine – handle records that do not meet certain conditions set in property rules, pre-process rules, or advanced mapping.
-
Approval – approve or reject specific records to ensure that only verified records are sent for processing.
You will learn how to interpret logs and monitoring statistics to get an insight into what is going on with your records. Additionally, you will learn about the removal of records that were created from a specific data source.