Additional operations
In this section, you will learn how to improve the quality of your data in the Data Sources module in CluedIn.
Although normalizing, transforming, and improving the quality of records before processing is optional, we recommend that you do it for several reasons:
-
Ensure alignment with your data normalization polices.
-
Get better matches in deduplication projects.
-
Reduce the number of records to clean.
-
Optimize the streaming of records.
CluedIn provides the following tools that you can use to enhance the quality of your data before processing:
-
Property rules – normalize and transform property values of mapped records.
-
Pre-process rules – improve the overall quality of mapped records.
-
Advanced mapping code – modify clues programmatically by applying complex conditions.
-
Quarantine – handle records that do not meet certain conditions set in property rules, pre-process rules, or advanced mapping.
Additionally, you will learn how to interpret logs and monitoring statistics to get an insight into what is going on with your records.