AI assistants: this page is available as Markdown at https://documentation.cluedin.net/publish/streams/tutorial-stream-data.md. The index of the whole documentation, with instructions for AI assistants, is at https://documentation.cluedin.net/llms.txt.

Tutorial: stream data

tutorial Guided end-to-end learning

On this page

1. Set up an export target
2. Create a stream
3. Results & next steps

Streaming data from CluedIn involves setting up an export target and creating a stream.

In this guide, you will learn how to stream your records from CluedIn to a Microsoft SQL Server database.

Before you start: Make sure you have completed all steps in the Ingest data guide.

Set up an export target

An export target is a place where you can send the data out of CluedIn after it has been processed. In the following procedure, we are going to establish a connection with a Microsoft SQL Server database to use it as an export target.

To set up an export target

1. On the navigation pane, go to Consume > Export Targets.

2. Select Add Export Target.

3. On the Choose Target tab, find and select Sql Server Connector. Then, select Next.

   

4. On the Configure tab, enter the database connection details such as Name, Host, Database Name, Username, and Password. Optionally, you may add Port Number, Schema, and Connection pool size.

   

5. Select Test Connection. After you receive a notification that the connection is successful, select Add.

   The export target is added.

   

   Now, you can create a stream.

Create a stream

A stream is a trigger that starts the process of sending the data to the export target. In the following procedure, we are going to configure the stream and define the records that will be streamed to a Microsoft SQL Server database.

To create a stream

1. On the navigation pane, go to Consume > Streams.

2. Select Create Stream.

3. Enter the Stream Name, and then select Create.

   The stream details page opens.

4. On the Configuration tab, in the Filters section, select Add First Filter, and then specify what data you want to share.

   

   The fields for configuring a filter appear one by one. After you complete the previous field, the next field appears.

5. Select Save. To view the records that match stream filters, go to the Preview Condition tab.

6. Go to the Export Target Configuration tab.

7. On the Choose Connector tab, select the Sql Server connector, and then select Next.

   

8. On the Connector Properties tab, do the following:

   1. Enter the Target Name. This will be the name of the table in the database.

   2. Select the Streaming Mode. Two streaming modes are available:

      • Synchronized stream – the database and CluedIn contain the same data that is synchronized. For example, if you edit the record in CluedIn, the record is also edited in the database.

      • Event log stream – every time you make a change in CluedIn, a new record is added to the database instead of replacing the existing record.

      

9. On the Properties to Export tab, click Auto-select. All vocabulary keys associated with the records in the strem filter will be displayed on the page.

   

10. Select Save. To view the data that will be sent to the database, go to the Data tab.

11. Start the stream by selecting the play button. Then, confirm that you want to start the stream.

    

    The Streaming and Exporting labels appear under the stream name, indicatig that the stream is active.

    

    On the Monitoring tab, you can view different data performance metrics.

12. Go to the database and open the table.

    

    The records have been sent to the database.

Results & next steps

After completing all steps outlined in this guide, you learned how to configure the connection to a Microsoft SQL Server database and how to send data from CluedIn to the database. If you make any changes to the records in CluedIn, they will be automatically updated in the database. For example, if you fix the job title values in CluedIn, they will be automatically corrected in the database.

Next, learn how to create rules for automating data transformation and capturing data quality issues in the Create rules guide.