Perform the upgrade

Get current Helm user values

The Helm user values file is a YAML file that defines the configuration values to be applied when upgrading a Kubernetes cluster. The file will look similar to the following:

    global: 
      image: 
        tag: "2024.12.02" 
      strategy: 
        type: Recreate 

    application: 
      cluedin: 
        image: 
          tag: "2024.12.02" 
        components: 
          packages: 
          - name: "CluedIn.Connector.AzureEventHub" 
            version: "4.0.1" 
          - name: "CluedIn.Connector.Dataverse" 
            version: "4.4.0" 
          - name: "CluedIn.Connector.Http" 
            version: "4.0.0" 

We recommend the following format for naming your user values files: values-<environment>-<release-version>.yml, where:

<environment> – The target environment (for example, dev, staging, or prod).
<release-version> – The release identifier, written as a hyphen-separated date or version number.

For example, for release 2024.12.02 on the dev environment, the file should be named values-dev-2024-12-02.yml.

To get your Helm user values, run the following command in PowerShell (replace <environment> and <release-version> with your values):
```
 helm get values cluedin-platform -n cluedin -o yaml > ./ values-<environment>-<release-version>.yml 
```
YAML file is created in your working directory. Open the file in your preferred IDE (we typically use Visual Studio Code).
Check the contents of the file. It should look similar to the example above. If it appears empty, this usually means you are not connected to the cluster correctly. In that case, revisit the earlier steps to verify your connection.

Prepare new Helm user values

Make sure that your packages are ready to be installed in the new version. For details, see Prepare and test custom packages.
The required values for the target release are published in CluedIn documentation. Before proceeding, carefully verify that your current release is compatible with the target release. Once verified, duplicate the values file created in the previous step and rename it to match the target release.

For example, to go to release 2025.05.00 on the dev environment, the file should be named values-dev-2025-05-00.yml.
Carefully compare your existing values file with the new release’s required values. Update any outdated entries and add newly introduced values:
- In most cases, you will update container image tags and package versions, but additional configuration keys may also be required.
- Platform version (specified in tag) must align with the specified package versions.
- To learn which package version must be used, see the Release notes section. The Release notes column points to the release details.
  
  Package versions are listed in the Packages section of the release details page.
- An IDE such as Visual Studio Code will highlight any formatting or indentation issues as you edit the YAML file.
  
  YAML is whitespace-sensitive—use spaces (not tabs) and ensure correct indentation, or the deployment will fail.

Once everything looks correct, save the file.

Based on the earlier example, your YAML file should now look like this:

     global: 
       image: 
         tag: "2025.05.00" 
       strategy: 
         type: Recreate 
     
     application: 
       cluedin: 
         components: 
           packages: 
           - name: "CluedIn.Connector.AzureEventHub" 
             version: "4.5.0" 
           - name: "CluedIn.Connector.Dataverse" 
             version: "4.5.0" 
           - name: "CluedIn.Connector.Http" 
             version: "4.5.0" 

Perform system pre-checks

Complete the following checks:
- Verify that CluedIn UI is running correctly.
- Confirm that all pods are in a healthy (green) state.
- Review the server logs and ensure they are free of errors.
If any issues are detected, it is recommended to resolve them before proceeding with the upgrade. In some cases, the upgrade itself may address certain problems, but if you’re uncertain, it’s best to consult CluedIn support team for guidance.
Check CluedIn workload. Determine whether CluedIn is currently processing a high volume of data. If the system is under heavy load, it is best to allow all data to finish processing before performing the upgrade.

Although any data still in the queues should remain forward-compatible, reducing the workload helps minimize risk during the upgrade.
Check internal CluedIn queues.

Perform Helm upgrade

There are several methods to upgrade Helm:

Basic Helm upgrade – This is the default method. Use it unless you are explicitly instructed otherwise.
Helm upgrade with data upgrade – Use this method only when specifically instructed to do so.

Basic Helm upgrade

During a Helm upgrade, the UI will be temporarily unavailable. Make sure to notify all users in advance so that they are aware of the downtime.

A standard CluedIn upgrade typically results in 20–30 minutes of downtime. If the upgrade includes data migrations or additional updates, the outage may take longer.

To perform a basic Helm upgrade

Check the current CluedIn Helm chart version. To do this, run the following command to list all Helm releases in the cluedin namespace:
```
 helm list -a -n cluedin 
```
The output will display the currently installed chart version. For example:
```
 cluedin-platform-2.5.1 
```
This sample output indicates that the current Helm chart version in use is 2.5.1.
Prepare the Helm command. The exact Helm command for the current update will be provided in the target documentation. In general, the command follows this structure:
```
 # ${CustomValues} refers to the values file you have amended with the above changes. Please type the full path here. 
    
 helm upgrade cluedin-platform -n cluedin cluedin/cluedin-platform \ 
     --version {CluedInChartVersion} \ 
     --values ${CustomValues} \ 
     --set application.system.runDatabaseJobsOnUpgrade=true \ 
     --set application.system.runNugetFullRestore=true \ 
     --wait \ 
     --timeout 10m0s 
```
Parameter details:
- {CluedInChartVersion} – The target chart version specified in the documentation.
- ${CustomValues} – The full path to the user values file you created in the previous step.
- runDatabaseJobsOnUpgrade=true – Ensures that the database is refreshed during the upgrade.
- runNugetFullRestore=true – Ensures that all packages are fully restored.
- --timeout 10m0s – A best practice is to allow a 10-minute timeout. If the upgrade fails due to a timeout, follow the documented mitigation steps.
When you are ready, execute the Helm command in PowerShell.
- The PowerShell prompt will remain active for several minutes—this is expected. Please be patient while the process runs.
- During the upgrade, the Helm release status will progress through the following states: Active → Pending Upgrade → Upgrading.
Allow the process to complete without interruption.
- After about 10 minutes, the Helm command in PowerShell should complete and display a confirmation message. This indicates that the Helm command executed successfully.
- However, while the Helm upgrade itself will be finished, some pods may still be starting up. It can take an additional 10–15 minutes for all pods to become fully healthy (green).
Verify that CluedIn Helm chart version matches the target version. Run the following command:
```
 helm list -a -n cluedin 
```
The output should display the newly installed chart version. For example:
```
 cluedin-platform-2.5.2 
```

Helm upgrade with data upgrade

In some cases, an upgrade also requires changes to the underlying data. When this occurs, CluedIn must be placed into the Upgrade Mode during installation. Upgrade Mode ensures that no data is ingested or processed while the upgrade is in progress, preventing inconsistencies and maintaining data integrity.

During a Helm upgrade, the UI will be temporarily unavailable. Make sure to notify all users in advance so that they are aware of the downtime.

A standard CluedIn upgrade typically results in 20–30 minutes of downtime. If the upgrade includes data migrations or additional updates, the outage may take longer.

Use this method of upgrading Helm only if you are explicitly instructed to do so. It replaces the steps provided in Basic Helm upgrade.

To perform Helm upgrade with data upgrade

Verify the current CluedIn Helm chart version. Run the following command to list all Helm releases in the cluedin namespace:
```
 helm list -a -n cluedin 
```
The output will display the currently installed chart version. For example:
```
 cluedin-platform-2.5.1 
```
This sample output indicates that the current Helm chart version in use is 2.5.1.
Prepare the Helm command:
```
 # ${CustomValues} refers to the values file you have amended above. Enter the full path here. 
    
 helm upgrade cluedin-platform -n cluedin cluedin/cluedin-platform \
      --version {CluedInChartVersion} \ 
      --values ${CustomValues} \ 
      --set application.system.upgradeMode=true \
      --set application.system.runDatabaseJobsOnUpgrade=true \
      --set application.system.runNugetFullRestore=true \
      --wait \
      --timeout 10m0s
```
Parameter details:
- {CluedInChartVersion} – The target chart version specified in the documentation.
- ${CustomValues} – The full path to the user values file created in the previous step.
- upgradeMode=true – Ensures no data is ingested or processed during the upgrade.
- runDatabaseJobsOnUpgrade=true – Ensures the database is refreshed as part of the upgrade.
- runNugetFullRestore=true – Ensures all packages are fully restored.
- --timeout 10m0s – Allows a 10-minute timeout (recommended). If the upgrade fails due to a timeout, follow the documented mitigation steps.
Run the command. After a few minutes, it should successfully complete.
Wait until the deployment finishes. Make sure that all pods are healthy and all jobs are completed.

Do not proceed further if any of the pods have health issues.

Run the data upgrade. The data upgrade requires a trigger which is applied by running the following command:

 kubectl apply -f - <<EOF
 apiVersion: api.cluedin.com/v1
 kind: DataUpgrade
 metadata:
   annotations:
     meta.helm.sh/release-name: cluedin-platform
     meta.helm.sh/release-namespace: cluedin
   labels:
     app: cluedin
     app.kubernetes.io/instance: cluedin-platform
     app.kubernetes.io/managed-by: Helm
     app.kubernetes.io/name: application
     helm.sh/chart: application-2.5.1
     release: cluedin-platform
   name: data-upgrade-450
   namespace: cluedin
 spec:
   toVersion: Version450
 EOF

Check the CluedIn server pod logs. The logs should display the following (the Version450 label will vary):

 [#098 05:57:02 INF] Performing Upgrade Scenario Apply DataSource and Manual Data Entry Project Owners Upgrade for version Version450
 [#090 05:57:03 INF] Completed Upgrade Scenario Apply DataSource and Manual Data Entry Project Owners Upgrade for version Version450
 [#090 05:57:03 INF] Performing Upgrade Scenario Elastic search upgrade mapping for version Version450
 [#095 05:57:04 INF] Completed Upgrade Scenario Elastic search upgrade mapping for version Version450
 [#095 05:57:04 INF] HTTP POST /api/upgradeto/Version450 responded 200 in 1857.3003 ms

The responded 200 part indicates a successful upgrade.

Finalise the upgrade. CluedIn must now be taken back out of the Upgrade Mode. To bring CluedIn back online, run the following command:

  # ${CustomValues} refers to the values file you have amended above. Enter the full path here.
    
  helm upgrade cluedin-platform -n cluedin cluedin/cluedin-platform \
    --version {CluedInChartVersion} \ 
    --values ${CustomValues} \ 
    --set application.system.upgradeMode=false \
    --set application.system.runDatabaseJobsOnUpgrade=false \
    --set application.system.runNugetFullRestore=false

After a few minutes, the command should complete.

Verify the upgrade

To monitor the upgrade progress, check the following:
- CluedIn pods
- CluedIn server logs
  
  When checking the server logs, look for the following message:
```
  Application started
```
  This indicates a successful startup.
Check CluedIn UI and ensure everything is running smoothly.

Notify about upgrade completion

Once the upgrade completes and the cluster is back online, send a notification to all relevant teams and stakeholders to inform them that the cluster is ready for use.