2024.01

Prerequisites

Access to the helm chart version 2.0.0. You may need to helm repo update to grab the latest.
Download the prerequisite zip artifact here
kubectl and helm

Guide

Included in the zipped artifact are a number of .yaml files along with a PowerShell upgrade script. The script is very generic and may not be usable in your environment if your instance of CluedIn is heavily modified. As a result, it should only be used as a reference point rather than something you run. Manual upgrade steps are provided below.

Before any upgrade takes place, ensure you have taken a snapshot or backup of all your running PVCs in your AKS cluster. This is very important as databases may be upgraded during this process.

Preparation

Export your current running helm values files. Keep one for this process, and also keep one as a backup.

Note: You can export by running helm get values cluedin-platform -n cluedin --output yaml > /path/to/file.yaml. This will export the currently connected kubeconfig.

Extract the downloaded artifact to a location where we’ll be working from.
Before the upgrade takes place, you need to apply the data upgrade CRD. You can do this by running kubectl apply -f /path/to/extract/artifact/dataupgrade-crd.yaml -n cluedin. You do not need to modify this file.
Update the platform-custom-values.yaml patch file to include the neo4j password and potentially change the global image tag as it’s shipped with 2024.01.00. This file is part of the extracted artifact. The field is infrastructure.neo4j.neo4j.password. It should be set to xxx in the file, and failing to update this will potentially break the upgrade process.

Note: You can get the current neo4j password by running kubectl get secret/cluedin-neo4j-secrets -n cluedin -o jsonpath='{.data.neo4j-password}' | base64 --decode

In the same file, remove any application.cluedin.components.packages which you do not use. This will patch against what is set in your exported values file.

Now, you are ready to perform the actual upgrade process. It’s done in 3 stages.

Upgrade - Stage 1

Scale down a majority of the running deployments and stateful sets. To do this, run the following:

helm upgrade cluedin-platform -n cluedin cluedin/cluedin-platform \
    --version 2.0.0 \
    --values ${ExportedOriginalValues} \
    --values platform-custom-values.yaml \
    --values platform-upgrade-v2-stage1.yaml \
    --wait \
    --timeout 10m0s

When the above has been ran, you should begin to see a majority of your pods scaling down. It may take 5 minutes, so please be patient. When it has completely scaled down, you’re ready for Stage 2.

Upgrade - Stage 2

Migrate the databases due to a large number of application upgrades. To do this, run the following:

helm upgrade cluedin-platform -n cluedin cluedin/cluedin-platform \
    --version 2.0.0 \
    --values ${ExportedOriginalValues} \
    --values platform-custom-values.yaml \
    --values platform-upgrade-v2-stage2.yaml \
    --wait \
    --timeout 20m0s

This will trigger a neo4j database backup and migration along with a SQL upgrade. It can take up to 20 minutes to complete. It’ll be ready when all init-jobs have completed and the core pods are all in a running state. Please note because of the previous step scale down, only a few pods will be brought up during this stage. When successfully done, it’s ready for the final stage.

Upgrade - Stage 3

Assuming the above has been successful, it’s time to actually bring the environment back up into a running and usable state. To do this, run the final helm command below:

helm upgrade cluedin-platform -n cluedin cluedin/cluedin-platform \
    --version 2.0.0 \
    --values ${ExportedOriginalValues} \
    --values platform-custom-values.yaml \
    --set 'application.system.runDatabaseJobsOnUpgrade=true' \
    --set 'application.system.runNugetFullRestore=true' \
    --wait \
    --timeout 20m0s

After approximately 10-15 minutes, your environment should be 100% completed, and you should be able to access the front end of the UI.

Troubleshooting

If you get stuck at all during the upgrade process, reach out to support@cluedin.com. Below are some common issues due to the upgrade process.

Ensure any custom connectors are updated to .NET 6. This is due to upgrading the libraries within the application to .NET 6. For some connectors, data may not appear. Note that no data should be lost during this transition, it may just be hidden until updated.
CluedIn Developer feed is no longer supported. If your environment connects to the CluedIn Developer nuget feed, it may prevent the environment from standing up correctly. You can validate this by searching for https://pkgs.dev.azure.com/CluedIn-io/_packaging/develop/ in your values files. If a result is returned, it’s best to remove it from values.