2024.12
On this page
This document covers the upgrade process from 2024.07 to 2024.12.
We do not support upgrading directly to this version from prior to 2024.07.
Prerequisites
- Access to the helm chart version
2.4.1. You may need tohelm repo updateto grab the latest. kubectlandhelm
Guide
Before any upgrade takes place, ensure you have taken a snapshot or backup of all your running PVCs in your AKS cluster. This is very important as databases may be upgraded during this process.
Stage 1: Prepare for Upgrade
- Connect to your cluster via the kubeconfig.
- Export your current running helm values file. Keep one for this process, and also keep one as a backup.
-
In the values file we’ll be working with, update the following properties:
global: image: tag: "2024.12" strategy: type: Recreate application: cluedin: image: tag: 2024.12 components: packages: - name: "CluedIn.Connector.AzureEventHub" version: "4.0.1" - name: "CluedIn.Connector.Dataverse" version: "4.4.0" - name: "CluedIn.Connector.Http" version: "4.0.0" - name: "CluedIn.DataVerse" version: "4.2.2" - name: "CluedIn.EventHub" version: "4.4.0" - name: "CluedIn.Vocabularies.CommonDataModel" version: "4.4.0" - name: "CluedIn.Connector.AzureDataLake" version: "4.4.0" - name: "CluedIn.Connector.AzureDedicatedSqlPool" version: "4.0.2" - name: "CluedIn.Connector.AzureServiceBus" version: "4.0.2" - name: "CluedIn.Connector.OneLake" version: "4.4.0" - name: "CluedIn.ExternalSearch.Providers.DuckDuckGo.Provider" version: "4.4.0" - name: "CluedIn.ExternalSearch.Providers.PermId.Provider" version: "4.4.0" - name: "CluedIn.ExternalSearch.Providers.Web" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.Bregg" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.ClearBit" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.CompanyHouse" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.CVR" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.Gleif" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.GoogleMaps" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.KnowledgeGraph" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.Libpostal" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.OpenCorporates" version: "4.4.0" - name: "CluedIn.Provider.ExternalSearch.Providers.VatLayer" version: "4.4.0" - name: "CluedIn.Purview" version: "4.4.0" - name: "CluedIn.PowerApps" version: "4.4.0" - name: "CluedIn.EnterpriseFlows.PowerAutomate" version: "4.4.0" - name: "CluedIn.Connector.SqlServer" version: "4.1.1" submitter: resources: requests: memory: 1G limits: memory: 1G -
Use the Elasticsearch custom init container values from elasticsearch-upgrade.yml .
- Execute the following Helm command:
# ${CustomValues} refers to the values file you have amended with the above changes. Please type the full path here. helm upgrade cluedin-platform -n cluedin cluedin/cluedin-platform \ --version 2.4.1 \ --values ${CustomValues} \ --values elasticsearch-upgrade.yml \ --set application.system.upgradeMode=true \ --set application.system.runDatabaseJobsOnUpgrade=true \ --set application.system.runNugetFullRestore=trueAfter a few minutes, it should successfully complete.
-
Validation:
- ❗ Wait until deployment finishes. Make sure that all pods are healthy and all jobs are completed.
- ❗ Don’t proceed further if any of the pods have health issues. Rather investigate the issues. This will apply dynamic mappings to every CluedIn index
- If any errors occur whilst running the elastic search upgrade we restart the pod and try to recover. You can find saved logs in /usr/share/elasticsearch/data/
- ⚠ We also store a list of indexes and what stage they last completed on the last run through and some completion files that will stop us reprocessing things that have already completed the upgrade within that folder.
Stage 2: Run Data Upgrade
- Apply data upgrade by running following command:
kubectl apply -f - <<EOF apiVersion: api.cluedin.com/v1 kind: DataUpgrade metadata: annotations: meta.helm.sh/release-name: cluedin-platform meta.helm.sh/release-namespace: cluedin labels: app: cluedin app.kubernetes.io/instance: cluedin-platform app.kubernetes.io/managed-by: Helm app.kubernetes.io/name: application helm.sh/chart: application-2.4.1 release: cluedin-platform name: data-upgrade-440 namespace: cluedin spec: toVersion: Version440 EOF -
Validation:
- ❗ Confirm the data upgrade completed successfully.
- When the data upgrade is completed the cluedin-server pod should have below logs.
[#093 12:12:33 INF] Performing Upgrade Scenario Remove legacy DeduplicateEntities jobs for version Version440 [#093 12:12:33 INF] Completed Upgrade Scenario Remove legacy DeduplicateEntities jobs for version Version440 [#093 12:18:33 INF] Performing Upgrade Scenario Migrate clean projects to dedicated hashes table for version Version440 [#124 12:18:33 INF] Completed Upgrade Scenario Migrate clean projects to dedicated hashes table for version Version440 [#124 13:03:17 INF] Performing Upgrade Scenario Elastic Search Repopulate Index for version Version440 [#067 13:03:17 INF] Completed Upgrade Scenario Elastic Search Repopulate Index for version Version440 [#067 13:03:17 INF] HTTP POST /api/upgradeto/Version440 responded 200 in 3044054.6755 ms
Stage 3: Finalize Upgrade
-
Run the final stage of the upgrade to bring the system back online (disable upgrade mode):
# ${CustomValues} refers to the values file you have amended with the above changes. Please type the full path here. helm upgrade cluedin-platform -n cluedin cluedin/cluedin-platform \ --version 2.4.1 \ --values ${CustomValues} --set application.system.upgradeMode=false \ --set application.system.runDatabaseJobsOnUpgrade=false \ --set application.system.runNugetFullRestore=false -
Validation:
- Ensure no errors in the pod logs.
- Verify the server starts in normal mode.
- Ensure cluedin-ui pod is Running..
If there are any issues during the upgrade process, please do not hesitate to reach out to CluedIn support.