Upgrade AIStor on Kubernetes in Airgapped Environments

This page describes how to upgrade AIStor Helm deployments in completely airgapped environments with no direct internet access.

If your deployment has internet access, see:

Overview

AIStor Helm upgrades involve updating two components:

Operator Chart - Controls CustomResourceDefinitions and core operator resources
Object Store Chart - Deploys and manages the AIStor Object Store instances

In airgapped environments, you must perform preparation steps on a system with internet access, then transfer the artifacts (container images and Helm charts) to your airgapped environment. This page covers the complete upgrade workflow for airgapped deployments.

All AIStor software supports non-disruptive upgrades with zero downtime. In optimal environments, cluster-wide upgrades typically complete in under 500 milliseconds with large clusters (1000+ nodes) completing in less than 5 seconds.

S3 SDKs and applications typically implement retry mechanisms that mitigate the impact of any reduced availability.

Downgrades or rollbacks are not guaranteed due to potentially breaking changes in underlying resources like the AIStor CustomResourceDefinition.

Prerequisites

Complete all standard upgrade prerequisites, then prepare the following for your airgapped environment:

Identify a system with internet access for downloading images and charts
Prepare a method to transfer artifacts to the airgapped environment (removable media, secure file transfer, etc.)
Verify you have credentials and push access to your private container registry
Ensure your Kubernetes cluster has pull access configured for your private registry
Verify that image pull secrets exist in the aistor namespace with valid credentials:
```
kubectl get secret -n aistor private-repo-pull-secret
```
If the secret does not exist or credentials have changed, recreate it before upgrading

Step 1. Prepare operator artifacts (external system)

Perform these steps on a system with internet access.

Copy the operator image to your private registry.

Use skopeo to copy the operator image:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/operator:RELEASE.2025-XX-XXT00-00-00Z \
  docker://PRIVATE_REGISTRY/aistor/operator:RELEASE.2025-XX-XXT00-00-00Z

Replace:

RELEASE.2025-XX-XXT00-00-00Z with your target operator version
PRIVATE_REGISTRY with your private registry hostname

If your private registry is not accessible from this system, you can save the image as a tarball and transfer it:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/operator:RELEASE.2025-XX-XXT00-00-00Z \
  docker-archive:aistor-operator-RELEASE.2025-XX-XXT00-00-00Z.tar

Then transfer the tarball and load it into your private registry from within the airgapped environment.

Download the operator Helm chart.

Pull the updated Helm chart:
```
helm pull minio/aistor-objectstore-operator
```
This downloads a .tgz file (for example, aistor-objectstore-operator-6.0.5.tgz).
Transfer to airgapped environment.

Transfer the following to your airgapped environment:
- The operator container image (either pushed to your registry or as a tarball)
- The operator Helm chart .tgz file

Step 2: Deploy operator chart (airgapped system)

Perform these steps within your airgapped environment.

Verify the existing installation.

Check the health and status of the operator:
```
kubectl get all -n aistor
```

Upgrade the operator.

Choose one of the following methods based on how you store Helm charts in your airgapped environment:

OCI Registry Local Chart File

Replace the license value with your decoded JWT token from SUBNET.

Ensure your aistor-objectstore-operator-values.yaml file contains the repositories configuration pointing to your private registry. For example:

repositories:
  aistor:
    hostname: PRIVATE_CONTAINER_REGISTRY
    pathPrefix: aistor/
    imagePullSecrets:
    - name: private-repo-pull-secret
  minio:
    hostname: PRIVATE_CONTAINER_REGISTRY
    pathPrefix: minio/
    imagePullSecrets:
    - name: private-repo-pull-secret

Replace PRIVATE_CONTAINER_REGISTRY with the hostname of your private container registry (for example, registry.example.com or registry.example.com:5000).

If you need to override specific image versions, add an images section:

images:
  operator:
    repository: aistor
    image: operator:RELEASE.2025-XX-XXT00-00-00Z

Validate the operator upgrade.

Validate the installation by running the following command:

kubectl get all -n aistor

The output should show running pods similar to the following:

NAME                                         READY   STATUS    RESTARTS   AGE
   pod/adminjob-operator-cfc97d9f-hjbp5         1/1     Running   0          4m16s
   pod/object-store-operator-78c9f84b85-kmwlv   1/1     Running   0          4m16s
   
   NAME                            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
   service/object-store-operator   ClusterIP   10.43.210.230   <none>        4221/TCP   4m16s
   
   NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
   deployment.apps/adminjob-operator       1/1     1            1           4m16s
   deployment.apps/object-store-operator   1/1     1            1           4m16s
   
   NAME                                               DESIRED   CURRENT   READY   AGE
   replicaset.apps/adminjob-operator-cfc97d9f         1         1         1       4m16s
   replicaset.apps/object-store-operator-78c9f84b85   1         1         1       4m16s

Step 3: Prepare object store artifacts (external system)

Return to the system with internet access to prepare the object store artifacts.

Copy the core object store images.

Copy the main AIStor images to your private registry:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/minio:RELEASE.2025-XX-XXT00-00-00Z \
  docker://PRIVATE_REGISTRY/aistor/minio:RELEASE.2025-XX-XXT00-00-00Z

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/minio-sidecar:RELEASE.2025-XX-XXT00-00-00Z \
  docker://PRIVATE_REGISTRY/aistor/minio-sidecar:RELEASE.2025-XX-XXT00-00-00Z

Replace:

RELEASE.2025-XX-XXT00-00-00Z with your target version
PRIVATE_REGISTRY with your registry hostname

If your private registry is not accessible from this system, save the images as tarballs:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/minio:RELEASE.2025-XX-XXT00-00-00Z \
  docker-archive:aistor-minio-RELEASE.2025-XX-XXT00-00-00Z.tar

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/minio-sidecar:RELEASE.2025-XX-XXT00-00-00Z \
  docker-archive:aistor-minio-sidecar-RELEASE.2025-XX-XXT00-00-00Z.tar

Copy optional images.

If you use additional features, copy those images as well:

# KES (Key Encryption Service)
skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/kes:RELEASE.2025-XX-XXT00-00-00Z \
  docker://PRIVATE_REGISTRY/aistor/kes:RELEASE.2025-XX-XXT00-00-00Z

# MinIO Client (mc)
skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/mc:RELEASE.2025-XX-XXT00-00-00Z \
  docker://PRIVATE_REGISTRY/aistor/mc:RELEASE.2025-XX-XXT00-00-00Z

# AI Hub (if using AI features)
skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/aihub:RELEASE.2025-XX-XXT00-00-00Z \
  docker://PRIVATE_REGISTRY/aistor/aihub:RELEASE.2025-XX-XXT00-00-00Z

# Prompt (if using AI features)
skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/prompt:RELEASE.2025-XX-XXT00-00-00Z \
  docker://PRIVATE_REGISTRY/aistor/prompt:RELEASE.2025-XX-XXT00-00-00Z

# WARP (benchmarking tool)
skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/warp:RELEASE.2025-XX-XXT00-00-00Z \
  docker://PRIVATE_REGISTRY/aistor/warp:RELEASE.2025-XX-XXT00-00-00Z

Download the object store Helm chart.

Pull the updated Helm chart:
```
helm pull minio/aistor-objectstore
```
This downloads a .tgz file (for example, aistor-objectstore-6.0.5.tgz).
Transfer to airgapped environment.

Transfer the following to your airgapped environment:
- All object store container images (either pushed to your registry or as tarballs)
- The object store Helm chart .tgz file

Step 4: Deploy object store chart (airgapped system)

Return to your airgapped environment to deploy the object store chart update.

Verify the existing installation.

Check the health and status of your object store:
```
kubectl get all -n OBJECT-STORE-NAMESPACE
```
Replace OBJECT-STORE-NAMESPACE with your object store’s namespace.
Upgrade the object store chart.

Choose one of the following methods based on how you store Helm charts:
OCI Registry
```
helm upgrade -n OBJECT-STORE-NAMESPACE OBJECT-STORE-NAME oci://PRIVATE_REGISTRY/helm-charts/aistor-objectstore \
  -f aistor-objectstore-values.yaml
```
Local Chart File
```
helm upgrade -n OBJECT-STORE-NAMESPACE OBJECT-STORE-NAME ./aistor-objectstore-*.tgz \
  -f aistor-objectstore-values.yaml
```
Replace:
- OBJECT-STORE-NAMESPACE with your object store’s namespace
- OBJECT-STORE-NAME with your object store’s Helm release name
- aistor-objectstore-values.yaml with the path to your values file

You must pass in your existing chart values file to ensure your customized settings are preserved during the upgrade.

Your values file should already contain the repositories configuration from your initial installation. The Helm chart automatically references images from these repositories. For example:

objectStore:
  name: primary-object-store
  pools:
  - name: pool-0
    servers: 8
    volumesPerServer: 8
    size: 2Ti
  services:
    minio:
      serviceType: NodePort
      nodePort: 31000

The operator uses the repositories configuration from the operator values file to determine where to pull images. If you need to override specific image versions in the object store deployment, you can add image overrides:

objectStore:
  image:
    repository: aistor  # References repositories.aistor
    image: minio:RELEASE.2025-XX-XXT00-00-00Z
  sidecar:
    image:
      repository: aistor  # References repositories.aistor
      image: minio-sidecar:RELEASE.2025-XX-XXT00-00-00Z

Validate the object store upgrade.

Confirm the object store pods are running with the new version:
```
kubectl get all -n OBJECT-STORE-NAMESPACE
```
Check the pods are ready and the deployment is healthy.

Upgrade only the object store image

If you want to update only the AIStor Object Store binary version without upgrading the chart itself, you can specify a new image tag. This approach keeps your existing chart version but updates the running software.

Prepare the image (External System).

On a system with internet access, copy the new image version:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/minio:RELEASE.2025-05-30T11-03-39Z \
  docker://PRIVATE_REGISTRY/aistor/minio:RELEASE.2025-05-30T11-03-39Z

Or save as a tarball for transfer:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/minio:RELEASE.2025-05-30T11-03-39Z \
  docker-archive:aistor-minio-RELEASE.2025-05-30T11-03-39Z.tar

Replace the tag with your target release version.

Transfer the image to your airgapped environment and load it into your private registry if using tarballs.

Update the values file (Airgapped System).

Edit your aistor-objectstore-values.yaml file and update the image tag:

objectStore:
  image:
    repository: PRIVATE_REGISTRY/aistor/minio
    tag: RELEASE.2025-05-30T11-03-39Z

Deploy the image update.

Apply the updated configuration using the same method you used for the initial installation:

OCI Registry HTTP Repository Local Chart File

Validate the deployment.

Verify the pods are running with the new image:
```
kubectl get pods -n OBJECT-STORE-NAMESPACE -o jsonpath='{.items[*].spec.containers[?(@.name=="minio")].image}'
```
This command displays the image versions running in your pods. Confirm they reference your private registry with the new tag.

Upgrade AIStor on Kubernetes in Airgapped Environments

Overview

Prerequisites

Step 1. Prepare operator artifacts (external system)

Copy the operator image to your private registry.

Download the operator Helm chart.

Transfer to airgapped environment.

Step 2: Deploy operator chart (airgapped system)

Verify the existing installation.

Upgrade the operator.

Validate the operator upgrade.

Validate the installation by running the following command:

Step 3: Prepare object store artifacts (external system)

Copy the core object store images.

Copy optional images.

Download the object store Helm chart.

Transfer to airgapped environment.

Step 4: Deploy object store chart (airgapped system)

Verify the existing installation.

Upgrade the object store chart.

Validate the object store upgrade.

Upgrade only the object store image

Prepare the image (External System).

Update the values file (Airgapped System).

Deploy the image update.

Validate the deployment.