Deploy AIStor with a Private Container Registry

This procedure deploys AIStor Object Store on Kubernetes clusters using a private container registry for images while installing Helm charts from the public MinIO Helm repository.

Use this procedure when your environment has internet access to install Helm charts from helm.min.io but you need to pull container images from a private container registry instead of the public quay.io registry.

This procedure requires installation of Kubernetes operators and associated resources including CustomResourceDefinitions, StatefulSets, and secrets into new or existing namespaces. You must perform the operations in this procedure as a user that has broad permissions to create resources within multiple namespaces.

When to use this procedure

Use this deployment method when:

Your Kubernetes cluster has internet access to reach helm.min.io for Helm chart installation.
Your organization requires all container images to be pulled from a private container registry instead of public registries, such as for security, compliance, or governance reasons.

Comparison with airgap deployment:

If your Kubernetes cluster is in an airgapped environment with no internet access, use the Deploy AIStor in an Airgapped Environment procedure instead. That procedure covers copying both Helm charts and container images to private registries for deployment in environments without internet connectivity.

Prerequisites

Before beginning this procedure, ensure you have the following:

Kubernetes cluster running an in-support release
Helm version 3.17 or later
Private container registry accessible from the Kubernetes cluster
Network access to copy images from quay.io to the private container registry
Skopeo installed for copying container images
Valid AIStor license from SUBNET
kubectl configured to access the target Kubernetes cluster
Network access to helm.min.io from your deployment environment

Deploy AIStor using Helm with private container registry

The following procedure copies required container images to a private container registry and deploys AIStor using Helm charts from the public MinIO repository.

Copy container images to private registry

Copy mandatory images and any optional images you need from quay.io to your private container registry. Perform these steps from a system that has network access to both quay.io and your private registry.

In each step, replace PRIVATE_REGISTRY with the hostname and path to your private container registry. If you use a different software or method for copying images to your private repository, defer to the documentation or procedure for that tooling.

The following commands use --override-os linux --override-arch amd64 flags to ensure images are copied for Linux AMD64 architecture. If your Kubernetes cluster uses a different architecture (such as ARM64), replace amd64 with the appropriate architecture for your cluster.

If your private registry does not have TLS configured, add the --dest-tls-verify=false flag to each skopeo command. For production environments, configure your registry with proper TLS certificates instead of disabling verification.

Copy the operator image:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/operator:RELEASE.2025-08-19T17-53-00Z \
  docker://PRIVATE_REGISTRY/aistor/operator:RELEASE.2025-08-19T17-53-00Z

Copy the minio image:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/minio:RELEASE.2025-08-21T02-50-21Z \
  docker://PRIVATE_REGISTRY/aistor/minio:RELEASE.2025-08-21T02-50-21Z

Copy the minio-sidecar image:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/minio-sidecar:RELEASE.2025-06-30T21-59-04Z \
  docker://PRIVATE_REGISTRY/aistor/minio-sidecar:RELEASE.2025-06-30T21-59-04Z

Copy optional images based on the features you plan to use.

Copy the kes and kes-sidecar images if you plan to use server-side encryption with Key Encryption Service:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/kes:RELEASE.2025-07-15T11-40-19Z \
  docker://PRIVATE_REGISTRY/aistor/kes:RELEASE.2025-07-15T11-40-19Z

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/kes-sidecar:RELEASE.2025-07-01T00-09-56Z \
  docker://PRIVATE_REGISTRY/aistor/kes-sidecar:RELEASE.2025-07-01T00-09-56Z

Copy the mc image if you plan to automate jobs using the AIStor Client:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/mc:RELEASE.2025-08-21T03-14-05Z \
  docker://PRIVATE_REGISTRY/aistor/mc:RELEASE.2025-08-21T03-14-05Z

Copy the aihub image if you plan to store AI models and datasets:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/aihub:RELEASE.2024-12-13T23-00-47Z \
  docker://PRIVATE_REGISTRY/aistor/aihub:RELEASE.2024-12-13T23-00-47Z

Copy the prompt image if you plan to query your objects with generative AI:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/prompt:RELEASE.2025-01-17T21-55-43Z \
  docker://PRIVATE_REGISTRY/aistor/prompt:RELEASE.2025-01-17T21-55-43Z

Copy the warp image if you plan to run benchmark tests on your cluster:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/warp:v1.3.0 \
  docker://PRIVATE_REGISTRY/minio/warp:v1.3.0

Verify that the images are accessible in your private registry.

You can use skopeo inspect to verify each image was copied successfully:
```
skopeo inspect docker://PRIVATE_REGISTRY/aistor/operator:RELEASE.2025-08-19T17-53-00Z
```
If your registry has a REST API endpoint, you can also verify using curl:
```
curl -k https://PRIVATE_REGISTRY/v2/_catalog
curl -k https://PRIVATE_REGISTRY/v2/aistor/operator/tags/list
```
Replace https:// with http:// if your registry does not use TLS.

Create image pull secret

If your private container registry requires authentication, create a Kubernetes secret for pulling images.

Create the image pull secret in the aistor namespace:

kubectl create secret docker-registry private-repo-pull-secret \
  --docker-server=PRIVATE_REGISTRY \
  --docker-username=USERNAME \
  --docker-password=PASSWORD \
  --docker-email=EMAIL \
  -n aistor

Replace the following values:

PRIVATE_REGISTRY: The hostname of your private container registry
USERNAME: Username for authenticating to the private container registry
PASSWORD: Password for authenticating to the private container registry
EMAIL: Email address associated with the container registry account

Add the AIStor Helm repository

Add the public MinIO Helm repository to install the AIStor Helm charts.

helm repo add minio https://helm.min.io/
helm repo update

Configure Helm values for private container registry

Create a Helm values file that configures AIStor to use your private container registry for pulling images.

Retrieve your license file.

Log in to SUBNET and select the License button in the Deployments view. Save the content of the file in a secure location for use in the next step. You need the decoded JWT token value from the license file (beginning with eyJ...) for installation.
Create a YAML manifest to customize the minio/aistor-objectstore-operator chart.

Use your preferred text editor to create a YAML manifest for the chart named aistor-objectstore-operator-values.yaml.

The following YAML example configures the private container registry settings:
```
repositories:
  aistor:
    hostname: PRIVATE_REGISTRY
    pathPrefix: aistor/
    imagePullSecrets:
    - name: private-repo-pull-secret
  minio:
    hostname: PRIVATE_REGISTRY
    pathPrefix: minio/
    imagePullSecrets:
    - name: private-repo-pull-secret
```
Replace PRIVATE_REGISTRY with the hostname of your private container registry (for example, registry.example.com or registry.example.com:5000).

The repositories section configures the Helm chart to pull all AIStor images from your private container registry instead of quay.io. The pathPrefix value specifies the path within your container registry where you copied the images.
Why two repository configurations?
The configuration includes two repository entries:
- aistor: Used for AIStor-specific images (operator, minio, minio-sidecar, kes, kes-sidecar, mc, aihub, prompt)
- minio: Used for MinIO community images (warp)
This separation allows you to organize images in your private registry using different path structures if needed. In most cases, you can use the same hostname and similar path prefixes for both repositories.
If you need to override specific image versions, add an images section to the aistor-objectstore-operator-values.yaml file:
```
images:
  minio:
    repository: aistor
    image: minio:RELEASE.2025-08-29T21-27-49Z
```
This example overrides the default minio image version with a different release. Refer to the AIStor Operator values reference for all available image customizations.

Do not update values in the object-store-operator configuration map directly, as helm upgrade overwrites them. Instead, always use an custom YAML file such as aistor-objectstore-operator-values.yaml to install and upgrade AIStor.

Install the AIStor Operator

Install the AIStor Operator using the customized values file and your SUBNET license.

Install the chart to the aistor namespace with the customization file and license:
```
helm install aistor minio/aistor-objectstore-operator \
  -n aistor --create-namespace \
  --set license="eyJhbGciOiJFUzM4NCIsInR..." \
  -f aistor-objectstore-operator-values.yaml
```
Replace the license value with your decoded JWT token from SUBNET.

This installs the aistor-objectstore-operator chart from the public MinIO Helm repository.

If successful, the command outputs a summary of installed resources.

Validate the installation by running the following command:

kubectl get all -n aistor

The output should show running pods similar to the following:

NAME                                         READY   STATUS    RESTARTS   AGE
pod/adminjob-operator-cfc97d9f-hjbp5         1/1     Running   0          4m16s
pod/object-store-operator-78c9f84b85-kmwlv   1/1     Running   0          4m16s

NAME                            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/object-store-operator   ClusterIP   10.43.210.230   <none>        4221/TCP   4m16s

NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/adminjob-operator       1/1     1            1           4m16s
deployment.apps/object-store-operator   1/1     1            1           4m16s

NAME                                               DESIRED   CURRENT   READY   AGE
replicaset.apps/adminjob-operator-cfc97d9f         1         1         1       4m16s
replicaset.apps/object-store-operator-78c9f84b85   1         1         1       4m16s

Deploy an AIStor Object Store

After the Operator is running, deploy an AIStor Object Store instance.

Create a YAML manifest to customize the object store chart.

Use your preferred text editor to create a file named aistor-objectstore-values.yaml. The following example has a minimal set of fields for deploying an 8x8 object store using the AIStor Volume Manager for storage provisioning:
```
objectStore:
  name: primary-object-store
  pools:
  - name: pool-0
    servers: 8
    volumesPerServer: 8
    size: 2Ti
  services:
    minio:
      serviceType: NodePort
      nodePort: 31000
```
Modify the values to reflect your deployment requirements. Remove any default or unmodified values such that the file reflects only your changes.
Install the object store chart:
```
helm install primary-object-store minio/aistor-objectstore \
  -n primary-object-store --create-namespace \
  -f aistor-objectstore-values.yaml
```
This installs the aistor-objectstore chart from the public MinIO Helm repository.

The command deploys an AIStor Object Store with the name and namespace of primary-object-store.

Connect to the deployment

After the object store is deployed, connect to it using either the Console web interface or the AIStor Client command-line tool.

Console

Configure ingress to route traffic to the -console service in the object store namespace. You can temporarily access the Console using kubectl port-forward:

kubectl port-forward -n primary-object-store svc/primary-object-store-console 9443:9443

You can use the browser interface for managing the object store installation.

CLI

Download the mc tool to your local host machine.

See AIStor Client Quickstart for installation instructions.
Create a new alias for the AIStor server.

The S3 API services are configured via NodePort so you can access them through the IP or hostname of any worker node in the Kubernetes cluster. Use kubectl get nodes -o wide to determine the appropriate IP or hostname to use for access.
```
mc --insecure alias set myaistor https://NODE_IP_OR_HOSTNAME:31000 minio minio123
```
Replace NODE_IP_OR_HOSTNAME with the externally accessible hostname or IP of any worker node in the deployment.

Create a bucket and copy a file to that bucket:

mc --insecure mb myaistor/data
mc --insecure cp ~/somefile.txt myaistor/data

Troubleshooting

Architecture mismatch errors when copying images

If you encounter errors like no image found in image index for architecture when using Skopeo, ensure you are using the --override-os and --override-arch flags with the correct values for your Kubernetes cluster.

The images must match the architecture of your Kubernetes nodes, not the system where you run the skopeo commands. For example, if copying from a Mac to a Linux AMD64 cluster:

skopeo copy --override-os linux --override-arch amd64 \
  docker://quay.io/minio/aistor/operator:RELEASE.2025-08-19T17-53-00Z \
  docker://PRIVATE_REGISTRY/aistor/operator:RELEASE.2025-08-19T17-53-00Z

TLS/SSL certificate errors

If you see errors related to certificate verification when copying images, you can use one of the following options depending on your environment:

For development/testing: Add --dest-tls-verify=false to disable TLS verification:

skopeo copy --override-os linux --override-arch amd64 \
  --dest-tls-verify=false \
  docker://quay.io/minio/aistor/operator:RELEASE.2025-08-19T17-53-00Z \
  docker://PRIVATE_REGISTRY/aistor/operator:RELEASE.2025-08-19T17-53-00Z

For production environments: Configure your private registry with valid TLS certificates instead of disabling verification.

Connection refused or EOF errors

If Skopeo cannot connect to your private registry:

Verify the registry is running and accessible:
```
curl -k https://PRIVATE_REGISTRY/v2/
```
This should return an empty JSON object {} if the registry is accessible.
Check that the hostname and port are correct in your PRIVATE_REGISTRY value.
Verify network connectivity between the system running Skopeo and the private registry.
If using a firewall, ensure the registry port is open.

Images fail to pull in Kubernetes

If images were copied successfully but fail to pull in your Kubernetes cluster:

Verify the image pull secret was created correctly:

kubectl get secret private-repo-pull-secret -n aistor

Verify the Helm values file contains the correct registry hostname and image pull secret name.
Check pod events for specific error messages:
```
kubectl describe pod POD_NAME -n NAMESPACE
```
Verify network connectivity from the Kubernetes nodes to the private registry.