File Transfer Protocol

You can use the File Transfer Protocol (FTP) to interact with the objects on an AIStor Server.

You must specifically enable FTP or SFTP when starting the server. Enabling either server type does not affect other AIStor Server features.

This page uses the abbreviation FTP throughout, but you can use any of the supported FTP protocols described below.

Supported protocols

When enabled, MinIO AIStor supports FTP access over the following protocols:

• SSH File Transfer Protocol (SFTP)

  SFTP is defined by the Internet Engineering Task Force (IETF) as an extension of SSH 2.0. SFTP allows file transfer over SSH for use with Transport Layer Security (TLS) and virtual private network (VPN) applications.

  Your FTP client must support SFTP.

• File Transfer Protocol over SSL/TLS (FTPS)

  FTPS allows for encrypted FTP communication with TLS certificates over the standard FTP communication channel. FTPS should not be confused with SFTP, as FTPS does not communicate over a Secure Shell (SSH).

  Your FTP client must support FTPS.

• File Transfer Protocol (FTP)

  Unencrypted file transfer.

  MinIO does not recommend using unencrypted FTP for file transfer.

Supported commands

When enabled, MinIO AIStor supports the following SFTP operations:

• get
• put
• ls
• mkdir
• rmdir
• delete

MinIO does not support either append or rename operations.

Considerations

Versioning

SFTP clients can only operate on the latest version of an object. Specifically:

• For read operations, MinIO AIStor only returns the latest version of the requested object(s) to the SFTP client.
• For write operations, MinIO AIStor applies normal versioning behavior and creates a new object version at the specified namespace. rm and rmdir operations create DeleteMarker objects.

Authentication and access

SFTP access requires the same authentication as any other S3 client. MinIO AIStor supports the following authentication providers:

• MinIO AIStor Plugin users and their service accounts
• Active Directory/LDAP users and their service accounts
• OpenID/OIDC service accounts

STS credentials cannot access buckets or objects over SFTP.

Authenticated users can access buckets and objects based on the policies assigned to the user or parent user account.

The SFTP protocol does not require any of the admin:* permissions. You may not perform other MinIO AIStor admin actions with SFTP.

Prerequisites

• Enable an FTP or SFTP port for the server.
• A port to use for the FTP commands and a range of ports to allow the FTP server to request to use for the data transfer.

Procedure

1. Start MinIO AIStor with an FTP and/or SFTP port enabled.

   FTPS

   The following example starts MinIO AIStor with FTPS enabled.

   ./minio http://server{1...4}/disk{1...4}      \
   --ftp="address=:8021"                         \
   --ftp="passive-port-range=30000-40000"        \
   --ftp="tls-private-key=path/to/private.key"   \
   --ftp="tls-public-cert=path/to/public.crt"    \
   ...

   Omit tls-private-key and tls-public-cert to use the MinIO AIStor default TLS keys for FTPS. For more information, see the TLS on MinIO AIStor documentation.

   SFTP/FTP

   ./minio http://server{1...4}/disk{1...4}             \
   --ftp="address=:8021"                                \
   --ftp="passive-port-range=30000-40000"               \
   --sftp="address=:8022"                               \
   --sftp="ssh-private-key=/home/miniouser/.ssh/id_rsa" \
   ...

   See the minio server --ftp and minio server --sftp for details on using these flags to start the MinIO service. To connect to the an FTP port with TLS (FTPS), pass the tls-private-key and tls-public-cert keys and values, as well, unless using the MinIO default TLS keys.

   The output of the command should return a response that resembles the following:

   MinIO FTP Server listening on :8021
   MinIO SFTP Server listening on :8022

2. Use your preferred FTP client to connect to the AIStor Server. You must connect as a user whose policies allow access to the desired buckets and objects.

   The specifics of connecting to MinIO AIStor depend on your FTP client. Refer to the documentation for your client.

   To connect over TLS or through SSH, you must use a client that supports the desired protocol.

3. Connect to MinIO AIStor.

   SFTP/FTP

   The following example connects to an SFTP server, and lists the contents of a bucket named runner.

   > sftp -P 8022 aistor@localhost
   minio@localhost's password:
   Connected to localhost.
   sftp> ls runner/
   chunkdocs  testdir

   FTPS

   The following uses the Linux FTP CLI client to connect to the AIStor Server using aistor credentials to list contents in a bucket named runner

   > ftp localhost -P 8021
   Connected to localhost.
   220 Welcome to MinIO FTP Server
   Name (localhost:user): aistor
   331 User name ok, password required
   Password:
   230 Password ok, continue
   Remote system type is UNIX.
   Using binary mode to transfer files.
   ftp> ls runner/
   229 Entering Extended Passive Mode (|||39155|)
   150 Opening ASCII mode data connection for file list
   drwxrwxrwx 1 nobody nobody            0 Jan  1 00:00 chunkdocs/
   drwxrwxrwx 1 nobody nobody            0 Jan  1 00:00 testdir/
   ...

4. Download an object.

   SFTP/FTP

   This example lists items in a bucket, then downloads the contents of the bucket.

   > sftp -P 8022 aistor@localhost
   aistor@localhost's password:
   Connected to localhost.
   sftp> ls runner/
   chunkdocs  testdir
   sftp> get runner/chunkdocs/metadata metadata
   Fetching /runner/chunkdocs/metadata to metadata
   metadata                               100%  226    16.6KB/s   00:00
   sftp>

   FTPS

   This example lists items in a bucket, then downloads the contents of the bucket.

   > ftp localhost -P 8021
   Connected to localhost.
   220 Welcome to MinIO FTP Server
   Name (localhost:user): aistor
   331 User name ok, password required
   Password:
   230 Password ok, continue
   Remote system type is UNIX.
   Using binary mode to transfer files.ftp> ls runner/chunkdocs/metadata
   229 Entering Extended Passive Mode (|||44269|)
   150 Opening ASCII mode data connection for file list
   -rwxrwxrwx 1 nobody nobody           45 Apr  1 06:13 chunkdocs/metadata
   226 Closing data connection, sent 75 bytes
   ftp> get
   (remote-file) runner/chunkdocs/metadata
   (local-file) test
   local: test remote: runner/chunkdocs/metadata
   229 Entering Extended Passive Mode (|||37785|)
   150 Data transfer starting 45 bytes
      45        3.58 KiB/s
   226 Closing data connection, sent 45 bytes
   45 bytes received in 00:00 (3.55 KiB/s)
   ...

Enable SFTP on Kubernetes

On Kubernetes, you do not start the server yourself. Set objectStore.sftp in the Object Store Helm values and the AIStor Operator adds the SFTP flags to the server container for you.

The Operator exposes SFTP only. It does not start the FTP or FTPS listeners described above.

Enable the SFTP service

objectStore:
  sftp:
    enabled: true

Apply the configuration when deploying or upgrading your Object Store:

helm upgrade --install -n OBJECT-STORE-NAMESPACE OBJECT-STORE-NAME minio/aistor-objectstore -f aistor-objectstore-values.yaml

Replace OBJECT-STORE-NAMESPACE with your Object Store namespace and OBJECT-STORE-NAME with the Helm release name.

The Operator then:

• Starts the server with --sftp="address=:8022" and --sftp="ssh-private-key=...".
• Opens container port 8022 on each Object Store Pod.
• Creates a Service named minio-service-sftp in the Object Store namespace, listening on port 8022.

Setting enabled back to false removes the Service and restarts the Pods without the SFTP listener.

SSH host key

The Operator does not take a separate SSH host key. The server presents the Object Store’s TLS private key as its SSH host key.

By default, that is the first certificate in objectStore.certificates.server. If you list no certificates and leave objectStore.certificates.disableAutoCert at false, it is the certificate the Operator generates.

Because the host key is the TLS key, rotating the Object Store’s TLS certificate also changes the SSH host key. Clients that recorded the old key report a host key mismatch on their next connection.

To choose a different certificate, list it in objectStore.certificates.server after the first entry, then set domainName to a domain that certificate covers:

objectStore:
  certificates:
    server:
      - name: objectstore-default-tls
      - name: objectstore-sftp-tls
  sftp:
    enabled: true
    domainName: sftp.example.net

The server then uses the private key from objectstore-sftp-tls as its SSH host key.

domainName only selects the host key. It does not create DNS records, and it does not change how the Service is exposed.

Expose SFTP outside the cluster

The minio-service-sftp Service is a ClusterIP by default, which is reachable only from inside the cluster. Use the remaining objectStore.sftp fields to expose it:

For example, to reach SFTP on a fixed port on every node:

objectStore:
  sftp:
    enabled: true
    serviceType: NodePort
    nodePort: 31002

Connect

To test without exposing the Service, forward the port locally:

kubectl port-forward -n OBJECT-STORE-NAMESPACE svc/minio-service-sftp 8022:8022

Then connect as any user the Object Store can authenticate:

sftp -P 8022 MINIO-USER@localhost

The supported commands, versioning behavior, and authentication rules described on this page apply on Kubernetes without change.

Authenticate with SSH certificates

SSH certificate authentication on Kubernetes requires the aistor-operator Helm chart 6.1.0 or later.

Users can sign in with an SSH certificate instead of a password. A certificate authority (CA) signs each user’s certificate, and the server accepts any certificate signed by the CA you trust. See Connect to MinIO AIStor using SFTP with a certificate key file for how to create the CA and sign user certificates.

1. Store the CA public key in a ConfigMap in the Object Store namespace:

   kubectl create configmap sftp-ca -n OBJECT-STORE-NAMESPACE --from-file=ca.pub=/path/to/ca_user_key.pub

2. Reference the ConfigMap key in your Helm values:

   objectStore:
     sftp:
       enabled: true
       trustedUserCAKey:
         name: sftp-ca
         key: ca.pub

3. Apply the configuration with helm upgrade. The Operator restarts the Pods to turn on certificate authentication.

Create the ConfigMap before you apply the values. If the ConfigMap or key is missing, or the key is not a valid OpenSSH public key, the Operator stops updating the Object Store and the running servers keep their current settings. Check the Operator logs for the error, fix the ConfigMap, and the Operator continues.

To rotate the CA, update the public key in the ConfigMap. The Operator detects the change and restarts the AIStor Server, which then accepts only certificates signed by the new CA. The Pods themselves are not replaced.

Each certificate must list the user name in its principals, and that user must already exist in the Object Store. Password authentication keeps working for users who do not have a certificate.

Pass additional SFTP settings

Use extraParameters to pass other SFTP settings to the server. The server accepts pub-key-algos, kex-algos, cipher-algos, mac-algos, and disable-password-auth, and ignores other names. For example, to restrict the ciphers and MAC algorithms:

objectStore:
  sftp:
    enabled: true
    extraParameters:
      cipher-algos: aes128-gcm@openssh.com,aes256-gcm@openssh.com
      mac-algos: hmac-sha2-256-etm@openssh.com

The Operator manages address, ssh-private-key, and trusted-user-ca-key itself. The Object Store rejects these names in extraParameters.

The server stops with an error if a value is not valid, for example an unknown algorithm name. Check the Pod logs if the Pods do not become ready after you change extraParameters.

Connect to MinIO AIStor using SFTP with a certificate key file

MinIO AIStor supports OpenSSH certificate authentication on SFTP. A trusted user certificate authority (CA) signs each user’s public key to create an OpenSSH user certificate. These are OpenSSH certificates, not the X.509 certificates that TLS uses.

This type of authentication requires the following:

1. An SSH key pair for the trusted user CA. The AIStor Server uses the CA public key to verify user certificates. Keep the CA private key secret and use it only to sign user certificates.
2. An SSH key pair for each user who connects by SFTP.
3. An OpenSSH user certificate for each user, signed with the CA private key and stored next to the user’s private key, such as in the user’s .ssh folder (or equivalent for the operating system).

Each certificate must include a principals list of the user(s) that can authenticate with the certificate:

ssh-keygen -s ~/.ssh/ca_user_key -I miniouser -n miniouser -V +1h -z 1 miniouser1.pub

This command signs the user public key miniouser1.pub and writes the certificate to miniouser1-cert.pub.

• -s specifies the path to the CA private key that signs the certificate. Do not pass the CA public key (.pub file).
• -I specifies the key identity for the certificate.
• -n sets the principals list for the certificate. You must include the user for which this certificate is valid, and the user must match the username in MinIO.
• -V limits the duration for which the certificate is valid. In this example, the certificate is valid for one hour. Adjust the duration for your requirements.
• -z adds a serial number to the certificate to distinguish it from other certificates signed by the same CA.

MinIO AIStor requires specifying the Certificate Authority used to sign the certificates for SFTP access. Start or restart the AIStor Server and specify the path to the trusted certificate authority’s public key using an --sftp="trusted-user-ca-key=PATH" flag:

minio server {path-to-server} --sftp="trusted-user-ca-key=/path/to/.ssh/ca_user_key.pub" {...other flags}

When the client connects with SFTP, it passes its user certificate to the AIStor Server. MinIO AIStor checks that the CA public key provided at server startup signed the certificate, and that the certificate principals include the username.

Once MinIO AIStor verifies the client’s certificate, the user can connect to MinIO AIStor over SFTP:

sftp -P <SFTP port> <server IP>

Procedure

The following procedure generates three key pairs: one for the user CA, one for the AIStor Server host, and one for the user. It uses the CA private key to sign the user public key, then uses the resulting certificate to log in to the SFTP server.

1. Generate a key pair for the user CA.

   ssh-keygen -f ./ca_user_key

   Store the CA private key ca_user_key in a secure location. The AIStor Server needs only the CA public key ca_user_key.pub.

2. Generate a host key pair for the AIStor Server.

   ssh-keygen -f ./sftp_host_key

   The AIStor Server uses this key to identify itself to SFTP clients. Do not use the CA private key as the host key.

3. Generate a key pair for the user.

   ssh-keygen -f ./minioadmin

   Replace minioadmin with the user accessing the AIStor Server by SFTP.

4. Sign the user public key with the CA private key.

   ssh-keygen -s ca_user_key -I minioadmin -n minioadmin -V +30d -z 1 minioadmin.pub

   This creates the certificate minioadmin-cert.pub. Keep it in the same directory as the minioadmin private key, such as ~/.ssh/meaningful-directory.

5. Start or restart the AIStor Server with the host private key and the CA public key.

   minio server --sftp="address=:8022" --sftp="ssh-private-key=/path/to/sftp_host_key" --sftp="trusted-user-ca-key=/path/to/ca_user_key.pub"

6. Connect to the AIStor Server by SFTP.

   sftp -i ./minioadmin -oPort=8022 minioadmin@localhost

Require service account or LDAP for authentication

To force authentication to SFTP using LDAP or service account credentials, append a suffix to the username. Valid suffixes are either =ldap or =svc.

sftp -P 8022 my-ldap-user=svc@example.com:/bucket

sftp -P 8022 my-ldap-user=ldap@example.com:/bucket

• Replace my-ldap-user with the LDAP username for =ldap, or with the service account access key for =svc.
• Replace example.com with the domain name of the MinIO AIStor server.