warp run

Execute benchmarks defined in YAML configuration files for repeatable and version-controlled testing.

warp run [FLAGS] CONFIG_FILE

warp run benchmark-config.yaml

The path to a YAML configuration file that defines the benchmark parameters. The configuration file specifies the benchmark type, S3 connection details, and test parameters. YAML configuration supports all flags and options available in standard Warp benchmark commands.

This parameter is required.

Default: no variables set

Set template variables for dynamic parameter substitution in the YAML configuration. The default behavior uses the configuration file as written without substitution. Use this flag to parameterize configuration files for different test scenarios.

Specify variables in the format Name=Value. You can use this flag multiple times to set multiple variables. Variables are referenced in YAML files using Go template syntax like {{.VariableName}}.

Example:

warp run config.yaml --var Host=s3.example.com --var Duration=5m

Default: false

Enable detailed debug output for troubleshooting. The default behavior shows only benchmark execution output. Use this flag to see template processing and configuration parsing details.

Example:

warp run config.yaml --debug

YAML configuration files define all benchmark parameters in a structured format. This structure makes benchmark configurations more maintainable and shareable than long command-line invocations.

All Warp configurations use a top-level warp element with an api version and benchmark type.

warp:
  api: v1
  benchmark: get
  host: s3.example.com:9000
  access-key: ACCESS_KEY
  secret-key: SECRET_KEY
  bucket: benchmark-bucket
  duration: 5m
  concurrent: 100
  objects: 10000

The api field specifies the configuration format version. Currently only v1 is supported.

The benchmark field specifies which benchmark operation to run. Valid values include get, put, delete, list, stat, mixed, and all other Warp benchmark types.

Connection parameters define how Warp connects to the S3 endpoint.

warp:
  api: v1
  benchmark: mixed
  host: s3.example.com:9000
  access-key: ACCESS_KEY
  secret-key: SECRET_KEY
  bucket: benchmark-bucket
  tls: true
  region: us-east-1

The host field specifies the S3 endpoint address and port. The access-key and secret-key fields provide S3 credentials. The bucket field names the bucket for benchmark operations. The tls field enables or disables TLS encryption. The region field specifies the S3 region.

Benchmark parameters control test execution.

warp:
  api: v1
  benchmark: get
  host: s3.example.com:9000
  access-key: ACCESS_KEY
  secret-key: SECRET_KEY
  duration: 10m
  concurrent: 200
  objects: 100000
  obj:
    size: 10MB

The duration field sets the benchmark run duration. The concurrent field controls the number of concurrent operations. The objects field specifies how many objects to use in the benchmark.

Object configuration uses a nested obj section. The obj.size field defines object size.

For mixed workload benchmarks, distribution parameters control operation ratios.

warp:
  api: v1
  benchmark: mixed
  host: s3.example.com:9000
  access-key: ACCESS_KEY
  secret-key: SECRET_KEY
  distribution:
    get: 45
    put: 15
    stat: 30
    delete: 10

Distribution values specify the percentage of each operation type. The values should sum to 100 for correct operation distribution.

Analysis parameters control result processing and output.

warp:
  api: v1
  benchmark: get
  host: s3.example.com:9000
  access-key: ACCESS_KEY
  secret-key: SECRET_KEY
  analyze:
    dur: 30s
    verbose: true
    skip-duration: 1m
  benchdata: results.csv.zst

The analyze.dur field sets time segment duration for analysis. The analyze.verbose field enables detailed output. The analyze.skip-duration field excludes warmup time from analysis. The benchdata field specifies the output filename for benchmark data.

Use Go template syntax to parameterize configuration files.

warp:
  api: v1
  benchmark: {{.Benchmark}}
  host: {{.Host}}
  access-key: {{.AccessKey}}
  secret-key: {{.SecretKey}}
  duration: {{.Duration}}
  concurrent: {{.Concurrent}}
  obj:
    size: {{.ObjSize}}

Set template variables using the --var flag when running the benchmark.

warp run config.yaml \
  --var Benchmark=get \
  --var Host=s3.example.com:9000 \
  --var AccessKey=ACCESS_KEY \
  --var SecretKey=SECRET_KEY \
  --var Duration=5m \
  --var Concurrent=100 \
  --var ObjSize=10MB

This approach enables a single configuration file to support multiple test scenarios.

Advanced sections organize related parameters.

The io section groups input/output parameters. The remote section contains distributed benchmark settings. The params section holds test-specific parameters. The advanced section includes advanced tuning options.

These sections do not add prefixes to flag names. They organize the configuration file for readability.

warp:
  api: v1
  benchmark: get
  host: s3.example.com:9000
  io:
    part-size: 5MB
  remote:
    host: storage-server.example.com
  params:
    custom-setting: value

The Warp repository includes sample YAML configurations in the yml-samples/ directory. These samples demonstrate various benchmark scenarios and configuration patterns.

Sample configurations cover mixed workload benchmarks, specific operation benchmarks, distributed benchmark configurations, and advanced feature testing.

The run command executes the specified benchmark and displays results.

The output includes real-time progress during benchmark execution. Operation counts and throughput appear as the benchmark runs. After completion, the command displays comprehensive statistics.

If the configuration specifies a benchdata output file, the command saves detailed benchmark data. This data file is compatible with warp analyze and other analysis commands.

Execute a benchmark from a YAML file:

warp run get-benchmark.yaml

The configuration file defines all benchmark parameters.

Execute a benchmark with template variable substitution:

warp run config.yaml \
  --var Host=s3.example.com:9000 \
  --var Duration=10m \
  --var Concurrent=200

The variables replace template placeholders in the configuration file.

Test performance with different object sizes using a single configuration:

# Small objects
warp run config.yaml --var ObjSize=1KB --var Concurrent=500

# Medium objects
warp run config.yaml --var ObjSize=1MB --var Concurrent=100

# Large objects
warp run config.yaml --var ObjSize=100MB --var Concurrent=10

This approach enables systematic testing across object size ranges.

Execute with debug logging to troubleshoot configuration issues:

warp run config.yaml --debug

The debug output shows template processing and parameter mapping.

Execute a mixed workload benchmark from configuration:

warp run mixed-workload.yaml

Example configuration:

warp:
  api: v1
  benchmark: mixed
  host: s3.example.com:9000
  access-key: ACCESS_KEY
  secret-key: SECRET_KEY
  bucket: benchmark-bucket
  duration: 10m
  concurrent: 100
  objects: 50000
  distribution:
    get: 45
    put: 15
    stat: 30
    delete: 10
  obj:
    size: 10MB

Execute a distributed benchmark using YAML configuration:

warp run distributed-config.yaml

Example configuration:

warp:
  api: v1
  benchmark: get
  host: s3.example.com:9000
  access-key: ACCESS_KEY
  secret-key: SECRET_KEY
  bucket: benchmark-bucket
  duration: 5m
  concurrent: 100
  warp-client:
    - client1.example.com:7761
    - client2.example.com:7761
    - client3.example.com:7761

The configuration specifies multiple client endpoints for distributed load generation.

Execute with specific analysis parameters:

warp run config.yaml

Example configuration:

warp:
  api: v1
  benchmark: get
  host: s3.example.com:9000
  access-key: ACCESS_KEY
  secret-key: SECRET_KEY
  duration: 10m
  analyze:
    dur: 30s
    verbose: true
    skip-duration: 1m
  benchdata: detailed-results.csv.zst

This configuration segments analysis into 30-second intervals, skips the first minute, and saves detailed data.