Configuring Tests

Configure test parameters to match your storage environment and evaluation needs. These basic settings control how Warp operates.

Test data

Warp generates its own set of objects to use for testing. Use the --bucket flag to choose the bucket that holds them. The default bucket is warp-benchmark-bucket.

Warp deletes every object in the benchmark bucket

Before each run, Warp deletes every object in the benchmark bucket, not only the objects it is about to create. Setting a custom object prefix with --prefix does not narrow that deletion. After the run, Warp deletes the objects it created. Point --bucket at a bucket that holds no data you need to keep.

Pass --noclear to skip both deletions. Use it when you run Warp manually on several machines against one bucket, so the runs do not delete each other’s data at startup.

Test duration

Control how long tests run using the --duration flag. Specify durations in seconds (s), minutes (m), or hours (h). See Duration notation in the Reference for more examples and the accepted suffixes.

# Run for 5 minutes
warp mixed --duration 5m --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY

# Run for 1 hour
warp mixed --duration 1h --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY
  • Short tests (1-5 minutes) work for initial evaluation.
  • Longer tests (10-30 minutes) provide more reliable performance data.
  • Extended tests (1+ hours) help identify performance degradation under sustained load.

Tests shorter than 5 minutes may not provide reliable results.

Auto termination

The --autoterm flag automatically stops tests when performance stabilizes, saving time while ensuring valid results. By default, Warp considers the test to be stable when the last 7 of 25 samples stays within a 7.5% range. For exact byte mappings and suffix rules, see Object size notation.

For details on stability or how to adjust the factors that determine stability, see the GitHub readme. The Benchmark control flags table lists --autoterm, --autoterm.dur, and --autoterm.pct.

Object sizes

Set object sizes to match your typical data.

  • Small objects (1KB-100KB) test metadata performance.
  • Medium objects (1MB-10MB) test balanced workloads.
  • Large objects (100MB+) test streaming performance.
# Test with 10MB objects
warp put --obj.size 10MB --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY

# Test with mixed sizes
warp put --obj.size 1MB,10MB,100MB --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY

Use object sizes that represent your actual application data. For example, photo storage systems might use 5-20MB objects and video storage systems 100MB-10GB objects.

Concurrency levels

The --concurrent flag controls how many operations run simultaneously. Start with the default of 20 concurrent operations. Increase or decrease concurrency to test a range of load scenarios.

# Default concurrency
warp mixed --concurrent 80 --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY # Test at 60 and increase in increments of 10.

# Higher load
warp mixed --concurrent 100 --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY

# Lower load
warp mixed --concurrent 5 --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY

In a distributed run, the --concurrent value applies to each Warp client rather than to the run as a whole. Total concurrency is the --concurrent value multiplied by the number of clients, so three clients at --concurrent 100 run 300 concurrent operations.

Higher concurrency reveals system limits and maximum throughput. Adjust the value based on your expected application load. Too much concurrency can overwhelm the storage system and produce unrealistic results.

Number of objects

Control the object pool size with --objects. Larger object pools better simulate production scenarios with thousands or millions of objects. Smaller pools work for quick tests, but may not reflect real performance.

# Test with 10,000 objects
warp get --objects 10000 --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY

# Test with larger pool
warp get --objects 100000 --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY

Warp creates objects during test setup, then performs operations on this pool. Larger pools take longer to initialize but provide more realistic results.

Quick start examples 

# Basic upload test
warp put --obj.size 5MB --concurrent 20 --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY

# Basic download test
warp get --concurrent 50 --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY

# Realistic mixed workload
warp mixed --duration 15m --obj.size 1MB,10MB --concurrent 30 --access-key YOUR_ACCESS_KEY --secret-key YOUR_SECRET_KEY