mc support telemetry analyze

mc support telemetry analyze provides an interactive terminal interface for analyzing OpenTelemetry trace data. Use this command to perform quick latency analysis on recorded telemetry files.

The command reads trace files created by mc support telemetry record and displays them in an interactive tree view with duration metrics, percentage breakdowns, and color-coded latency indicators.

Syntax

Parameters

INPUT

Required

The path to the telemetry file to analyze. You can analyze any of the following telemetry files:

  • Local files (.otlp, .bin, .enc, .zst)
  • Encrypted files (requires --private-key)
  • Remote S3/AIStor objects (for example, myaistor/testbucket/mytrace.otlp)

--private-key

Optional

Path to the private key for decrypting encrypted telemetry files.

Default: support_private.pem

The command automatically searches for numbered key variants, such as file.pem, file-2.pem, file-3.pem.

--min-duration

Optional

Filter out spans shorter than the specified duration.

Accepts duration strings such as 100ms, 1s, or 500us.

Use this to focus on slow operations and reduce noise from fast spans.

--service

Optional

Filter spans by service name.

Only spans from the specified service appear in the analysis.

AIStor telemetry uses the service name AIStor. This filter is useful when analyzing telemetry files that contain traces from multiple services or when filtering traces collected from an OpenTelemetry collector that aggregates data from multiple sources.

--filter

Optional

Filter spans by name using substring matching.

For example, --filter=PutObject shows only spans with “PutObject” in their name.

--type

Optional

Filter by trace type. You can specify this flag multiple times to include multiple types.

Valid types include:

Type Description
admin Admin API calls
batchexpire Batch expiration operation calls
batchkeyrotation Batch key rotation operation calls
batchreplication Batch replication operation calls
bootstrap Cluster bootstrap events
decommission Decommission operation calls
formatting Formatting events
ftp FTP server calls
healing Healing operation calls
iam Identity and access management calls
ilm Lifecycle management calls
internal Internal operations and grid RPC calls
kms KMS interaction calls
object Object layer calls
os Operating system calls
rebalance Rebalance operation calls
replication Replication events
replicationresync Replication resync operation calls
s3 S3 API calls
scanner Scanner operation calls
storage Storage layer calls

The AIStor Client ignores capitalization when specifying trace types.

--top

Optional

Show only the top N slowest traces.

Set to 0 to show all traces.

Default: 0 (all traces)

--export

Optional

Export the analysis to a JSON file at the specified path.

The exported file contains the full trace tree structure with durations and percentages.

Global flags

This command supports any of the global flags.

Required permissions

This command performs client-side analysis and does not require any AIStor admin permissions.

For local files, no permissions are required.

For remote S3/AIStor objects, the command requires the s3:GetObject permission on the bucket and object path. The authenticated user must have read access to the telemetry file stored in the bucket.

Interactive interface

The command launches a terminal user interface (TUI) with two view modes.

Tree view

The default view displays traces as an expandable tree hierarchy. Each span shows:

  • Name: The operation or span name
  • Duration: Time taken, color-coded by latency:
    • Green: Less than 50ms
    • Yellow: 50ms to 200ms
    • Red: More than 200ms
  • Percentage bar: Visual representation of time relative to the root span
  • Percentage: Numeric percentage of the root span duration

Grouped view

Press Tab or g to switch to the grouped view. This view aggregates root spans by operation name and displays statistics:

  • Count: Number of occurrences
  • Avg: Average duration
  • Min: Minimum duration
  • p50/p95/p99: Percentile durations
  • Max: Maximum duration
  • Err: Error count
  • Err%: Error percentage

Select an operation and press Enter or → to view individual instances.

Keyboard controls

Key Action
↑/↓ or j/k Navigate up/down
→ or l Expand node
← or h Collapse node or go to parent
Space Toggle expand/collapse
Enter or d View span details
Tab or g Switch between tree and grouped views
s Cycle sort mode (grouped view)
c Collapse all nodes
Home/End Jump to first/last item
PgUp/PgDn Page up/down
q or Esc Quit

Span details

The detail view shows the following information for a span:

  • Span ID and parent span ID
  • Start and end timestamps
  • Duration
  • Span kind, which indicates the role of the span:
    • INTERNAL: An operation within the service
    • SERVER: Handling an incoming request
    • CLIENT: Making an outgoing request to another service or node
    • PRODUCER: Creating a message for asynchronous processing
    • CONSUMER: Receiving a message from asynchronous processing
  • Status code and message
  • Service name
  • All span and resource attributes
  • Events with timestamps

Examples

Analyze a local telemetry file

The following command opens the interactive analyzer for a local file:

mc support telemetry analyze ./metrics-2025-01-15.bin

Analyze an encrypted file

The following command analyzes an encrypted telemetry file using a private key:

mc support telemetry analyze --private-key=support_private.pem ./metrics.enc

Filter to S3 operations slower than 100ms

The following command shows only S3 API calls that took longer than 100ms:

mc support telemetry analyze --type=s3 --min-duration=100ms ./mytrace.otlp

Export analysis to JSON

The following command exports the analysis to a JSON file for scripting or further processing:

mc support telemetry analyze --export=analysis.json ./mytrace.otlp

Use --json for JSON output to stdout instead of the interactive interface:

mc support telemetry analyze --json ./mytrace.otlp