aimx

aimx

English | 简体中文

aimx is a safe, additive, CLI-first companion for native Aim.

It adds focused terminal workflows for querying, comparing, previewing, and exporting Aim run data. Commands that aimx does not own are delegated to the native aim executable already available in the user's environment.

Quick Start

Install

# install into the current project
uv add aimx

# Or use pip
pip install aimx

Install the agent skills

This repository also includes agent skills for Aimx AutoResearch workflows. The existing aimx skill is the Observe subsystem for log_experiment, where an LLM collects run parameters, metric summaries, traces, and image evidence from a local Aim repository. The build skills help create or audit repositories that are ready for that loop.

npx skills install blizhan/aimx

After installation, invoke:

• $aimx for read-only experiment evidence collection.
• $aimx-autoresearch for bounded research rounds with multiple subagents, per-agent model/effort settings, and explicit checkpoint resume.
• $aimx-hydra-lightning-builder for Hydra + Lightning + Aim scaffold and migration-audit workflows.

The skills assume the aimx CLI is available in the environment that performs experiment inspection.

Check your environment

aimx --help
aimx version
aimx doctor

Query an Aim repository

If your current working directory is an Aim repo root, --repo can be omitted. When provided, --repo accepts either the repository root, such as data, or the metadata directory itself, such as data/.aim.

# Summarize metrics from all runs
aimx query metrics --repo data

# Preview images in supported terminals
aimx query images --repo data

# Compare run parameters and durations
aimx query params --repo data

# Plot metric time series from all runs
aimx trace --repo data

Features

• Safe native Aim coexistence: aimx does not replace the aim executable or modify installed Aim packages.
• Explicit command ownership: owned aimx commands stay small and focused; everything else is passed through to native Aim.
• Scriptable query output: query metrics, images, and run params as rich terminal tables, plain text, or JSON.
• Terminal image previews: render Aim image artifacts inline when the terminal supports graphics, with a safe text fallback.
• Metric tracing: plot, tabulate, or export matching time series with step filters and sampling controls.
• Read-only defaults: inspection, query, diagnostic, and passthrough flows do not mutate .aim repository data.
• Durable AutoResearch control plane: keep Findings, lineage, human steering, bounded context, and next-experiment contracts in the Aimx-owned .aimx/research sidecar while Aim remains the evidence source.

AutoResearch

Aimx provides a durable research control plane around Aim experiment evidence. It deliberately separates experiment data, research meaning, and agent execution:

• Aim stores what happened: runs, params, metrics, traces, images, and distributions.
• Aimx Research State stores what was learned and what should happen next: Findings, Lineage, Frontier policy, Agenda items, bounded context, and provenance.
• External agents such as Codex or Claude reason, change project code, and execute experiments. Aimx does not host an agent or scheduler.
• Humans can review, accept/reject, comment on, or redirect durable research state without editing an agent prompt or relying on a previous chat session.

Aim stores what happened. Aimx stores what we learned and what to do next. The agent decides and executes.

Architecture

flowchart TB
    H[Human] -->|steer / review / govern| RS
    A[External Agent\nCodex / Claude / others] <--> RS[Aimx Research Control Plane\nFinding · Lineage · Frontier · Agenda\nResearch Context · Next]
    A -->|change code / run| P[Project / Experiment]
    P -->|logs experiment evidence| AIM[Aim\nruns · params · metrics · traces · images]
    AIM -->|read-only evidence| Q[aimx query / trace]
    Q --> A
    RS --> DB[.aimx/research/state.sqlite3]

Aim data under .aim remains the evidence source and stays read-only during Research State workflows. Explicit research writes go only to the Aimx-owned .aimx/research/state.sqlite3 sidecar.

Core concepts

One research round

research context
    -> research next / agenda
    -> agent reasons, changes code, and runs the experiment
    -> Aim records evidence
    -> aimx query / trace observes the result
    -> agent interprets the evidence
    -> research update
    -> Finding / Lineage / Frontier / Agenda evolve
    -> next round

The next agent session does not need the previous chat transcript. It can read the same Research State, compile a fresh bounded context, and continue from the persisted Agenda and Findings.

aimx research state --repo data --json
aimx research context --repo data \
  --objective "improve low-data accuracy" \
  --budget 12000 --json
aimx research next --repo data --json

# After the external experiment and read-only Aim inspection:
aimx research update --repo data --file update.json --dry-run --json
aimx research update --repo data --file update.json --json

The context budget covers the complete UTF-8 JSON response, including metadata, evidence references, selection notes, and the terminating LF. This is stricter than the earlier items-only budget, so the same limit can include fewer items or report that a complete correction group will not fit. Research reads do not create .aimx; explicit updates write only data/.aimx/research/state.sqlite3. Revision conflicts return exit status 3, requiring the caller to refresh context and reconsider its update rather than silently replay a stale decision.

See the AutoResearch protocol reference and the 008 research handoff quickstart for file evidence, receipts, and correction-aware context contracts.

Bounded multi-agent research

$aimx-autoresearch connects the research control plane to the current agent host. Supply a goal and success criteria, a maximum number of rounds, and a default agent list. Each agent can specify model and reasoning_effort; per-round overrides replace the full agent list for that round. Omitted settings inherit host defaults. Explicit settings must be supported by the host; the skill stops rather than silently substituting them.

Only the lead agent authors, dry-runs, and submits ResearchUpdate. Subagents run isolated tasks and return structured results/evidence. They must not write Research State, use its convenience write commands, or modify loop checkpoints. The lead integrates selected changes through a validated integration worktree, then uses the merged primary-workspace version as the next round's baseline.

For example, invoke the installed skill with:

Use $aimx-autoresearch with autoresearch.json. Run until its goal is supported
by evidence or its round budget is exhausted. Only the lead writes ResearchUpdate.

Start from the example config. Paths are relative to the config file; update paths and native model IDs for your project/host. Keep code-changing loops' primary workspace clean, including user-authored config files. The complete loop contract describes dispatch, results, and checkpoint patches.

From this repository, the standard-library helper can be used directly:

# Read-only validation and resolved settings for every round
uv run python skills/aimx-autoresearch/scripts/control.py validate --config autoresearch.json

# Explicitly create a loop checkpoint (does not launch any tasks)
uv run python skills/aimx-autoresearch/scripts/control.py init --config autoresearch.json --loop-id accuracy-study

# Inspect the existing loop and verify its original config
uv run python skills/aimx-autoresearch/scripts/control.py status \
  --loop-dir data/.aimx/autoresearch/accuracy-study --config autoresearch.json

When installed elsewhere, use the script inside that installed skill directory, not a path assumed to exist in the research project.

One round is one configured batch: it consumes budget before the first spawn attempt, including failed rounds. Host capacity may require waves within the same round. Failed slots are not retried within that round. The lead evaluates all original criteria after aggregation; a final-round success takes precedence over budget exhaustion.

To resume, invoke $aimx-autoresearch with the existing loop_dir and original config. Budget and recorded task identities remain unchanged; unknown live task state blocks further dispatch until reconciled. Checkpoints are stored in <aim-repo>/.aimx/autoresearch/<loop-id>/, independently of Research State. The helper neither enforces agent roles nor starts agents/training: role rules and native model support are the host/lead's responsibility. The lead verifies the actual Aimx commands, evaluator preflight, round model settings, dataset observation snapshot, and host failure policy before dispatch. Research receipts separate saved artifacts from committed knowledge; terminal follow-up uses a separate activity and leaves the finished loop unchanged. This V1 provides explicit resume, not background scheduling, process recovery, or cost/GPU budgets.

Roadmap

Research Control Plane V1 — implemented

• Durable Findings, annotations, governance, and Lineage
• Atomic ResearchUpdate with revision conflicts and idempotency
• Deterministic bounded Research Context
• Human/agent-steerable Frontier
• Durable Agenda / Experiment Contracts and deterministic research next
• Cross-session and cross-agent handoff
• Read-only Aim evidence integration
• Bounded multi-agent AutoResearch skill with configuration and checkpoints

Next directions

• Agent execution/orchestration adapters
• Experiment lifecycle adapters
• Richer automated Finding synthesis
• Research State visualization and exploration
• Longer-running autonomous research loops

These are directions rather than compatibility promises. Aimx will continue to keep the control-plane boundary explicit instead of turning into an implicit Aim mutator or a general-purpose scheduler.

Commands

Owned commands

Both aimx query and aimx trace accept optional AimQL expressions as their filter argument. When the expression is omitted or blank, aimx uses run.hash != ''. AimQL is Aim's native Python-like query language.

aimx query metrics "metric.name == 'loss' and run.hparams.learning_rate > 0.001"

For syntax, supported properties (run.*, metric.*, images.*), and security restrictions, see Aim - Query language basics.

Query metrics

aimx query metrics groups matching metric series by run and reports useful statistics such as step count, last value, min value, and max value.

# Rich table, colored in terminals by default
aimx query metrics --repo data

# Filter with AimQL
aimx query metrics "metric.name == 'loss'" --repo data

# Short run hashes are transparently expanded to full hashes
aimx query metrics "run.hash == 'eca37394' and metric.name == 'loss'" --repo data

# Tab-separated output for shell tools
aimx query metrics "metric.name == 'loss'" --repo data --oneline

# Structured JSON, nested by run
aimx query metrics "metric.name == 'loss'" --repo data --json

# Step or epoch windows
aimx query metrics "metric.name == 'loss'" --repo data --steps 100:500
aimx query metrics "metric.name == 'loss'" --repo data --epochs 1:10

# Density sampling
aimx query metrics "metric.name == 'loss'" --repo data --head 20
aimx query metrics "metric.name == 'loss'" --repo data --tail 20
aimx query metrics "metric.name == 'loss'" --repo data --every 5

Query images

aimx query images reads image metadata and, when possible, renders matched images directly in the terminal.

# Inline preview in modern terminals
aimx query images --repo data

# Metadata output only
aimx query images --repo data --plain
aimx query images --repo data --json

# Filter and sample matching image rows
aimx query images "images" --repo data --epochs 10:50 --head 10

# Control the TTY preview cap
aimx query images "images" --repo data --max-images 20
aimx query images "images" --repo data --max-images 0

When stdout is a TTY and aimx detects a graphics-capable terminal, matched images render inline. On plain ANSI terminals, aimx falls back to half-block character art and still exits with code 0.

Terminal rendering is provided by textual-image. Confirmed working terminals include iTerm2, Kitty, Konsole, WezTerm, foot, tmux (Sixel), xterm (Sixel), Windows Terminal, and VS Code integrated terminal. Warp and GNOME Terminal are not supported.

To disable inline rendering, redirect stdout or use --plain / --json.

Query run params

aimx query params reads run-level Aim metadata without modifying the repository. By default, it shows a readable set of discovered parameter columns. Use --param KEY one or more times to align specific flattened params across matching runs.

# Compare discovered params across all matching runs
aimx query params --repo data

# Select specific params
aimx query params "run.experiment == 'cloud-segmentation'" --repo data \
  --param hparam.lr \
  --param hparam.optimizer

# Script-friendly output
aimx query params "run.experiment == 'cloud-segmentation'" --repo data --plain
aimx query params "run.experiment == 'cloud-segmentation'" --repo data --json

# Filter with AimQL run fields
aimx query params "run.hparam.lr == 0.0001" --repo data --param hparam.lr

Missing selected params are displayed as - in terminal/plain output and listed under missing_params in JSON. Params output also includes run duration as a table column, a plain-output field after run name, and a JSON duration object with seconds, status, and source.

Trace metrics

aimx trace fetches the full value sequence for one or more matching metrics and renders a curve, table, CSV, or JSON export. Multiple matching runs are overlaid on the same plot.

# Plot all matching loss curves
aimx trace --repo data

# Filter with AimQL
aimx trace "metric.name == 'loss'" --repo data

# Plot one run by short hash
aimx trace "run.hash == 'eca37394' and metric.name == 'loss'" --repo data

# Step-by-step table
aimx trace "metric.name == 'loss'" --repo data --table

# CSV or JSON export
aimx trace "metric.name == 'loss'" --repo data --csv > loss.csv
aimx trace "metric.name == 'loss'" --repo data --json

# Step filtering and sampling
aimx trace "metric.name == 'loss'" --repo data --steps 100:500
aimx trace "metric.name == 'loss'" --repo data --head 50
aimx trace "metric.name == 'loss'" --repo data --every 10

Output modes: default plot, --table, --csv, --json. Display controls: --width W, --height H, --no-color.

Trace distributions

aimx trace distribution fetches tracked Aim distribution sequences. By default it prints the matched distribution names, selects the first match, and renders a non-interactive Rich terminal visual with a web-style blue-gradient current-step histogram and step-by-bin heatmap. Use --table, --csv, or --json for tensor inspection and scripting.

# Show a web-like terminal visual for the first matched distribution
aimx trace distribution --repo data

# Inspect a specific training step; nearest tracked step is used if needed
aimx trace distribution "distribution.name != ''" --repo data --step 12300

# Show distribution tensors in a readable table
aimx trace distribution "distribution.name == 'weights'" --repo data --table

# Export distribution histograms for scripting
aimx trace distribution "distribution.name == 'weights'" --repo data --csv
aimx trace distribution "distribution.name == 'weights'" --repo data --json

Common query options

• Output: --json, --oneline / --plain, or the default rich terminal view.
• Filtering: --steps start:end or --epochs start:end where supported.
• Sampling: --head N, --tail N, --every K.
• Images: --max-images N controls the TTY preview cap.
• Params: --param KEY can be repeated to select parameter columns.
• Diagnostics: --verbose prints additional details where supported.

Native Aim passthrough

Any unowned command path is passed through to native aim.

aimx up
aimx init --help
aimx runs --help
aimx runs ls

Documentation

• AimQL query language explains the filter syntax used by aimx query and aimx trace.
• textual-image terminal support lists terminals that can render inline images.
• CONSTITUTION.md documents the project safety and scope rules.
• specs/ contains feature specs, plans, contracts, and quickstarts for implemented aimx capabilities.
• TODO.md tracks early roadmap notes.

Runtime Contract

• aimx does not replace the aim executable.
• aimx does not modify the installed aim package.
• aimx does not mutate .aim data during help, version, doctor, query, trace, or passthrough flows.
• Native Aim remains an external runtime prerequisite for delegated commands.
• The repository's development dependency on Aim is only for local development and testing convenience.

Development

The project uses Python 3.12 for local development and supports >=3.10,<3.13 at runtime.

uv python install 3.12
uv venv --python 3.12
uv sync --group dev
uv run pytest

Useful local checks:

uv run aimx --help
uv run aimx version
uv run aimx doctor
uv run aimx query metrics --repo data
uv run aimx query images --repo data/.aim --json
uv run aimx query params --repo data --json