agent-virtual-shell

virtual-shell

A virtual shell-over-database filesystem for AI agents.

virtual-shell gives an AI agent a private filesystem with a shell-style interface (ls, cat, grep, echo) where the backend is a database, not a disk. Agents are trained on millions of real terminal sessions, so a shell is the interface they use most reliably — but a real shell executes arbitrary code and a real disk needs per-agent compute. This package keeps the familiar language and replaces everything underneath with a strict, parameterized, append-only storage layer.

It is a memory layer only. It reads, writes, searches, and organizes files. It never executes programs, accesses the network, or spawns processes.

Status: early development (spec phase). The behavior is fully specified in openspec/ (change: add-virtual-shell) and being implemented incrementally across milestones M1–M4. This is a Python port of the v1.0 technical specification; the spec is the source of truth where the two disagree.

Documentation

• New here? Read the plain-language guide. — what it is, why, and how it's built, for dummies.
• Architecture snapshot — module map, data model, boundaries, extensibility seams.
• Technical specification v1.0 — the normative source of truth (open in a browser).
• OpenSpec change add-virtual-shell — the proposal, design, capability specs, and step-by-step tasks under openspec/changes/. Browse with openspec show add-virtual-shell.

Why

• Phrasebook, not translator. A whitelist parser recognizes a frozen shell dialect and compiles it to a typed AST. There is no general shell interpretation and no path by which agent text reaches SQL as anything other than a bind parameter.
• Append-only. Every write is a new version row; deletes are tombstones. Nothing an agent does is destructive. Retention is a separate, operator-run compaction policy.
• Identity is injected. Tenant and agent IDs come from the host application; the dialect has no syntax for identity, and no syntax to escape a scope.
• Teaching errors. Every rejection tells the model what to do instead, so it self-corrects in one turn.
• Two doors, one table. Agents use the Shell (strings in/out). Humans and UIs use the Inspector (structured JSON, read-only). Same storage, same scope rules.

Install

pip install agent-virtual-shell            # core, zero runtime dependencies
pip install "agent-virtual-shell[postgres]"   # + asyncpg convenience extra
pip install "agent-virtual-shell[s3]"          # + aioboto3 for the blob tier

Installs as agent-virtual-shell on PyPI but imports as virtual_shell (from virtual_shell import create_shell). The exact name virtual-shell is blocked by PyPI's similarity guard, so the distribution carries the agent- prefix.

The Postgres adapter accepts any object satisfying a small async Queryable protocol, so it works with asyncpg, a SQLAlchemy async session, or your own wrapper — no hard dependency.

Quick start (planned API)

from virtual_shell import create_shell, Scope
from virtual_shell.adapters import MemoryAdapter

shell = create_shell(adapter=MemoryAdapter(), scope=Scope(tenant_id="t1", agent_id="a1"))

print((await shell.exec('echo "hello" > /notes/todo.md')).stdout)
# wrote 6 bytes to /notes/todo.md (v1)
print((await shell.exec("cat /notes/todo.md")).stdout)
# hello

shell.exec never raises: every input — including pathological ones — resolves to a ShellResult with ok, exit_code, stdout, and a teaching stderr.

Architecture

 agent (LLM)                         dashboard / human
      |                                     |
 tool call: shell(cmd)              host app's HTTP API
      |                                     |
   Shell  (string in/out)            Inspector (read-only JSON)
   tokenize → parse → interpret      tree / read / history / changes
      \                                   /
       \         scope injected by host  /
        v                               v
        StorageAdapter (interface)
          MemoryAdapter    — reference impl, tests/dev
          PostgresAdapter  — parameterized SQL, dedicated `virtual_shell` schema
          TieredAdapter    — wraps any adapter + BlobStore for large files

Database

Tables live in a dedicated Postgres schema (virtual_shell by default) in whatever database you point the adapter at — including one shared with a host application. Apply schema.sql once:

psql "$DATABASE_URL" -f schema.sql

Wiring up the PostgresAdapter

PostgresAdapter imports no database driver — it accepts any object satisfying the async Queryable protocol (query(sql, params) -> rows, native $1, $2, … placeholders). A wrapper is ~5 lines:

# asyncpg
import asyncpg
from virtual_shell.adapters import PostgresAdapter

pool = await asyncpg.create_pool(dsn)

class AsyncpgQueryable:
    async def query(self, sql, params=()):
        return [dict(r) for r in await pool.fetch(sql, *params)]

adapter = PostgresAdapter(AsyncpgQueryable())            # schema="virtual_shell", table="files"

# SQLAlchemy async (asyncpg driver, $-style params)
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession

class SqlAlchemyQueryable:
    def __init__(self, session: AsyncSession): self._s = session
    async def query(self, sql, params=()):
        # translate $1.. to :p1.. for SQLAlchemy bind params
        bound = {f"p{i}": v for i, v in enumerate(params, 1)}
        stmt = text(__import__("re").sub(r"\$(\d+)", r":p\1", sql))
        return [dict(r._mapping) for r in (await self._s.execute(stmt, bound)).fetchall()]

compact(adapter, scope, RetentionPolicy(keep_versions=N)) prunes old versions (the only operation that deletes rows; never the latest version or a latest tombstone).

Inspector (read-only dashboard API)

The second door: structured, read-only JSON for humans and UIs — same adapter, same scope rules, no parser. Build one per request with the scope from your auth, and wrap the four methods as HTTP endpoints:

from virtual_shell import create_inspector, Scope

inspector = create_inspector(adapter, Scope(tenant_id=customer.id, agent_id=agent.id))

await inspector.tree("/")              # GET …/fs/tree?prefix=     → metadata, never content
await inspector.read(path, version)    # GET …/fs/file?path=&version= → inline record or {blob, url}
await inspector.history(path)          # GET …/fs/history?path=    → all versions, tombstones flagged
await inspector.changes(cursor)        # GET …/fs/changes?cursor=  → append-only feed (poll every 2–3 s)

read returns an InlineRead (full content) for inline files, or a BlobRead (metadata + a presigned GET URL) for blob-tier files. changes(None) starts "from now"; pass the returned cursor on the next poll to get only new events. The Inspector is strictly read-only — any future human edit must be routed as an ordinary adapter write (a new version).

Large files (blob tier)

Wrap any adapter in a TieredAdapter to offload files over a threshold (default 100 KB) to a BlobStore, keeping an index (preview, type metadata, search text) in the row. Bytes go to the blob store; Postgres finds the file, a blob scan finds the line.

import aioboto3
from virtual_shell import TieredAdapter, S3BlobStore, create_shell, Scope

session = aioboto3.Session()
async with session.client("s3") as s3:
    blobs = S3BlobStore(client=s3, bucket="agent-memory")
    adapter = TieredAdapter(pg_adapter, blobs)          # inline_threshold_bytes=102400
    shell = create_shell(adapter, Scope(tenant_id=customer.id, agent_id=agent.id))

S3BlobStore takes an injected client (the AWS SDK stays an optional peer). Keys are content-addressed (<tenant>/<sha256>), so identical bytes are stored once and mv copies no bytes. grep is two-stage: a prefix search finds large files by index; grep "term" /path/to/large/file scans that one file's lines.

Recommended bucket configuration (host's responsibility, not code): server-side encryption (SSE-S3 or SSE-KMS), a private bucket policy, and a lifecycle rule transitioning objects to Infrequent Access at ~90 days and Glacier at ~365 days. Content-addressed keys are immutable, so transitions are safe.

Development

uv sync --extra dev
uv run pytest
uv run mypy
uv run ruff check

License

MIT © Camilo Jiménez