adapterci

AdapterCI

Regression CI for fine-tuned local models. Runs entirely against your own OpenAI-compatible endpoint (Ollama, LM Studio, llama.cpp, vLLM). Only metric names and floats are ever sent to the AdapterCI API — no prompts, no completions, no weights.

Install

pip install adapterci

Quickstart

adapterci init      # writes adapterci.yaml
# edit adapterci.yaml: api_key, project, endpoint, baseline_model, candidate_model
adapterci            # runs the suite, pushes scores, exits 1 on regression

Golden sets

Add domain-specific test cases in a local JSONL file (never uploaded):

{"prompt": "Summarize in one sentence: ...", "check": "contains", "expected": "..."}

check can be contains, regex, numeric_answer, or json_schema_keys.

Pointing at a different API (staging/local testing)

export ADAPTERCI_API_BASE=http://localhost:8787

License

MIT