PkgFoundry
Catalog PyPI
Back to packageRELEASE DETAILS

0.0.1b3

PyPI

ablator

Model Ablation Tool-Kit

Published2 Dec 2023
Direct dependencies39
Transitive dependenciesNot calculated

README

Version 0.0.1b3 documentation

Ablator Image

Documentation Twitter Follow Discord Slack

Version Python 3.10 codecov

🚀 ABLATOR is a DISTRIBUTED EXECUTION FRAMEWORK designed to enhance ablation studies in complex machine learning models. It automates the process of configuration and conducts multiple experiments in parallel.

What are Ablation Studies?

It involves removing specific parts of a neural network architecture or changing different aspects of the training process to examine their contributions to the model's performance.

Why ABLATOR?

As machine learning models grow in complexity, the number of components that need to be ablated also increases. This consequently expands the search space of possible configurations, requiring an efficient approach to horizontally scale multiple parallel experimental trials. ABLATOR is a tool that aids in the horizontal scaling of experimental trials.

Instead of manually configuring and conducting multiple experiments with various hyperparameter settings, ABLATOR automates this process. It initializes experiments based on different hyperparameter configurations, tracks the state of each experiment, and provides experiment persistence on the cloud.

Key Features

  • It is a tool that simplifies the process of prototyping of models.
  • It streamlines model experimentation and evaluation.
  • It offers a flexible configuration system.
  • It facilitates result interpretation through visualization.
  • "Auto-Trainer" feature reduces redundant coding tasks.

ABLATOR vs. Without ABLATOR

Left, ABLATOR efficiently conducts multiple trials in parallel based and log the experiment results.

Right, manually, one would need to run trials sequentially, demanding more effort and independent analysis.

Comparison of ABLATOR and Manual Proces

Install

For MacOS and Linux systems directly install via pip.

pip install ablator

If you are using Windows, you will need to install WSL using the official from Microsoft.

WSL is a Linux subsystem and for ABLATOR purposes is identical to using Linux.

Basic Concepts

1. Create your Configuration

from torch import nn
import torch
from ablator import (
    ModelConfig,
    ModelWrapper,
    OptimizerConfig,
    TrainConfig,
    configclass,
    Literal,
    ParallelTrainer,
    SearchSpace,
)
from ablator.config.mp import ParallelConfig


@configclass
class TrainConfig(TrainConfig):
    dataset: str = "random"
    dataset_size: int


@configclass
class ModelConfig(ModelConfig):
    layer: Literal["layer_a", "layer_b"] = "layer_a"


@configclass
class ParallelConfig(ParallelConfig):
    model_config: ModelConfig
    train_config: TrainConfig


config = ParallelConfig(
    experiment_dir="ablator-exp",
    train_config=TrainConfig(
        batch_size=128,
        epochs=2,
        dataset_size=100,
        optimizer_config=OptimizerConfig(name="sgd", arguments={"lr": 0.1}),
        scheduler_config=None,
    ),
    model_config=ModelConfig(),
    device="cpu",
    search_space={
        "model_config.layer": SearchSpace(categorical_values=["layer_a", "layer_b"])
    },
    total_trials=2,
)

2. Define your Model


class SimpleModel(nn.Module):
    def __init__(self, config: ModelConfig) -> None:
        super().__init__()
        if config.layer == "layer_a":
            self.param = nn.Parameter(torch.ones(100, 1))
        else:
            self.param = nn.Parameter(torch.randn(200, 1))

    def forward(self, x: torch.Tensor):
        x = self.param
        return {"preds": x}, x.sum().abs()


class SimpleWrapper(ModelWrapper):
    def make_dataloader_train(self, run_config: ParallelConfig):
        dl = [torch.rand(100) for i in range(run_config.train_config.dataset_size)]
        return dl

    def make_dataloader_val(self, run_config: ParallelConfig):
        dl = [torch.rand(100) for i in range(run_config.train_config.dataset_size)]
        return dl

3. Launch 🚀

mywrapper = SimpleWrapper(SimpleModel)
with ParallelTrainer(mywrapper, config) as ablator:
    ablator.launch(".")

Learn More about ABLATOR Modules

Configuration Icon Process Icon Results Icon Analysis Icon
Configuration ModuleTraining ModuleExperiment and Metrics ModuleAnalysis Module

Tutorials

Explore a variety of tutorials and examples on how to utilize ABLATOR. Ready to dive in? 👉 Ablation Tutorials

Contribution Guidelines

ABLATOR is open source, and we value contributions from our community! Check out our Development Guide for details on our development process and insights into the internals of the ABLATOR library.

For any bugs or feature requests related to ABLATOR, please visit our GitHub Issues or reach out to slack

Ablator Community

PlatformPurposeSupport Level
GitHub IssuesTo report issues or suggest new features.ABLATOR Team
SlackTo collaborate with fellow ABLATOR users.Community
DiscordTo inquire about ABLATOR usage and collaborate with other ABLATOR enthusiasts.Community
TwitterFor staying up-to-date on new features of Ablator.ABLATOR Team

References

@inproceedings{fostiropoulos2023ablator,
  title={ABLATOR: Robust Horizontal-Scaling of Machine Learning Ablation Experiments},
  author={Fostiropoulos, Iordanis and Itti, Laurent},
  booktitle={AutoML Conference 2023 (ABCD Track)},
  year={2023}
}

Downloads & popularity

Awaiting observations

No metric observations yet

Download counts and popularity will appear here as the collector receives them.

Declared dependencies 40

Native requirements
39 direct targetsNot calculated transitive count
Package / targetRequirementRelationship
numpy
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
numpy >=1.22runtime
pandas
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pandas >=2.0.0runtime
scikit-learn
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
scikit-learn >=1.2.2runtime
torch
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
torch >=2.0.0runtime
torchvision
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
torchvision >=0.14.1runtime
tqdm
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
tqdm >=4.64.1runtime
tensorboardX
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
tensorboardX >=2.6runtime
matplotlib
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
matplotlib >=3.7.1runtime
omegaconf
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
omegaconf >=2.2.3runtime
scipy
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
scipy >=1.10.1runtime
setproctitle
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
setproctitle >=1.3.2runtime
ray
Environment
{
  "extras": "default",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
ray[default] ==2.7.0runtime
pydantic
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pydantic ==1.10.11runtime
pynvml
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pynvml ==11.5.0runtime
optuna
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
optuna ==3.1.1runtime
tabulate
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
tabulate >=0.9.0runtime
seaborn
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
seaborn >=0.13runtime
numpydoc
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
numpydoc >=1.5.0runtime
paramiko
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
paramiko >=3.2.0runtime
GitPython
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
GitPython >=3.1.32runtime
gpustat
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
gpustat ==1.0runtime
rmount
Environment
{
  "extras": "",
  "marker": "platform_system == \"Linux\"",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
rmount >=0.0.6 ; platform_system == "Linux"runtime
windows-curses
Environment
{
  "extras": "",
  "marker": "platform_system == \"Windows\"",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
windows-curses ; platform_system == "Windows"runtime
mypy
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
mypy >=1.2.0 ; extra == 'dev'runtime
pytest
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pytest >=7.3.0 ; extra == 'dev'runtime
black
Environment
{
  "extras": "jupyter",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
black[jupyter] ==23.3.0 ; extra == 'dev'runtime
flake8
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
flake8 >=6.0.0 ; extra == 'dev'runtime
pylint
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pylint >=2.17.2 ; extra == 'dev'runtime
tensorboard
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
tensorboard >=2.12.2 ; extra == 'dev'runtime
mock
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
mock >=5.0.2 ; extra == 'dev'runtime
types-tabulate
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
types-tabulate >=0.9.0.2 ; extra == 'dev'runtime
types-paramiko
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
types-paramiko >=3.2.0.0 ; extra == 'dev'runtime
docker
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
docker >=6.1.3 ; extra == 'dev'runtime
pytest-xdist
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pytest-xdist >=3.3.1 ; extra == 'dev'runtime
pytest-order
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pytest-order >=1.1.0 ; extra == 'dev'runtime
pytest-rerunfailures
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pytest-rerunfailures >=12.0 ; extra == 'dev'runtime
pydoclint
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pydoclint >=0.1.0 ; extra == 'dev'runtime
pytest-cov
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
pytest-cov >=3.0.0 ; extra == 'dev'runtime
codecov-cli
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
codecov-cli >=0.2.2 ; extra == 'dev'runtime
rmount
Environment
{
  "extras": "server",
  "marker": "(platform_system == \"Linux\") and extra == 'dev'",
  "requires_python": "<3.11,>=3.10",
  "evaluation": "NOT_EVALUATED"
}
rmount[server] >=0.0.6 ; (platform_system == "Linux") and extra == 'dev'runtime

Distribution files

10 collected
ablator-0.0.1b3-py3-none-macosx_10_9_x86_64.whl

bdist_wheel · 254.99 KB · Python py3

Source file
ablator-0.0.1b3-py3-none-macosx_11_0_arm64.whl

bdist_wheel · 254.99 KB · Python py3

Source file
ablator-0.0.1b3-py3-none-manylinux2014_aarch64.whl

bdist_wheel · 254.99 KB · Python py3

Source file
ablator-0.0.1b3-py3-none-manylinux2014_armv7l.whl

bdist_wheel · 254.99 KB · Python py3

Source file
ablator-0.0.1b3-py3-none-manylinux2014_i686.whl

bdist_wheel · 254.99 KB · Python py3

Source file
ablator-0.0.1b3-py3-none-manylinux2014_ppc64le.whl

bdist_wheel · 254.99 KB · Python py3

Source file
ablator-0.0.1b3-py3-none-manylinux2014_ppc64.whl

bdist_wheel · 254.99 KB · Python py3

Source file
ablator-0.0.1b3-py3-none-manylinux2014_s390x.whl

bdist_wheel · 254.99 KB · Python py3

Source file
ablator-0.0.1b3-py3-none-manylinux2014_x86_64.whl

bdist_wheel · 254.99 KB · Python py3

Source file
ablator-0.0.1b3.tar.gz

sdist · 226.13 KB · Python source

Source file

Metadata history

Meaningful changes
Current collected state Current

Observed since 20 Oct, 17:28 UTC

No superseded metadata states collected. Unchanged observations do not create new revisions.

Record identity

package_version:2e70809d99e70ea6771abf1ffa1f49eed8d3ea7280b4c30e416f9204dc5b893c

Other collected versions

View all collected versions →