PkgFoundry
Catalog PyPI
Back to packageRELEASE DETAILS

0.0.1

PyPI

ablator-ken-test3

Model Ablation Tool-Kit

Published27 Jun 2023
Direct dependencies23
Transitive dependenciesNot calculated

README

Version 0.0.1 documentation

ABLATOR

A distributed experiment execution framework for ablation studies. ABLATOR provides a wrapper for your model and a Trainer class for you to prototype on your method and scale to thousands of experimental trials with 1 code change.

Ablation studies are experiments used to identify the causal effects on a method performance. For example, does your novel layer really improve performance?

What are Ablators?

Ablators are materials that are depleted during operation (NASA). An experimental ABLATOR should not interfere with the experimental result.

Why ABLATOR?

  1. Strictly typed configuration system prevents errors.
  2. Seamless prototyping to production
  3. Stateful experiment design. Stop, Resume, Share your experiments
  4. Automated analysis artifacts
  5. Template Training

What is the difference with using xxx

Comparison table with existing framework:

FrameworkHPOConfigurationTrainingTuningAnalysis
Ray:white_check_mark::x::x::white_check_mark::x:
Lighting:x::x::white_check_mark::x::x:
Optuna:white_check_mark::x::x::x::white_check_mark:
Hydra:x::white_check_mark::x::x::x:
ABLATOR:heavy_check_mark::heavy_check_mark::heavy_check_mark::heavy_check_mark::heavy_check_mark:

Features compared, hyperparameter selection (HPO), removing boilerplate code for configuring experiments (Configuration), removing boiler plate code for running experiments at scale (Tuning) and performing analysis on the hyperparameter selection (Analysis).

Using:

  1. Ray: You will need to write boiler-plate code for integrating with a configuration system (i.e. Hydra), saving experiments artifacts or logging (i.e. integrate with Wandb).

  2. Lighting: You will need to write boiler-plate code for HPO (i.e. using Optuna), Configuring experiments (i.e. Hydra) and horizontal distributed execution (i.e. integrate with Ray)

  3. Hydra: The configuration system is not strongly typed (ABLATOR), and does not provide support for common ML use-cases where configuration attributes are Derived (inferred during run-time) or Stateless (change between trials). Additionally, ABLATOR provides support for custom objects that are dynamically inferred and initialized during execution.

  4. ABLATOR: Combines Ray back-end, with Optuna for HPO and removes boiler-plate code for fault tollerant strategies, training, and analyzing the results.

Integrating different tools, for distributed execution, fault tollerance, training, checkpointing and analysis is error prone! Poor compatibility between tools, verisioning errors will lead to errors in your analysis.

You can use ABLATOR with any other library i.e. PyTorch Lighting. Just wrap a Lighting model with ModelWrapper. For examples please look examples

Spend more time in the creative process of ML research and less time on dev-ops.

Pre-Release - Phase

The library is under active development and a lot of the API endpoints will be removed / renamed or their functionality changed without notice.

Install

Use a python virtual enviroment to avoid version conflicts.

pip install git+https://github.com/fostiropoulos/ablator.git

For Development

  1. git clone git@github.com:fostiropoulos/ablator.git
  2. cd ablator
  3. pip install -e .[dev]

Downloads & popularity

Awaiting observations

No metric observations yet

Download counts and popularity will appear here as the collector receives them.

Declared dependencies 23

Native requirements
23 direct targetsNot calculated transitive count
Package / targetRequirementRelationship
numpy
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
numpy (==1.24.1)runtime
pandas
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
pandas (==2.0.0)runtime
scikit-learn
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
scikit-learn (==1.2.2)runtime
torch
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
torch (==1.13.1)runtime
torchvision
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
torchvision (==0.14.1)runtime
tqdm
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
tqdm (==4.64.1)runtime
tensorboardX
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
tensorboardX (==2.6)runtime
matplotlib
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
matplotlib (==3.7.1)runtime
omegaconf
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
omegaconf (==2.2.3)runtime
scipy
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
scipy (==1.10.1)runtime
setproctitle
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
setproctitle (==1.3.2)runtime
ray
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
ray (<=2.2.0,>=2.1.0)runtime
pynvml
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
pynvml (==11.5.0)runtime
optuna
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
optuna (==3.1.1)runtime
tabulate
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
tabulate (==0.9.0)runtime
seaborn
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
seaborn (==0.12.2)runtime
numpydoc
Environment
{
  "extras": "",
  "marker": null,
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
numpydoc (==1.5.0)runtime
mypy
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
mypy (==1.2.0) ; extra == 'dev'runtime
pytest
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
pytest (==7.3.0) ; extra == 'dev'runtime
black
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
black (==23.3.0) ; extra == 'dev'runtime
flake8
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
flake8 (==6.0.0) ; extra == 'dev'runtime
pylint
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
pylint (==2.17.2) ; extra == 'dev'runtime
tensorboard
Environment
{
  "extras": "",
  "marker": "extra == 'dev'",
  "requires_python": ">3.10",
  "evaluation": "NOT_EVALUATED"
}
tensorboard (==2.12.2) ; extra == 'dev'runtime

Distribution files

2 collected
ablator_ken_test3-0.0.1-py3-none-any.whl

bdist_wheel · 94.45 KB · Python py3

Source file
ablator_ken_test3-0.0.1.tar.gz

sdist · 83.35 KB · Python source

Source file

Metadata history

Meaningful changes
Current collected state Current

Observed since 20 Oct, 17:28 UTC

No superseded metadata states collected. Unchanged observations do not create new revisions.

Record identity

package_version:58ca7e5e50fca1c0359e050491ce3861e5a2cd3d5286376402e2b7fbf492a0df

Other collected versions

View all collected versions →