aimip-intake-icechunk

aimip-intake-icechunk

A small classic-Intake driver that opens icechunk repositories as xarray.Datasets. Defaults are tuned for the AIMIP catalog on DKRZ S3, but the driver is general — point it at any icechunk repo (local path, file:// URL, or s3://...).

import intake

# YAML catalog references it as
#   driver: aimip_intake_icechunk.IcechunkSource
#   args:
#     urlpath: s3://your-bucket/path/to/icechunk-repo
cat = intake.open_catalog("path/to/catalog.yaml")
ds = cat['my_store'].to_dask()

Or directly in Python:

from aimip_intake_icechunk import IcechunkSource

src = IcechunkSource(urlpath="s3://ai-mip/catalog/CMIP6/AIMIP/NVIDIA/cBottle-1-3/aimip/r1i1p4f1/Amon/gn")
ds = src.to_dask()

Why a separate package

Consumers of the aimip-data catalog who only need read access shouldn't have to install the build-time machinery (VirtualiZarr, icechunk's writers, the facet parser, the scanner, the YAML emitter, etc.). This driver is ~150 lines and depends only on icechunk, intake, xarray, zarr.

Install

pip install aimip-intake-icechunk

Or from a checkout of the aimip-data repo:

pip install -e packages/aimip-intake-icechunk

Defaults

For convenience, the driver defaults to the AIMIP layout:

Override at construction time for other deployments:

IcechunkSource(
    urlpath="s3://my-bucket/repo",
    s3_options={"endpoint_url": "https://s3.amazonaws.com", "anonymous": False},
    virtual_chunk_auth={"s3://my-bucket/": my_credentials},
)

License

MIT.