FEAT add pandas support

.github/workflows/build-test.yml

@@ -46,13 +46,30 @@ jobs:
         shell: bash
 
       - uses: prefix-dev/setup-pixi@v0.11.2
+        if: matrix.environment != 'ci-sklearn-nightly'
         with:
           pixi-version: v0.68.0
           environments: ${{ matrix.environment }}
           # we can freeze the environment and manually bump the dependencies to the
           # latest version time to time.
           frozen: true
 
+      # The nightly environment is installed by hand: the locked pandas nightly
+      # wheel is dropped from its index after a few days, so it is refreshed
+      # first. The action accepts no other input together with run-install.
+      - uses: prefix-dev/setup-pixi@v0.11.2
+        if: matrix.environment == 'ci-sklearn-nightly'
+        with:
+          pixi-version: v0.68.0
+          run-install: false
+
+      - name: Install the nightly environment with the latest pandas nightly
+        if: matrix.environment == 'ci-sklearn-nightly'
+        run: |
+          pixi update --environment ci-sklearn-nightly pandas
+          pixi install --environment ci-sklearn-nightly --frozen
+        shell: bash
+
       - name: Linters
         run: pixi run -e lint lint
 

docs/changes.rst

@@ -21,6 +21,25 @@ v0.17
   with an error while it is read, before anything in it is audited or
   constructed, instead of failing with an unrelated error, or being accepted,
   during construction. :pr:`547` by `Adrin Jalali`_.
+- Add support for pandas objects: :class:`~pandas.DataFrame`,
+  :class:`~pandas.Series`, every kind of :class:`~pandas.Index`, and the
+  extension arrays and extension dtypes of pandas itself, not those of other
+  libraries, can now be saved and loaded. They are stored as
+  the numpy arrays and scalars they are made of and rebuilt through the public
+  pandas constructors, so no pandas internals end up in the file. They are
+  not trusted by default: the pandas types a file holds, which
+  :func:`skops.io.get_untrusted_types` lists, have to be passed as
+  ``trusted``. The scalars :class:`~pandas.Timestamp`,
+  :class:`~pandas.Timedelta` and ``pandas.NA`` are saved as well. The pyarrow
+  backed arrays and dtypes, those with an ``ArrowDtype``, are not supported
+  yet; the string dtype with pyarrow storage is, as a string dtype.
+  Estimators from other libraries that keep pandas objects
+  in their fitted attributes, such as ``category_encoders``, can now be
+  persisted. A file written with one pandas version loads with any other from
+  2.0 on, keeping the dtypes of the version that wrote it. Not preserved are
+  the ``freq`` of datetime-like indexes and arrays, the ``attrs`` and ``flags``
+  of a Series or DataFrame, and the storage, python or pyarrow, of a string
+  dtype. :pr:`552` by `Adrin Jalali`_.
 - Fix a regression since v0.12.0 where saving an object whose ``__reduce__``
   raises failed at dump time. ``__reduce__`` is called on every object to
   detect a plain constructor call, but Cython extension types with a

docs/persistence.rst

@@ -250,7 +250,22 @@ Supported libraries
 Skops intends to support all of **scikit-learn**, that is, not only its
 estimators, but also other classes like cross validation splitters. Furthermore,
 most types from **numpy** and **scipy** should be supported, such as (sparse)
-arrays, dtypes, random generators, and ufuncs.
+arrays, dtypes, random generators, and ufuncs. **pandas** objects, that is
+``DataFrame``, ``Series``, every kind of ``Index``, and the extension arrays
+and extension dtypes of pandas itself, not those of other libraries, are
+supported as well with pandas 2.0 or later: they are
+stored as the arrays they are made of and rebuilt through the public pandas
+constructors, so that no pandas internals end up in the file, and a file
+written with one pandas version loads with any other. They are not trusted by
+default: the pandas types a file holds, which
+:func:`skops.io.get_untrusted_types` lists, have to be passed as ``trusted``.
+The scalars ``Timestamp``, ``Timedelta`` and ``NA`` are supported as well. The
+pyarrow backed arrays and dtypes, those with an ``ArrowDtype``, are not
+supported yet; the string dtype with pyarrow storage is, as a string dtype.
+Not preserved are the
+``freq`` of datetime-like indexes and arrays, the ``attrs`` and ``flags`` of a
+``Series`` or ``DataFrame``, and the storage, python or pyarrow, of a string
+dtype, which is an environment choice over the same values.
 
 Apart from this core, we plan to support machine learning libraries commonly
 used be the community. So far, we have tested the following libraries:

docs/requirements.txt

@@ -1,6 +1,6 @@
 # to be synced with the versions in pyproject.toml
 matplotlib>=3.3
-pandas>=1
+pandas>=2
 fairlearn>=0.7.0
 sphinx>=3.2.0
 sphinx-gallery>=0.7.0