Update CIs

.github/PULL_REQUEST_TEMPLATE.md

@@ -12,7 +12,7 @@
 
 - [ ] Tests cover the new behaviour / the fixed bug (regression test)
 - [ ] Docs updated (`docs/`, `README.md`, `config/config_template.yaml` for new config keys)
-- [ ] `CHANGELOG.md` entry under the development version
+- [ ] `CHANGELOG.md` updated for functional changes, compared to the latest release ([what goes in](https://github.com/lithops-cloud/lithops/blob/master/CONTRIBUTING.md#changelog))
 
 ---
 

.github/workflows/docs-check.yml → .github/workflows/docs-build.yml

@@ -1,4 +1,4 @@
-name: Docs
+name: Docs build
 
 # Builds the Sphinx docs on pull requests that change them, and fails on any warning, so
 # broken markup, missing pages or bad docstrings are caught before they are merged
@@ -16,7 +16,21 @@ on:
       - 'lithops/storage/storage.py'
       - 'lithops/utils.py'
       - 'pyproject.toml'
-      - '.github/workflows/docs-check.yml'
+      - '.github/workflows/docs-build.yml'
+
+  # after a merge too, so master has a status of its own (the README badge shows it)
+  push:
+    branches:
+      - master
+    paths:
+      - 'docs/**'
+      - 'lithops/executors.py'
+      - 'lithops/concurrent/futures.py'
+      - 'lithops/retries.py'
+      - 'lithops/storage/storage.py'
+      - 'lithops/utils.py'
+      - 'pyproject.toml'
+      - '.github/workflows/docs-build.yml'
 
   workflow_dispatch:
   # this allows to run the workflow manually through the github dashboard

.github/workflows/docs.yml → .github/workflows/publish-docs.yml

@@ -4,9 +4,9 @@ name: Publish docs
 # repository (<owner>/lithops-cloud.github.io), authored by whoever runs the workflow.
 #
 # It always builds master, and names the commit after the latest GitHub release
-# ("Update docs to X"). The Release workflow calls it at the end of every release; run it by
-# hand from the Actions tab (Publish docs -> Run workflow) to republish the docs without a
-# release, e.g. after fixing them.
+# ("Update docs to X"). The Publish release workflow calls it at the end of every release;
+# run it by hand from the Actions tab (Publish docs -> Run workflow) to republish the docs
+# without a release, e.g. after fixing them.
 #
 # Pushing to the docs repository needs a deploy key with write access: its private key goes
 # in the DOCS_DEPLOY_KEY secret of this repository; without it the docs are only built.
@@ -29,6 +29,11 @@ jobs:
       cancel-in-progress: false
     permissions:
       contents: read
+    # shows every publish in the Deployments section of this repository, linking to the docs
+    # (the GitHub Pages site of the docs repository, e.g. https://lithops-cloud.github.io/docs/)
+    environment:
+      name: github-pages
+      url: https://${{ github.repository_owner }}.github.io/docs/
     env:
       # the GitHub Pages repository of the same owner, e.g. lithops-cloud/lithops-cloud.github.io
       DOCS_REPOSITORY: ${{ github.repository_owner }}/lithops-cloud.github.io

.github/workflows/release.yml

@@ -1,14 +1,14 @@
-name: Release
+name: Publish release
 
-# Run manually from the Actions tab: Release -> Run workflow -> version (e.g. 3.7.1).
+# Run manually from the Actions tab: Publish release -> Run workflow -> version (e.g. 3.8.1).
 # It does what the release used to take by hand:
 #   1. sets lithops/version.py to the version and renames the development heading of
 #      CHANGELOG.md to it, commits "Update version to X" and tags it X
 #   2. builds the sdist and wheel and publishes them (trusted publishing, no token)
 #   3. creates the GitHub release "Lithops-X" with the changelog section as notes
 #   4. bumps the branch to the next development version (X.Y.Z+1.dev0)
 #   5. at the end, publishes the docs to the GitHub Pages repository as "Update docs to X"
-#      (the Publish docs workflow, docs.yml, which can also be run by hand to republish them)
+#      (the Publish docs workflow, publish-docs.yml, which can also be run by hand to republish them)
 # Commits go to the branch the workflow runs on, authored by whoever runs it.
 # With "dry run" checked it only does the edits and the package build, and pushes or
 # publishes nothing.
@@ -20,7 +20,7 @@ on:
   workflow_dispatch:
     inputs:
       version:
-        description: "Version to release, e.g. 3.7.1"
+        description: "Version to release, e.g. 3.8.1"
         required: true
       dry_run:
         description: "Dry run: build and check only, do not push, publish or release"
@@ -86,7 +86,7 @@ jobs:
 
           version = os.environ['VERSION'].strip()
           if not re.fullmatch(r'\d+\.\d+\.\d+', version):
-              sys.exit(f'Version must look like 3.7.1, got {version!r}')
+              sys.exit(f'Version must look like 3.8.1, got {version!r}')
           if subprocess.run(['git', 'rev-parse', '-q', '--verify', f'refs/tags/{version}'],
                             capture_output=True).returncode == 0:
               sys.exit(f'Tag {version} already exists')
@@ -103,7 +103,7 @@ jobs:
               changelog = f.read()
           heading = re.search(r'^## \[v?(\d+\.\d+\.\d+)\.dev\d*\][ \t]*$', changelog, re.M)
           if not heading:
-              sys.exit('The first CHANGELOG.md section must be the development version, e.g. ## [v3.7.1.dev0]')
+              sys.exit('The first CHANGELOG.md section must be the development version, e.g. ## [v3.8.1.dev0]')
           changelog = changelog[:heading.start()] + f'## [v{version}]' + changelog[heading.end():]
           with open('CHANGELOG.md', 'w') as f:
               f.write(changelog)
@@ -253,5 +253,5 @@ jobs:
     needs: [github-release, bump-dev]
     permissions:
       contents: read
-    uses: ./.github/workflows/docs.yml
+    uses: ./.github/workflows/publish-docs.yml
     secrets: inherit

.github/workflows/tests.yml

@@ -9,6 +9,15 @@ on:
       - 'lithops/**'
       - '.github/workflows/tests.yml'
 
+  # after a merge too, so master has a status of its own (the README badge shows it)
+  push:
+    branches:
+      - master
+    paths:
+      - 'pyproject.toml'
+      - 'lithops/**'
+      - '.github/workflows/tests.yml'
+
   workflow_dispatch: 
   # this allows to run the workflow manually through the github dashboard
 

AGENTS.md

@@ -98,9 +98,15 @@ requests that touch the docs must build without warnings: CI runs that same comm
 - Every bug fix includes a regression test; every feature includes tests of its behaviour.
   Tests must run on the localhost backend and storage; backend-specific code that cannot be
   exercised locally is tested with fakes or mocks.
-- User-visible changes go into `CHANGELOG.md` under the topmost (development) version
-  heading, in the *Added / Changed / Fixed / Removed* section, prefixed with the component,
-  e.g. `- [Monitoring] ...`. Update `docs/` and `README.md` when behaviour changes.
+- Functional changes (API, backends, behaviour, bug fixes in the library) go into
+  `CHANGELOG.md` under the topmost (development) version heading, in the *Added / Changed /
+  Fixed / Removed* section, prefixed with the component, e.g. `- [Monitoring] ...`. Docs
+  fixes, packaging metadata and repository tooling (CI, templates, agent files) get no entry:
+  that section becomes the release notes. Entries describe the difference from the latest
+  release, not the history of the development cycle: a fix or change to something added since
+  that release updates its existing `Added` entry instead of adding a `Fixed` or `Changed` one,
+  and something added and removed again before a release has no entry. Update `docs/` and
+  `README.md` when behaviour changes.
 - `CONTRIBUTING.md` and `docs/source/contributing.rst` carry the same content: change both.
 - Pull requests target `master`. Keep changes small and focused; do not refactor unrelated
   code.

CONTRIBUTING.md

@@ -90,15 +90,23 @@ with any user-visible change.
 
 ## Changelog
 
-Add an entry for every user-visible change to [CHANGELOG.md](CHANGELOG.md), under the topmost
-(development) version, in the *Added*, *Changed*, *Fixed* or *Removed* section, prefixed with
-the component:
+Add an entry for every functional change (API, backends, behaviour, bug fixes in the library)
+to [CHANGELOG.md](CHANGELOG.md), under the topmost (development) version, in the *Added*,
+*Changed*, *Fixed* or *Removed* section, prefixed with the component. Docs fixes, packaging
+metadata and repository tooling need no entry, since that section becomes the release notes:
 
 ```markdown
 ### Fixed
 - [AWS Lambda] Short description of the fix.
 ```
 
+Entries describe what changed compared to the latest release, not the history of the
+development cycle. If you fix or change something that was added after the latest release, update
+its existing *Added* entry instead of adding a *Fixed* or *Changed* one: users of the release
+never had the broken version. Likewise, if something is added and removed again before a
+release, remove its entry. For example, a bug found in a backend added in this cycle is fixed
+without a *Fixed* entry, since the backend is still listed under *Added*.
+
 ## Pull requests
 
 1. Break your work into small, single-purpose pull requests. A large change with many
@@ -112,8 +120,8 @@ the component:
 
 ## Releasing (maintainers)
 
-Releases are made by the [Release workflow](.github/workflows/release.yml): *Actions* ->
-*Release* -> *Run workflow*, with the version to release (e.g. `3.7.1`). Before running it,
+Releases are made by the [Publish release workflow](.github/workflows/release.yml): *Actions* ->
+*Publish release* -> *Run workflow*, with the version to release (e.g. `3.8.1`). Before running it,
 review the development section at the top of `CHANGELOG.md`, which becomes the release notes.
 The workflow sets the version, tags it, publishes the sdist and wheel to PyPI, creates the
 GitHub release, publishes the docs to the website repository and bumps `master` to the next

README.md

@@ -5,7 +5,7 @@
 </p>
 
 <p align="center">
-  <a aria-label="License" href="https://github.com/lithops-cloud/lithops/blob/master/LICENSE"><img alt="" src="https://img.shields.io/github/license/lithops-cloud/lithops?style=for-the-badge&labelColor=000000"></a>&nbsp<a aria-label="PyPI" href="https://pypi.org/project/lithops/"><img alt="" src="https://img.shields.io/pypi/v/lithops?style=for-the-badge&labelColor=000000"></a>&nbsp<a aria-label="Python" href="#lithops"><img alt="" src="https://img.shields.io/pypi/pyversions/lithops?style=for-the-badge&labelColor=000000"></a>&nbsp<a href="https://pypistats.org/packages/lithops"><img alt="PyPI - Downloads" src="https://img.shields.io/pypi/dm/lithops?label=pypi%7Cdownloads&style=for-the-badge&labelColor=000000"></a>&nbsp<a href="https://deepwiki.com/lithops-cloud/lithops" target="_blank" rel="noopener"><img alt="Ask DeepWiki" src="https://img.shields.io/badge/DeepWiki-Ask%20DeepWiki-blue.svg?style=for-the-badge&labelColor=000000&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McDcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==" style="vertical-align:middle;"></a>
+  <a aria-label="Tests" href="https://github.com/lithops-cloud/lithops/actions/workflows/tests.yml?query=branch%3Amaster"><img alt="Tests" src="https://img.shields.io/github/actions/workflow/status/lithops-cloud/lithops/tests.yml?branch=master&label=tests&style=for-the-badge&labelColor=000000"></a>&nbsp<a aria-label="Docs" href="https://github.com/lithops-cloud/lithops/actions/workflows/docs-build.yml?query=branch%3Amaster"><img alt="Docs" src="https://img.shields.io/github/actions/workflow/status/lithops-cloud/lithops/docs-build.yml?branch=master&label=docs&style=for-the-badge&labelColor=000000"></a>&nbsp<a aria-label="License" href="https://github.com/lithops-cloud/lithops/blob/master/LICENSE"><img alt="" src="https://img.shields.io/github/license/lithops-cloud/lithops?style=for-the-badge&labelColor=000000"></a>&nbsp<a aria-label="PyPI" href="https://pypi.org/project/lithops/"><img alt="" src="https://img.shields.io/pypi/v/lithops?style=for-the-badge&labelColor=000000"></a>&nbsp<a aria-label="Python" href="#lithops"><img alt="" src="https://img.shields.io/pypi/pyversions/lithops?style=for-the-badge&labelColor=000000"></a>&nbsp<a href="https://pypistats.org/packages/lithops"><img alt="PyPI - Downloads" src="https://img.shields.io/pypi/dm/lithops?label=pypi%7Cdownloads&style=for-the-badge&labelColor=000000"></a>&nbsp<a href="https://deepwiki.com/lithops-cloud/lithops" target="_blank" rel="noopener"><img alt="Ask DeepWiki" src="https://img.shields.io/badge/DeepWiki-Ask%20DeepWiki-blue.svg?style=for-the-badge&labelColor=000000&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACwAAAAyCAYAAAAnWDnqAAAAAXNSR0IArs4c6QAAA05JREFUaEPtmUtyEzEQhtWTQyQLHNak2AB7ZnyXZMEjXMGeK/AIi+QuHrMnbChYY7MIh8g01fJoopFb0uhhEqqcbWTp06/uv1saEDv4O3n3dV60RfP947Mm9/SQc0ICFQgzfc4CYZoTPAswgSJCCUJUnAAoRHOAUOcATwbmVLWdGoH//PB8mnKqScAhsD0kYP3j/Yt5LPQe2KvcXmGvRHcDnpxfL2zOYJ1mFwrryWTz0advv1Ut4CJgf5uhDuDj5eUcAUoahrdY/56ebRWeraTjMt/00Sh3UDtjgHtQNHwcRGOC98BJEAEymycmYcWwOprTgcB6VZ5JK5TAJ+fXGLBm3FDAmn6oPPjR4rKCAoJCal2eAiQp2x0vxTPB3ALO2CRkwmDy5WohzBDwSEFKRwPbknEggCPB/imwrycgxX2NzoMCHhPkDwqYMr9tRcP5qNrMZHkVnOjRMWwLCcr8ohBVb1OMjxLwGCvjTikrsBOiA6fNyCrm8V1rP93iVPpwaE+gO0SsWmPiXB+jikdf6SizrT5qKasx5j8ABbHpFTx+vFXp9EnYQmLx02h1QTTrl6eDqxLnGjporxl3NL3agEvXdT0WmEost648sQOYAeJS9Q7bfUVoMGnjo4AZdUMQku50McDcMWcBPvr0SzbTAFDfvJqwLzgxwATnCgnp4wDl6Aa+Ax283gghmj+vj7feE2KBBRMW3FzOpLOADl0Isb5587h/U4gGvkt5v60Z1VLG8BhYjbzRwyQZemwAd6cCR5/XFWLYZRIMpX39AR0tjaGGiGzLVyhse5C9RKC6ai42ppWPKiBagOvaYk8lO7DajerabOZP46Lby5wKjw1HCRx7p9sVMOWGzb/vA1hwiWc6jm3MvQDTogQkiqIhJV0nBQBTU+3okKCFDy9WwferkHjtxib7t3xIUQtHxnIwtx4mpg26/HfwVNVDb4oI9RHmx5WGelRVlrtiw43zboCLaxv46AZeB3IlTkwouebTr1y2NjSpHz68WNFjHvupy3q8TFn3Hos2IAk4Ju5dCo8B3wP7VPr/FGaKiG+T+v+TQqIrOqMTL1VdWV1DdmcbO8KXBz6esmYWYKPwDL5b5FA1a0hwapHiom0r/cKaoqr+27/XcrS5UwSMbQAAAABJRU5ErkJggg==" style="vertical-align:middle;"></a>
 </p>
 
 Lithops is a Python multi-cloud distributed computing framework that lets you run unmodified Python code at massive scale across cloud, HPC, and on-premise platforms. It supports major cloud providers and Kubernetes platforms, running your code transparently without requiring you to manage deployment or infrastructure.

docs/README.md

@@ -1,8 +1,8 @@
 # Build Lithops documentation
 
-The [Release workflow](../.github/workflows/release.yml) builds these docs and publishes them to
+The [Publish release workflow](../.github/workflows/release.yml) builds these docs and publishes them to
 the website repository on every release. To republish them without a release (e.g. after fixing
-them), run the [Publish docs workflow](../.github/workflows/docs.yml) from the *Actions* tab: it
+them), run the [Publish docs workflow](../.github/workflows/publish-docs.yml) from the *Actions* tab: it
 builds `master` and publishes it as the docs of the latest release. The steps below are for building them locally.
 
 1. Install Lithops with [Sphinx](https://www.sphinx-doc.org/en/master/usage/installation.html) and all plugins, from the repository root:

docs/source/contributing.rst

@@ -106,15 +106,24 @@ together with any user-visible change.
 Changelog
 ---------
 
-Add an entry for every user-visible change to `CHANGELOG.md <https://github.com/lithops-cloud/lithops/blob/master/CHANGELOG.md>`_, under the
+Add an entry for every functional change (API, backends, behaviour, bug fixes in the library)
+to `CHANGELOG.md <https://github.com/lithops-cloud/lithops/blob/master/CHANGELOG.md>`_, under the
 topmost (development) version, in the *Added*, *Changed*, *Fixed* or *Removed* section, prefixed
-with the component:
+with the component. Docs fixes, packaging metadata and repository tooling need no entry, since
+that section becomes the release notes:
 
 .. code:: markdown
 
    ### Fixed
    - [AWS Lambda] Short description of the fix.
 
+Entries describe what changed compared to the latest release, not the history of the
+development cycle. If you fix or change something that was added after the latest release, update
+its existing *Added* entry instead of adding a *Fixed* or *Changed* one: users of the release
+never had the broken version. Likewise, if something is added and removed again before a
+release, remove its entry. For example, a bug found in a backend added in this cycle is fixed
+without a *Fixed* entry, since the backend is still listed under *Added*.
+
 
 Pull requests
 -------------
@@ -132,8 +141,8 @@ Pull requests
 Releasing (maintainers)
 -----------------------
 
-Releases are made by the `Release workflow <https://github.com/lithops-cloud/lithops/blob/master/.github/workflows/release.yml>`_:
-*Actions* -> *Release* -> *Run workflow*, with the version to release (e.g. ``3.7.1``). Before
+Releases are made by the `Publish release workflow <https://github.com/lithops-cloud/lithops/blob/master/.github/workflows/release.yml>`_:
+*Actions* -> *Publish release* -> *Run workflow*, with the version to release (e.g. ``3.8.1``). Before
 running it, review the development section at the top of ``CHANGELOG.md``, which becomes the
 release notes. The workflow sets the version, tags it, publishes the sdist and wheel to PyPI,
 creates the GitHub release, publishes the docs to the website repository and bumps ``master``