Skip to content

FIX recursive wrap, plus controlling thread cleanups and doc improvements - #294

Open
itamarst wants to merge 16 commits into
joblib:masterfrom
itamarst:292-controlling-thread-part-2
Open

itamarst wants to merge 16 commits into
joblib:masterfrom
itamarst:292-controlling-thread-part-2

Conversation

@itamarst

@itamarst itamarst commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #292
Fixes #293

@itamarst itamarst closed this Nov 2, 2026
@itamarst itamarst changed the title DOC controlling thread tweaks WIP controlling thread tweaks, doc improvements, test coverage improvement, and fix for recursive wrap Nov 2, 2026
@itamarst itamarst changed the title WIP controlling thread tweaks, doc improvements, test coverage improvement, and fix for recursive wrap FIX recursive wrap, plus controlling thread cleanups and doc improvements Nov 3, 2026
@itamarst itamarst reopened this Nov 3, 2026
@itamarst
itamarst marked this pull request as ready for review November 3, 2026 16:37

@ogrisel ogrisel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes. Overall LGTM besides the following minor suggestions.

Comment thread README.md Resolved
Comment thread threadpoolctl.py
total_original_info = self._original_info.stack.pop()
for lib_controller, original_info in zip(
self._controller.lib_controllers, total_original_info
):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes that if the decorated function triggers the import of a new controller (e.g. by importing an openmp extension or a BLAS linked package and then threadpool_info afterwards), it will be appended to the existing self._controller.lib_controllers and zipping with a shorter total_original_info list should automatically discard the newly imported trailing controllers. I think this should be fine but I think we need an inline comment to make that assumption explicit and maybe even a non-regression test for that particular case if none of the existing tests already cover that particular edge case.

assert lib.get_num_threads() == 1

recurse(1)
assert lib.get_num_threads() == orig_limit

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could assert that no controlling thread is registered at this point.


@wrap(limits=1, user_api="blas")
def recurse(depth):
assert lib.get_num_threads() == 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also assert that the current thread is in control here.


@threadpool_limits.wrap(limits=1, user_api="blas")
def recurse(depth):
assert lib.get_num_threads() == 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment about asserting that the current thread is in control.

assert lib.get_num_threads() == 1
if depth:
recurse(depth - 1)
assert lib.get_num_threads() == 1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could even spawn and then join a worker thread here to check that it cannot take control.

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recursive @threadpool_limits.wrap() calls do not behave as expected when recursing Follow-up fixes for OpenBLAS-pthreads controlling thread feature

2 participants