Skip to content

FEAT: Improve interactions of user pools - #1009

Open
ColmTalbot wants to merge 21 commits into
bilby-dev:mainfrom
ColmTalbot:user-pools
Open

FEAT: Improve interactions of user pools#1009
ColmTalbot wants to merge 21 commits into
bilby-dev:mainfrom
ColmTalbot:user-pools

Conversation

@ColmTalbot

Copy link
Copy Markdown
Collaborator

I noticed that it was difficult to pass a user-specified pool through run_sampler, and it isn't used at all in the post processing. This PR:

  • unifies how to create a pool for Bilby with a context (a la with multiprocessing.pool()...)
  • explicitly passes the user pool around including for post processing
  • moves logic out of Sampler._setup_pool and Sampler._close_pool
  • support MPI pools using schwimmbad. This change will effectively make parallel_bilby moot.

@ColmTalbot
ColmTalbot marked this pull request as draft October 30, 2025 14:22
@ColmTalbot
ColmTalbot marked this pull request as ready for review November 3, 2025 16:03
@ColmTalbot ColmTalbot added >100 lines refactoring sampling Issues about sampling algorithms, their efficiency and robustness labels Nov 4, 2025

@mj-will mj-will left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, I look forward to trying it! I've added some initial comments but might need to have a second look.

Comment thread bilby/core/sampler/__init__.py
Comment thread bilby/core/sampler/__init__.py
Comment thread bilby/core/sampler/ptemcee.py Outdated
Comment thread bilby/core/utils/parallel.py
Comment thread bilby/core/utils/parallel.py
Comment thread bilby/core/result.py Outdated
Comment thread bilby/gw/conversion.py Outdated
Comment thread bilby/gw/conversion.py Outdated

@ColmTalbot ColmTalbot left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember exactly what the behaviour for some of these questions is, so I'll go back and check, and ideally write docstrings about them.

Comment thread bilby/core/sampler/ptemcee.py Outdated
Comment thread bilby/core/sampler/__init__.py
Comment thread bilby/core/sampler/__init__.py
@ColmTalbot
ColmTalbot requested a review from mj-will January 26, 2026 16:30
@ColmTalbot ColmTalbot added this to the 3.0.0 milestone Jan 29, 2026
@mj-will
mj-will requested a review from a team April 30, 2026 14:43

@mj-will mj-will left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot to submit my comments as a review, but see them above.

@mj-will
mj-will requested a review from a team April 30, 2026 14:48
Comment thread bilby/core/utils/parallel.py
@mj-will
mj-will requested a review from a team April 30, 2026 16:03
@mj-will mj-will added the to discuss To be discussed on an upcoming call label Apr 30, 2026

@adivijaykumar adivijaykumar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test for schwimmbad/MPI support?

@mj-will

mj-will commented May 14, 2026

Copy link
Copy Markdown
Collaborator

We should documentation for user pools and note that the pool will only work correctly in the initializer has been called.

Comment thread bilby/core/utils/parallel.py Outdated
Comment thread bilby/core/sampler/__init__.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

refactoring sampling Issues about sampling algorithms, their efficiency and robustness to discuss To be discussed on an upcoming call >100 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants