CI: Use precompiled Director release - #341
Draft
neddp wants to merge 15 commits into
Draft
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
neddp
changed the title
cf-foundation-community-automation
Bot
moved this to Inbox
in Foundational Infrastructure Working Group
neddp
force-pushed
the
speed-up-bats-director
branch
2 times, most recently
from
August 28, 2026 14:14
288a7f6 to
6e96d5b
Compare
The BATS director previously deployed on a jammy stemcell with the bosh and bpm releases pulled from source (via misc/source-releases/bosh.yml plus a file:// override), so create-env recompiled every director package on the VM (~25 min). Let the director use bosh-deployment's compiled bosh/bpm releases instead. These are built for noble, and bosh-deployment/openstack/cpi.yml already pins a matching noble stemcell at /resource_pools/name=vms/ stemcell, kept in lockstep with the compiled releases by bosh-deployment CI. So: - Drop the source-releases/bosh.yml overlay and the file:// bosh override; bosh and bpm now resolve to the compiled noble URLs in bosh-deployment/bosh.yml. - Narrow deployment-configuration.yml to patch only cloud_properties, letting cpi.yml's noble stemcell stand as the director host OS. - Drop the now-unused stemcell-director input/get, the source-release fallback (prepare_bosh_release / find_bosh_compiled_release, which queried a bucket that no longer exists), and the bosh-release input. The stemcell under test (BATS/lifecycle) stays jammy. The director host OS tracks bosh-deployment's compiled release automatically, so there is no stemcell version to pin here. Co-authored-by: Saumya Dudeja <saumya.dudeja@sap.com>
bosh-deployment/openstack/cpi.yml wires the Ruby openstack_cpi job, whose openstack-ruby-3.1 package compiles at create-env time (~9 min on the worker plus ~5 min on the director VM). Add use-golang-cpi.yml, applied after cpi.yml, to point the director's cpi_job, cloud_provider template, and bosh instance-group job at openstack_cpi_golang instead. The Go job takes the same openstack.* properties and only needs the golang-1-linux package, so create-env no longer builds the Ruby runtime. BATS now exercises the Go CPI end-to-end against DevStack. Co-authored-by: Saumya Dudeja <saumya.dudeja@sap.com>
The bosh-release resource_type (dpb587/bosh-release-resource) is not
referenced by any resource, and teardown-director.{yml,sh} is not
referenced by the pipeline. Remove them.
Co-authored-by: Saumya Dudeja <saumya.dudeja@sap.com>
Co-authored-by: Saumya Dudeja <saumya.dudeja@sap.com>
Co-authored-by: Saumya Dudeja <saumya.dudeja@sap.com>
Co-authored-by: Saumya Dudeja <saumya.dudeja@sap.com>
… exists Co-authored-by: Saumya Dudeja <saumya.dudeja@sap.com>
Co-authored-by: Saumya Dudeja <saumya.dudeja@sap.com>
neddp
force-pushed
the
speed-up-bats-director
branch
from
August 30, 2026 07:14
7600cac to
31d64f8
Compare
The integration specs ran the CPI at LevelDebug and logged every mock request to stderr, producing thousands of lines of noise per run (and especially under ginkgo -v). Drop the CPI logger to LevelError and remove the per-request mock logging; failures still surface via the response JSON and assertion diffs. Co-authored-by: Saumya Dudeja <saumya.dudeja@sap.com>
Build-tag-gated (lifecycle) harness that drives cpi.Execute against a real OpenStack from BOSH_OPENSTACK_* env: config loader, execCPI (stdin/stdout), typed method wrappers with best-effort cleanup, and stemcell.MF parsing.
Covers CPI-method-level behaviors the mocked integration suite and BATS do not exercise against real OpenStack: stemcell Glance props + light stemcell, Cinder volume types + set_disk_metadata/snapshot/resize, calculate_vm_cloud_ properties, config_drive multi-NIC, VRRP/allowed_address_pairs, security groups by name/id, pre-existing port, use_nova_networking, boot-from-volume variants, human-readable names, and error paths.
Extracts terraform metadata + auth into BOSH_OPENSTACK_* env and runs the lifecycle-tagged Go suite via ginkgo. Wired into run-devstack-tests via the manual-apply pipeline.
The multiple-manual-network precondition errored when config_drive was set (ConfigDrive != "") — the inverse of the Ruby CPI (!config_drive) and of the error message itself, which requires config_drive to be present. Flip to ConfigDrive == "". Correct the mocked integration test that had encoded the inverted behavior and the unit fixtures that relied on it. Found by the real-OpenStack lifecycle suite (multi-NIC + config_drive).
- Light stemcell returns the resolved image id (no ' light' suffix, unlike the Ruby CPI); assert Equal instead of a substring. - preCreatePort now looks up the subnet id (ports.IP.SubnetID has no omitempty, so an empty value was rejected by Neutron).
VMs using the project 'default' security group deterministically fail to boot on the CI DevStack (nova ERROR state), while the specs that pass an explicit terraform-created group boot fine — matching how BATS deploys. Default to BOSH_OPENSTACK_SECURITY_GROUP_NAME when available.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.