Repository navigation
ci: Run the sunspot and sunspot_rails specs on Solr 9.10.1 - #6
Merged
Merged
Conversation
ActiveSupport passes quirks_mode to JSON.generate and JSON.parse. json 3 made unknown options an ArgumentError, so Sunspot.remove_all! in spec_helper blows up and every sunspot_rails spec fails with it. Rails 8 dropped quirks_mode, but the Appraisals file stops at Rails 7.0, so nothing in the matrix can run on json 3. Pinning it as a development dependency, next to the psych pin that is already there for the same kind of reason.
…type on Solr 7 and later
Root cause: `Sunspot::Query::Join#to_subquery` built `_query_:"{!field f=type}Photo"+_query_:"{!edismax …}keywords"`, marking only the keywords clause required. Solr 5's schema `defaultOperator="AND"` made the type clause required too, but Solr 7 removed that setting and the default operator is OR, so the type clause became optional. A join then matched a container through a document of any type with matching keywords: on Solr 9.10.1 `spec/integration/join_spec.rb` finds a `PhotoContainer` by a `Picture`'s description.
The type clause is now prefixed with `+` as well. Both join specs pass on Solr 6.6.6 and 9.10.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAYgm5SknHTiRNTgGA6LBY
Issue: CI ran the specs on the Solr 5.3.1 that `sunspot_solr` bundles, while the fork's users run Solr 9.10.1 in production. Behaviour that only newer Solr has, such as the `{!bool}` parser, couldn't be tested, and behaviour newer Solr changed went unnoticed.
With `SOLR_IMAGE` set, `ci/sunspot_test_script.sh` runs that Docker image with the configset in `ci/solr9`, and the workflow sets it to `solr:9.10.1` for the `sunspot` and `sunspot_rails` jobs. `sunspot_solr`'s own specs still test the Solr it bundles.
`ci/solr9` is `sunspot_solr`'s configset ported to Solr 9: point fields with docValues in place of trie fields, `LatLonPointSpatialField` in place of `LatLonType`, and the schema elements Solr 9 removed (`StandardFilterFactory`, `defaultSearchField`, `solrQueryParser`) taken out. In `solrconfig.xml`, `luceneMatchVersion` is 9.0.0, and `/select` defaults to `mm=100%`, which edismax defaulted to before Solr 7, and to the original highlighter, which Sunspot's `hl.simple.*` parameters are for.
Four integration specs depend on behaviour newer Solr changed, and `skip_from_solr` skips them there: index-time document boosts (removed in Solr 7), `facet.date` (removed by Solr 6), relative scores of term and range boost queries (BM25, the default similarity from `luceneMatchVersion` 6; the spec passes under `ClassicSimilarity`), and one ranking of boosted fulltext, which fails on Solr 6.6.6 and 9.10.1 under either similarity for a reason not yet found.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAYgm5SknHTiRNTgGA6LBY
…e can't be created Root cause: `start_solr_container` runs under `set +e` and waited for Solr with an unbounded `until curl` loop. When `docker run` failed, for example on an image pull error or a Docker Hub rate limit, the job spun until the workflow's 60-minute timeout. When copying the configset or creating the core failed, it still printed "done.", and the run showed up only as every spec failing. `docker run` failing now exits the script, the wait gives up after 120 tries and prints the container's log, and a failed `mkdir`, `docker cp`, `chown` or core `CREATE` prints the log and exits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAYgm5SknHTiRNTgGA6LBY
The quirks_mode incompatibility depends on the Rails version, so pin json next to the other version-specific gems instead of in the gemspec, and correct the comment: 8.0.x still passes quirks_mode; only 7.2.4+ and 8.1+ don't.
Pin `json` below 3 for the sunspot_rails suite
Root cause: Sunspot sends a field's `:boost` with its value, and with the JSON update format RSolr writes it as `"title_text":{"boost":2,"value":"pizza"}`. Solr 7 removed index-time boosts, and Solr 9 reads that object as a field named `boost`, rejecting the whole update with `400 unknown field 'boost'`. XML updates carry the boost as an attribute, which Solr 9 ignores, so only the `sunspot, json` CI jobs on Solr 9.10.1 failed: 180 integration examples.
`Sunspot.config.solr.index_time_boosts`, and `index_time_boosts` under `solr:` in `sunspot.yml`, defaults to true, which keeps what Sunspot sends unchanged for everyone. Set to false, the indexer removes field and document boosts from every document it sends, child documents included. The integration specs set it from the Solr they run against: false on Solr 7 and later. The README explains the setting, and gains an "Upgrading to Solr 9" section listing what changes for Sunspot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TAYgm5SknHTiRNTgGA6LBY
…s well as 9.10.1 Issue: the `sunspot` and `sunspot_rails` jobs ran only on Solr 9.10.1, so nothing tested the gem on the Solr 5.3.1 that `sunspot_solr` bundles, which upstream's CI tests against. Instead of choosing the Solr by gem, the matrix has a `solr` dimension, `bundled` and `9.10.1`, and `SOLR_IMAGE` is set from it. `sunspot_solr` runs only on `bundled`, since its specs test the Solr it bundles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAYgm5SknHTiRNTgGA6LBY
ContextBoundDelegate tries the receiver first and falls back to the calling context in a rescue, so every DSL call that belongs to the calling context raises a NoMethodError whose backtrace is then thrown away. Raising captures the whole stack at roughly 3 objects per frame, and a Rails request is already ~182 frames deep inside a search block, so each raise costs ~600 objects. In our app a block calling a few model helpers raised 70 times per request; dropping those took the request from 288,801 to 246,807 objects allocated, 14.6% less. The fast path uses respond_to? to dispatch straight to the calling context. Where respond_to? can't answer, method_missing without respond_to_missing? being the obvious case, it falls through to the old path.
The respond_to? fast path let the calling context win over receivers
that handle calls in method_missing without respond_to_missing?, such
as DSL::Function (function { sum(...) } in a model class method) and
DSL::Fields (double inside an RSpec example). It also called
respond_to? on BasicObject receivers that don't have it.
Only take the fast path when the receiver has Kernel, doesn't respond
to the method and uses the default method_missing, so it could only
have raised. Add specs for each case, and only count raises on the
test thread.
njakobsen
marked this pull request as ready for review
October 21, 2026 07:06
…or-in-dsl-dispatch Avoid NoMethodError when resolving DSL calls
…g it after the first index takes effect Root cause: `Session#indexer` copied `config.solr.index_time_boosts` into the `Indexer` when it built it, on the first index. Setting it to false after that, as an app that indexes during boot before an initializer runs would, left boosts on for the life of the session (found in review). Instead of copying the value, the indexer now keeps the session's config and reads the setting in `add_documents`. New API specs cover turning the setting off after indexing, a `batch`, and an atomic update, which sends a field's boost like a full update. All three, and the existing two, fail without the stripping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAYgm5SknHTiRNTgGA6LBY
…skip the field boost spec from Solr 7 The README, the setting's rdoc and the spec helper said Solr 9 rejects a JSON update carrying any index-time boost. On Solr 9.10.1 a document boost in a JSON update is accepted and ignored; only a field boost gets `400 unknown field 'boost'` (found in review). The "with field boost" integration spec kept passing on Solr 9, where no field boost is sent, so it no longer tested the boost there. It's now skipped from Solr 7, as the document boost spec beside it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAYgm5SknHTiRNTgGA6LBY
Brings in sunspot#1057, which avoids a `NoMethodError` when the DSL resolves a call against a receiver, and sunspot#1058, which pins `json` below 3 in the `sunspot_rails` appraisals, ahead of an upstream gem release. Upstream's json pin supersedes the fork's own from #5, which pinned only the Rails 6.1 and 7.0 appraisals, so the fork's line is dropped and `sunspot_rails/Appraisals` now equals upstream's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAYgm5SknHTiRNTgGA6LBY
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runs the
sunspotandsunspot_railsspecs on Solr 9.10.1, the version the fork's users run in production, as well as on the Solr 5.3.1 thatsunspot_solrbundles.sunspot_solr's own specs run only on the bundled Solr.ci/solr9issunspot_solr's configset ported to Solr 9: point fields with docValues,LatLonPointSpatialField, the removed schema elements taken out,luceneMatchVersion9.0.0, and/selectdefaulting tomm=100%and the original highlighter.solrmatrix dimension,bundledand9.10.1, setsSOLR_IMAGE, which makesci/sunspot_test_script.shrun that Docker image with the configset. That doubles thesunspotandsunspot_railsjobs. The script exits, with the container's log, if the container doesn't start within 120 seconds or its core can't be created.index_time_boostssetting: Solr 7 removed index-time boosts, and Solr 9 rejects a JSON update that carries one (400 unknown field 'boost'), which failed 180 examples in everysunspot, jsonjob.Sunspot.config.solr.index_time_boosts, orindex_time_boostsundersolr:insunspot.yml, defaults to true, so nothing changes unless it's set. Set to false, Sunspot sends no field or document boosts, child documents included, inindex,batchand atomic updates. It's read on every update, so changing it after the first index takes effect. The integration specs set it from the Solr they run against.defaultOperator, so a join's type clause became optional and a join matched a container through a document of any type. The type clause is now required (+).skip_from_solr: index-time document and field boosts (removed in 7),facet.date(removed by 6), BM25 relative scores, and one boosted fulltext ranking (scoped_search_spec.rb:596) that fails on Solr 6.6.6 and 9.10.1 for a reason not yet found.mm, BM25 scores, time range facets, and_root_for nested documents.Testing
Locally, in both update formats:
sunspotintegrationsunspot_rails(Rails 7.0)ci/solr9)These ran before the review fixes (
2f879292,16e91114) and the upstream sync. Since then, on #7, which contains all of this branch, thesunspotintegration specs pass on both Solrs in both formats (the field boost example is now pending on 9.10.1), andsunspot_railspasses on both.Without the indexer change, the JSON run on Solr 9.10.1 fails 171 examples with
unknown field 'boost'. The new API specs fail without it too. The API specs have the same 12 local-only failures asmaster.Merge order
This has #8 (upstream
master) merged in. Merge that first, then this with a merge commit, not a squash, so #7, stacked on this, retargets cleanly.🤖 Generated with Claude Code
https://claude.ai/code/session_01TAYgm5SknHTiRNTgGA6LBY