diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index a5d70ee55..0d522643d 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -38,6 +38,12 @@ jobs:
- 'head'
gem: ['sunspot', 'sunspot_rails', 'sunspot_solr']
update-format: ['xml', 'json']
+ # The Solr that sunspot_solr bundles, and the Solr Culture Code runs in production
+ solr: ['bundled', '9.10.1']
+ exclude:
+ # sunspot_solr's own specs test the Solr it bundles
+ - gem: sunspot_solr
+ solr: '9.10.1'
steps:
- uses: actions/checkout@v4
- name: Set up JDK 1.8
@@ -56,3 +62,4 @@ jobs:
env:
GEM: ${{ matrix.gem }}
UPDATE_FORMAT: ${{ matrix.update-format }}
+ SOLR_IMAGE: ${{ matrix.solr != 'bundled' && format('solr:{0}', matrix.solr) || '' }}
diff --git a/README.md b/README.md
index 010ed5057..883c5908f 100644
--- a/README.md
+++ b/README.md
@@ -1507,6 +1507,8 @@ class Post < ActiveRecord::Base
end
```
+Solr 7 removed index-time boosts, for fields and documents alike. Solr 9 ignores them in XML updates and a document boost in JSON updates, but rejects a JSON update that carries a field boost with `unknown field 'boost'`. On Solr 7 and later, set `index_time_boosts` to false so Sunspot leaves them out, and boost at query time instead: `:default_boost` on the field, or `boost_fields` in the search. See [Upgrading to Solr 9](#upgrading-to-solr-9).
+
### Stored Fields
Stored fields keep an original (untokenized/unanalyzed) version of their
@@ -1699,6 +1701,8 @@ test:
log_level: WARNING
```
+On Solr 7 and later, add `index_time_boosts: false` under `solr:`, as [Index-Time Boosts](#index-time-boosts) explains. Without Rails, set `Sunspot.config.solr.index_time_boosts = false`.
+
You may want to use SSL for production environments with a username and password. For example, set `SOLR_URL` to `https://username:password@production.solr.example.com/solr`.
You can examine the value of `Sunspot::Rails.configuration` at runtime.
@@ -1743,6 +1747,17 @@ solr:
where the `./solr/init` directory contains a shell script that does any initial setup like downloading and unzipping your cores.
In both cases, the solr images by default expects cores to be placed in `/opt/solr/server/solr/mycores`.
+### Upgrading to Solr 9
+
+Sunspot's specs pass on Solr 9.10.1 with the configset in `ci/solr9`. Coming from the Solr that `sunspot_solr` bundles, these are the changes that affect Sunspot:
+
+* **The schema.** Solr 9 won't load the bundled schema. `ci/solr9` is a port of it: point fields with docValues in place of trie fields, `LatLonPointSpatialField` in place of `LatLonType`, and `StandardFilterFactory`, `defaultSearchField` and `solrQueryParser` taken out. Use it as a starting point for your core, and reindex everything after switching, since the field types change.
+* **Index-time boosts.** Solr 7 removed them, and Solr 9 rejects a JSON update that carries a field boost. Set `index_time_boosts` to false ([Index-Time Boosts](#index-time-boosts)).
+* **Matching every keyword.** Since Solr 7, edismax no longer requires every term by default. `ci/solr9/conf/solrconfig.xml` sets `mm=100%` on `/select`, so a search for several keywords still matches only documents that have them all, as it did before.
+* **Scores.** BM25 has been the default similarity since `luceneMatchVersion` 6, so relative scores differ from the bundled Solr's, and results with close scores can change order.
+* **Time range facets.** `facet :published_at, :time_range => …` sends `facet.date`, which Solr 6 removed. Use a [query facet](#query-facets) with a row for each range instead.
+* **Nested documents.** A core needs a `_root_` field for them ([Adding `_root_` to an existing core](#adding-_root_-to-an-existing-core)).
+
### Adding `_root_` to an existing core
The bundled configset and `examples/solr7_core` define a `_root_` field, which [nested documents](#nested-documents-block-joins) need. If you bring an existing core's schema up to date with them on Solr 8 or above, the change affects every class, nested or not. Once the schema has `_root_`, Solr replaces and deletes documents by their `_root_` value, and documents indexed before the field was added have none. Reindexing one of them adds a new copy beside the old one, later reindexes replace only the new copy, and removing it by id leaves the old copy in place. The old copies stay until they're deleted, so clear and reindex every class (`rake sunspot:reindex`, which deletes each class's documents first) right after adding the field.
diff --git a/ci/solr9/conf/_rest_managed.json b/ci/solr9/conf/_rest_managed.json
new file mode 100644
index 000000000..6a4aec39e
--- /dev/null
+++ b/ci/solr9/conf/_rest_managed.json
@@ -0,0 +1 @@
+{"initArgs":{},"managedList":[]}
diff --git a/ci/solr9/conf/admin-extra.html b/ci/solr9/conf/admin-extra.html
new file mode 100644
index 000000000..aa739da86
--- /dev/null
+++ b/ci/solr9/conf/admin-extra.html
@@ -0,0 +1,31 @@
+
+
+
diff --git a/ci/solr9/conf/currency.xml b/ci/solr9/conf/currency.xml
new file mode 100644
index 000000000..3a9c58afe
--- /dev/null
+++ b/ci/solr9/conf/currency.xml
@@ -0,0 +1,67 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/ci/solr9/conf/elevate.xml b/ci/solr9/conf/elevate.xml
new file mode 100644
index 000000000..7630ebe20
--- /dev/null
+++ b/ci/solr9/conf/elevate.xml
@@ -0,0 +1,36 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/ci/solr9/conf/lang/stopwords_en.txt b/ci/solr9/conf/lang/stopwords_en.txt
new file mode 100644
index 000000000..2c164c0b2
--- /dev/null
+++ b/ci/solr9/conf/lang/stopwords_en.txt
@@ -0,0 +1,54 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# a couple of test stopwords to test that the words are really being
+# configured from this file:
+stopworda
+stopwordb
+
+# Standard english stop words taken from Lucene's StopAnalyzer
+a
+an
+and
+are
+as
+at
+be
+but
+by
+for
+if
+in
+into
+is
+it
+no
+not
+of
+on
+or
+such
+that
+the
+their
+then
+there
+these
+they
+this
+to
+was
+will
+with
diff --git a/ci/solr9/conf/mapping-ISOLatin1Accent.txt b/ci/solr9/conf/mapping-ISOLatin1Accent.txt
new file mode 100644
index 000000000..ede774258
--- /dev/null
+++ b/ci/solr9/conf/mapping-ISOLatin1Accent.txt
@@ -0,0 +1,246 @@
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syntax:
+# "source" => "target"
+# "source".length() > 0 (source cannot be empty.)
+# "target".length() >= 0 (target can be empty.)
+
+# example:
+# "À" => "A"
+# "\u00C0" => "A"
+# "\u00C0" => "\u0041"
+# "ß" => "ss"
+# "\t" => " "
+# "\n" => ""
+
+# À => A
+"\u00C0" => "A"
+
+# Á => A
+"\u00C1" => "A"
+
+# Â => A
+"\u00C2" => "A"
+
+# Ã => A
+"\u00C3" => "A"
+
+# Ä => A
+"\u00C4" => "A"
+
+# Å => A
+"\u00C5" => "A"
+
+# Æ => AE
+"\u00C6" => "AE"
+
+# Ç => C
+"\u00C7" => "C"
+
+# È => E
+"\u00C8" => "E"
+
+# É => E
+"\u00C9" => "E"
+
+# Ê => E
+"\u00CA" => "E"
+
+# Ë => E
+"\u00CB" => "E"
+
+# Ì => I
+"\u00CC" => "I"
+
+# Í => I
+"\u00CD" => "I"
+
+# Î => I
+"\u00CE" => "I"
+
+# Ï => I
+"\u00CF" => "I"
+
+# IJ => IJ
+"\u0132" => "IJ"
+
+# Ð => D
+"\u00D0" => "D"
+
+# Ñ => N
+"\u00D1" => "N"
+
+# Ò => O
+"\u00D2" => "O"
+
+# Ó => O
+"\u00D3" => "O"
+
+# Ô => O
+"\u00D4" => "O"
+
+# Õ => O
+"\u00D5" => "O"
+
+# Ö => O
+"\u00D6" => "O"
+
+# Ø => O
+"\u00D8" => "O"
+
+# Π=> OE
+"\u0152" => "OE"
+
+# Þ
+"\u00DE" => "TH"
+
+# Ù => U
+"\u00D9" => "U"
+
+# Ú => U
+"\u00DA" => "U"
+
+# Û => U
+"\u00DB" => "U"
+
+# Ü => U
+"\u00DC" => "U"
+
+# Ý => Y
+"\u00DD" => "Y"
+
+# Ÿ => Y
+"\u0178" => "Y"
+
+# à => a
+"\u00E0" => "a"
+
+# á => a
+"\u00E1" => "a"
+
+# â => a
+"\u00E2" => "a"
+
+# ã => a
+"\u00E3" => "a"
+
+# ä => a
+"\u00E4" => "a"
+
+# å => a
+"\u00E5" => "a"
+
+# æ => ae
+"\u00E6" => "ae"
+
+# ç => c
+"\u00E7" => "c"
+
+# è => e
+"\u00E8" => "e"
+
+# é => e
+"\u00E9" => "e"
+
+# ê => e
+"\u00EA" => "e"
+
+# ë => e
+"\u00EB" => "e"
+
+# ì => i
+"\u00EC" => "i"
+
+# í => i
+"\u00ED" => "i"
+
+# î => i
+"\u00EE" => "i"
+
+# ï => i
+"\u00EF" => "i"
+
+# ij => ij
+"\u0133" => "ij"
+
+# ð => d
+"\u00F0" => "d"
+
+# ñ => n
+"\u00F1" => "n"
+
+# ò => o
+"\u00F2" => "o"
+
+# ó => o
+"\u00F3" => "o"
+
+# ô => o
+"\u00F4" => "o"
+
+# õ => o
+"\u00F5" => "o"
+
+# ö => o
+"\u00F6" => "o"
+
+# ø => o
+"\u00F8" => "o"
+
+# œ => oe
+"\u0153" => "oe"
+
+# ß => ss
+"\u00DF" => "ss"
+
+# þ => th
+"\u00FE" => "th"
+
+# ù => u
+"\u00F9" => "u"
+
+# ú => u
+"\u00FA" => "u"
+
+# û => u
+"\u00FB" => "u"
+
+# ü => u
+"\u00FC" => "u"
+
+# ý => y
+"\u00FD" => "y"
+
+# ÿ => y
+"\u00FF" => "y"
+
+# ff => ff
+"\uFB00" => "ff"
+
+# fi => fi
+"\uFB01" => "fi"
+
+# fl => fl
+"\uFB02" => "fl"
+
+# ffi => ffi
+"\uFB03" => "ffi"
+
+# ffl => ffl
+"\uFB04" => "ffl"
+
+# ſt => ft
+"\uFB05" => "ft"
+
+# st => st
+"\uFB06" => "st"
diff --git a/ci/solr9/conf/protwords.txt b/ci/solr9/conf/protwords.txt
new file mode 100644
index 000000000..1dfc0abec
--- /dev/null
+++ b/ci/solr9/conf/protwords.txt
@@ -0,0 +1,21 @@
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#-----------------------------------------------------------------------
+# Use a protected word file to protect against the stemmer reducing two
+# unrelated words to the same base word.
+
+# Some non-words that normally won't be encountered,
+# just to test that they won't be stemmed.
+dontstems
+zwhacky
+
diff --git a/ci/solr9/conf/schema.xml b/ci/solr9/conf/schema.xml
new file mode 100644
index 000000000..82c560dd1
--- /dev/null
+++ b/ci/solr9/conf/schema.xml
@@ -0,0 +1,277 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ id
+
+
+
+
+
+
+
+
diff --git a/ci/solr9/conf/scripts.conf b/ci/solr9/conf/scripts.conf
new file mode 100644
index 000000000..f58b262ae
--- /dev/null
+++ b/ci/solr9/conf/scripts.conf
@@ -0,0 +1,24 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+user=
+solr_hostname=localhost
+solr_port=8983
+rsyncd_port=18983
+data_dir=
+webapp_name=solr
+master_host=
+master_data_dir=
+master_status_dir=
diff --git a/ci/solr9/conf/solrconfig.xml b/ci/solr9/conf/solrconfig.xml
new file mode 100644
index 000000000..2463b5d57
--- /dev/null
+++ b/ci/solr9/conf/solrconfig.xml
@@ -0,0 +1,633 @@
+
+
+
+
+
+
+
+
+ 9.0.0
+
+
+ ${solr.data.dir:}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ${solr.lock.type:native}
+
+
+ true
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ ${solr.ulog.dir:}
+
+
+
+
+ ${solr.autoCommit.maxTime:15000}
+ false
+
+
+
+
+ ${solr.autoSoftCommit.maxTime:-1}
+
+
+
+
+
+
+
+ 1024
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ true
+
+
+ 20
+
+
+ 200
+
+
+ false
+
+
+ 2
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ original
+ 100%
+ default
+ true
+ true
+
+
+ spellcheck
+
+
+
+
+
+
+ explicit
+ json
+ true
+ text
+
+
+ spellcheck
+
+
+
+
+
+
+ {!xport}
+ xsort
+ false
+
+
+
+ query
+
+
+
+
+
+
+ text
+
+
+
+
+
+
+
+
+
+
+
+
+
+ explicit
+ true
+
+
+
+
+
+ textSpell
+
+
+
+ default
+
+ textSpell
+ true
+
+
+
+ example
+ title_text
+ true
+ solr.DirectSolrSpellChecker
+
+ internal
+
+ 0.5
+
+ 2
+
+ 1
+
+ 5
+
+ 4
+
+ 0.01
+
+
+
+
+
+ wordbreak
+ solr.WordBreakSolrSpellChecker
+ name
+ true
+ true
+ 10
+
+ -->
+
+
+
+
+
+
+
+
+
+
+ true
+ false
+
+
+ terms
+
+
+
+
+
+ 1
+ 2
+
+
+
+
+
+ *:*
+
+
+
diff --git a/ci/solr9/conf/spellings.txt b/ci/solr9/conf/spellings.txt
new file mode 100644
index 000000000..d7ede6f56
--- /dev/null
+++ b/ci/solr9/conf/spellings.txt
@@ -0,0 +1,2 @@
+pizza
+history
\ No newline at end of file
diff --git a/ci/solr9/conf/synonyms.txt b/ci/solr9/conf/synonyms.txt
new file mode 100644
index 000000000..7f7212830
--- /dev/null
+++ b/ci/solr9/conf/synonyms.txt
@@ -0,0 +1,29 @@
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#-----------------------------------------------------------------------
+#some test synonym mappings unlikely to appear in real input text
+aaafoo => aaabar
+bbbfoo => bbbfoo bbbbar
+cccfoo => cccbar cccbaz
+fooaaa,baraaa,bazaaa
+
+# Some synonym groups specific to this example
+GB,gib,gigabyte,gigabytes
+MB,mib,megabyte,megabytes
+Television, Televisions, TV, TVs
+#notice we use "gib" instead of "GiB" so any WordDelimiterFilter coming
+#after us won't split it into two words.
+
+# Synonym mappings can be used for spelling correction too
+pixima => pixma
+
diff --git a/ci/sunspot_test_script.sh b/ci/sunspot_test_script.sh
index 0875d6224..3710ff0ae 100755
--- a/ci/sunspot_test_script.sh
+++ b/ci/sunspot_test_script.sh
@@ -9,7 +9,36 @@ solr_responding() {
curl -o /dev/null "http://localhost:$SOLR_PORT/solr/admin/ping" > /dev/null 2>&1
}
+# With SOLR_IMAGE set, runs that Docker image of Solr with the configset in ci/solr9, in place of the
+# Solr that sunspot_solr bundles.
+start_solr_container() {
+ /bin/echo -n "Starting $SOLR_IMAGE on port $SOLR_PORT for Sunspot specs..."
+ docker run -d --name sunspot-solr -p $SOLR_PORT:8983 $SOLR_IMAGE > /dev/null || exit 1
+ tries=0
+ until curl -s -o /dev/null "http://localhost:$SOLR_PORT/solr/admin/cores"; do
+ tries=$((tries + 1))
+ if [ $tries -gt 120 ]; then
+ /bin/echo " Solr didn't start."
+ docker logs sunspot-solr
+ exit 1
+ fi
+ /bin/echo -n "."
+ sleep 1
+ done
+ docker exec --user root sunspot-solr mkdir -p /var/solr/data/configsets &&
+ docker cp "$SUNSPOT_LIB_HOME/ci/solr9" sunspot-solr:/var/solr/data/configsets/sunspot &&
+ docker exec --user root sunspot-solr chown -R solr:solr /var/solr/data/configsets &&
+ curl -s -S -f -X POST "http://localhost:$SOLR_PORT/solr/admin/cores?action=CREATE&name=default&configSet=sunspot" > /dev/null ||
+ { /bin/echo " Couldn't create the default core."; docker logs sunspot-solr; exit 1; }
+ /bin/echo "done."
+}
+
start_solr_server() {
+ if [ -n "$SOLR_IMAGE" ]; then
+ start_solr_container
+ return
+ fi
+
# go to sunspot_solr folder and install dependencies
current_path=`pwd`
cd ../sunspot_solr
@@ -34,6 +63,11 @@ start_solr_server() {
}
stop_solr_server() {
+ if [ -n "$SOLR_IMAGE" ]; then
+ docker rm -f sunspot-solr > /dev/null
+ return
+ fi
+
cd ../sunspot_solr
/bin/echo -n "Stopping Solr... "
bundle exec sunspot-solr stop -p $SOLR_PORT
diff --git a/sunspot/lib/sunspot/configuration.rb b/sunspot/lib/sunspot/configuration.rb
index b9b7166f3..deecc50ac 100644
--- a/sunspot/lib/sunspot/configuration.rb
+++ b/sunspot/lib/sunspot/configuration.rb
@@ -5,6 +5,11 @@ module Sunspot
# Sunspot.config.solr.url::
# The URL at which to connect to Solr
# (default: 'http://localhost:8983/solr')
+ # Sunspot.config.solr.index_time_boosts::
+ # Whether to send the field and document boosts declared with +:boost+
+ # and +boost+ (default: true). Solr 7 removed index-time boosts, and
+ # Solr 9 rejects a JSON update that carries a field boost, so set it to
+ # false on Solr 7 and later.
# Sunspot.config.pagination.default_per_page::
# Solr always paginates its results. This sets Sunspot's default result
# count per page if it is not explicitly specified in the query.
@@ -27,6 +32,7 @@ def build #:nodoc:
open_timeout nil
proxy nil
update_format :xml
+ index_time_boosts true
end
master_solr do
url nil
diff --git a/sunspot/lib/sunspot/indexer.rb b/sunspot/lib/sunspot/indexer.rb
index e0952ab1e..d2fda8c9c 100644
--- a/sunspot/lib/sunspot/indexer.rb
+++ b/sunspot/lib/sunspot/indexer.rb
@@ -9,8 +9,9 @@ module Sunspot
#
class Indexer #:nodoc:
- def initialize(connection)
+ def initialize(connection, config)
@connection = connection
+ @config = config
end
#
@@ -240,9 +241,19 @@ def prepare_atomic_update(clazz, id, updates = {})
end
def add_documents(documents)
+ documents.each { |document| remove_boosts(document) } if @config.solr.index_time_boosts == false
@connection.add(documents)
end
+ # Removes the boosts from +document+, its fields and its child documents
+ def remove_boosts(document)
+ document.attrs.delete(:boost)
+ document.fields.each do |field|
+ field.attrs.delete(:boost)
+ remove_boosts(field.value) if field.value.respond_to?(:fields)
+ end
+ end
+
def add_batch_documents(documents)
if batcher.batching?
batcher.concat(documents)
diff --git a/sunspot/lib/sunspot/query/join.rb b/sunspot/lib/sunspot/query/join.rb
index e92cfe4ad..00cb0c97b 100644
--- a/sunspot/lib/sunspot/query/join.rb
+++ b/sunspot/lib/sunspot/query/join.rb
@@ -45,7 +45,7 @@ def to_subquery
{
:q => "_query_:\"{!join from=#{@from} to=#{@to} v=$#{q_name}}\"",
- q_name => "_query_:\"{!field f=type}#{@target.name}\"+_query_:\"{!edismax #{options}}#{keywords}\""
+ q_name => "+_query_:\"{!field f=type}#{@target.name}\"+_query_:\"{!edismax #{options}}#{keywords}\""
}
end
diff --git a/sunspot/lib/sunspot/session.rb b/sunspot/lib/sunspot/session.rb
index 846691574..0c87428d3 100644
--- a/sunspot/lib/sunspot/session.rb
+++ b/sunspot/lib/sunspot/session.rb
@@ -264,7 +264,7 @@ def connection
end
def indexer
- @indexer ||= Indexer.new(connection)
+ @indexer ||= Indexer.new(connection, config)
end
def setup_for_types(types)
diff --git a/sunspot/lib/sunspot/util.rb b/sunspot/lib/sunspot/util.rb
index 85ea7d245..e83be0378 100644
--- a/sunspot/lib/sunspot/util.rb
+++ b/sunspot/lib/sunspot/util.rb
@@ -298,20 +298,43 @@ def respond_to_missing?(method, _)
end
def __proxy_method__(method, *args, **kwargs, &block)
- if kwargs.empty?
- @__receiver__.__send__(method.to_sym, *args, &block)
- else
- @__receiver__.__send__(method.to_sym, *args, **kwargs, &block)
+ method = method.to_sym
+
+ # Fast path: skip the rescue below when the receiver can't handle the
+ # method, avoiding a raise on every call into the calling context.
+ if !__may_handle__?(@__receiver__, method) &&
+ ::Kernel === @__calling_context__ &&
+ @__calling_context__.respond_to?(method, true)
+ return __dispatch__(@__calling_context__, method, args, kwargs, &block)
end
- rescue ::NoMethodError => e
+
begin
- if kwargs.empty?
- @__calling_context__.__send__(method.to_sym, *args, &block)
- else
- @__calling_context__.__send__(method.to_sym, *args, **kwargs, &block)
+ __dispatch__(@__receiver__, method, args, kwargs, &block)
+ rescue ::NoMethodError => e
+ begin
+ __dispatch__(@__calling_context__, method, args, kwargs, &block)
+ rescue ::NoMethodError
+ raise(e)
end
- rescue ::NoMethodError
- raise(e)
+ end
+ end
+
+ KERNEL_METHOD = ::Kernel.instance_method(:method)
+
+ # respond_to? only settles it when method_missing is the default one;
+ # receivers like DSL::Function answer everything through method_missing.
+ # Objects without Kernel (BasicObject proxies) may not have respond_to?.
+ def __may_handle__?(target, method)
+ !(::Kernel === target) ||
+ target.respond_to?(method, true) ||
+ KERNEL_METHOD.bind(target).call(:method_missing).owner != ::BasicObject
+ end
+
+ def __dispatch__(target, method, args, kwargs, &block)
+ if kwargs.empty?
+ target.__send__(method, *args, &block)
+ else
+ target.__send__(method, *args, **kwargs, &block)
end
end
end
diff --git a/sunspot/spec/api/binding_spec.rb b/sunspot/spec/api/binding_spec.rb
index 14008252b..5c4467548 100644
--- a/sunspot/spec/api/binding_spec.rb
+++ b/sunspot/spec/api/binding_spec.rb
@@ -49,6 +49,50 @@
expect(value).to eq({ a: 10, b: 20 })
end
+ it 'should build function queries whose names the calling context also defines' do
+ # Like a model class method calling solr_search, where the model responds
+ # to ActiveRecord's #sum.
+ searcher = Class.new do
+ def search(session)
+ session.search(Post) do
+ keywords('pizza') do
+ boost(function { sum(:average_rating, 10) })
+ end
+ end
+ end
+
+ def sum(*)
+ 'from context'
+ end
+ end.new
+
+ searcher.search(session)
+ expect(connection).to have_last_search_including(:bf, 'sum(average_rating_ft,10)')
+ end
+
+ it 'should define typed fields whose names the calling context also defines' do
+ stub_const('BindingSpecDocument', Class.new)
+ # The example group responds to rspec-mocks' #double.
+ Sunspot::Setup.setup(BindingSpecDocument) { double :rating }
+ expect(Sunspot::Setup.for(BindingSpecDocument).field(:rating).type).to be_a(Sunspot::Type::DoubleType)
+ end
+
+ it 'should prefer the record\'s method_missing in index-time blocks' do
+ record_class = Class.new do
+ def method_missing(name, *args, &block)
+ name == :name ? 'from record' : super
+ end
+ # respond_to_missing? is deliberately not defined.
+ end
+ # Inside a searchable block the calling context is the model class, which
+ # responds to Class#name.
+ extractor = record_class.instance_eval do
+ Sunspot::DataExtractor::BlockExtractor.new { name }
+ end
+
+ expect(extractor.value_for(record_class.new)).to eq('from record')
+ end
+
private
def test_method
@@ -63,3 +107,120 @@ def kwargs_method(a:, b:)
{ a: a, b: b }
end
end
+
+describe Sunspot::Util::ContextBoundDelegate do
+ # Resolution must not depend on raising; the discarded backtraces are costly.
+ it 'does not raise NoMethodError when resolving a calling context method' do
+ raised = 0
+ test_thread = Thread.current
+ tracer = TracePoint.new(:raise) do |tp|
+ raised += 1 if Thread.current == test_thread && tp.raised_exception.is_a?(NoMethodError)
+ end
+
+ context = Class.new do
+ def resolve(receiver)
+ Sunspot::Util::ContextBoundDelegate
+ .instance_eval_with_context(receiver) { context_only_method }
+ end
+
+ def context_only_method
+ 'from context'
+ end
+ end.new
+
+ result = nil
+ tracer.enable { result = context.resolve(Object.new) }
+
+ expect(result).to eq('from context')
+ expect(raised).to eq(0)
+ end
+
+ # respond_to? cannot see method_missing, so this must still resolve via the
+ # rescue-based fallback.
+ it 'resolves a calling context method that only exists via method_missing' do
+ context = Class.new do
+ def resolve(receiver)
+ Sunspot::Util::ContextBoundDelegate
+ .instance_eval_with_context(receiver) { ghost_method }
+ end
+
+ def method_missing(name, *args, &block)
+ name == :ghost_method ? 'from method_missing' : super
+ end
+ # respond_to_missing? is deliberately not defined.
+ end.new
+
+ expect(context.resolve(Object.new)).to eq('from method_missing')
+ end
+
+ it 'raises NoMethodError when neither the receiver nor the context responds' do
+ context = Class.new do
+ def resolve(receiver)
+ Sunspot::Util::ContextBoundDelegate
+ .instance_eval_with_context(receiver) { totally_unknown_method }
+ end
+ end.new
+
+ expect { context.resolve(Object.new) }.to raise_error(NoMethodError)
+ end
+
+ it 'prefers the receiver when both define the method' do
+ receiver = Class.new do
+ def shared_method
+ 'from receiver'
+ end
+ end.new
+
+ context = Class.new do
+ def resolve(receiver)
+ Sunspot::Util::ContextBoundDelegate
+ .instance_eval_with_context(receiver) { shared_method }
+ end
+
+ def shared_method
+ 'from context'
+ end
+ end.new
+
+ expect(context.resolve(receiver)).to eq('from receiver')
+ end
+
+ it 'prefers a receiver method_missing over a context method of the same name' do
+ receiver = Class.new do
+ def method_missing(name, *args, &block)
+ name == :shared_method ? 'from receiver' : super
+ end
+ # respond_to_missing? is deliberately not defined.
+ end.new
+
+ context = Class.new do
+ def resolve(receiver)
+ Sunspot::Util::ContextBoundDelegate
+ .instance_eval_with_context(receiver) { shared_method }
+ end
+
+ def shared_method
+ 'from context'
+ end
+ end.new
+
+ expect(context.resolve(receiver)).to eq('from receiver')
+ end
+
+ it 'dispatches to a BasicObject receiver that does not implement respond_to?' do
+ receiver = Class.new(BasicObject) do
+ def method_missing(name, *args, &block)
+ name == :proxied_method ? 'from proxy' : super
+ end
+ end.new
+
+ context = Class.new do
+ def resolve(receiver)
+ Sunspot::Util::ContextBoundDelegate
+ .instance_eval_with_context(receiver) { proxied_method }
+ end
+ end.new
+
+ expect(context.resolve(receiver)).to eq('from proxy')
+ end
+end
diff --git a/sunspot/spec/api/indexer/fulltext_spec.rb b/sunspot/spec/api/indexer/fulltext_spec.rb
index 05c3ea327..4700e5567 100644
--- a/sunspot/spec/api/indexer/fulltext_spec.rb
+++ b/sunspot/spec/api/indexer/fulltext_spec.rb
@@ -40,4 +40,35 @@
session.index(Photo.new)
expect(connection.adds.last.first.attrs[:boost]).to eq(0.75)
end
+
+ describe 'with index_time_boosts off' do
+ before { config.solr.index_time_boosts = false }
+
+ it 'sends no field boost' do
+ session.index(post(:title => 'A Title'))
+ expect(connection.adds.last.first.field_by_name(:title_text).attrs).not_to have_key(:boost)
+ end
+
+ it 'sends no document boost' do
+ session.index(Photo.new)
+ expect(connection.adds.last.first.attrs).not_to have_key(:boost)
+ end
+
+ it 'sends no field boost from a batch' do
+ session.batch { session.index(post(:title => 'A Title')) }
+ expect(connection.adds.last.first.field_by_name(:title_text).attrs).not_to have_key(:boost)
+ end
+
+ it 'sends no field boost in an atomic update' do
+ session.atomic_update(Post, post.id => { :title => 'A Title' })
+ expect(connection.adds.last.first.field_by_name(:title_text).attrs).not_to have_key(:boost)
+ end
+ end
+
+ it 'sends no field boost once index_time_boosts is turned off after indexing' do
+ session.index(post(:title => 'A Title'))
+ config.solr.index_time_boosts = false
+ session.index(Post.new(:title => 'Another Title'))
+ expect(connection.adds.last.first.field_by_name(:title_text).attrs).not_to have_key(:boost)
+ end
end
diff --git a/sunspot/spec/api/nested_documents_spec.rb b/sunspot/spec/api/nested_documents_spec.rb
index d06f03adb..aa1f6e6f6 100644
--- a/sunspot/spec/api/nested_documents_spec.rb
+++ b/sunspot/spec/api/nested_documents_spec.rb
@@ -96,6 +96,12 @@ def children(document)
expect(children(connection.adds.last.first)).to be_empty
end
+ it 'sends no child field boost with index_time_boosts off' do
+ session.config.solr.index_time_boosts = false
+ session.index(Gadget.new(:milestones => [milestone]))
+ expect(children(connection.adds.last.first).first.field_by_name(:notes_text).attrs).not_to have_key(:boost)
+ end
+
end
describe 'indexing with an RSolr that has no child documents', :unless => child_documents do
diff --git a/sunspot/spec/api/query/fulltext_examples.rb b/sunspot/spec/api/query/fulltext_examples.rb
index e99d88345..2e3dcb62d 100644
--- a/sunspot/spec/api/query/fulltext_examples.rb
+++ b/sunspot/spec/api/query/fulltext_examples.rb
@@ -420,7 +420,7 @@
q_name = "qPhoto#{obj_id}"
expect(connection.searches.last[:q]).to eq "(_query_:\"{!join from=photo_container_id_i to=id_i v=$#{q_name}}\" OR _query_:\"{!edismax qf='description_text^1.2'}keyword2\")"
- expect(connection.searches.last[q_name]).to eq "_query_:\"{!field f=type}Photo\"+_query_:\"{!edismax qf='caption_text'}keyword1\""
+ expect(connection.searches.last[q_name]).to eq "+_query_:\"{!field f=type}Photo\"+_query_:\"{!edismax qf='caption_text'}keyword1\""
end
it "should be able to resolve name conflicts with the :prefix option" do
@@ -435,7 +435,7 @@
q_name = "qPhoto#{obj_id}"
expect(connection.searches.last[:q]).to eq "(_query_:\"{!edismax qf='description_text^1.2'}keyword1\" OR _query_:\"{!join from=photo_container_id_i to=id_i v=$#{q_name}}\")"
- expect(connection.searches.last[q_name]).to eq "_query_:\"{!field f=type}Photo\"+_query_:\"{!edismax qf='description_text'}keyword2\""
+ expect(connection.searches.last[q_name]).to eq "+_query_:\"{!field f=type}Photo\"+_query_:\"{!edismax qf='description_text'}keyword2\""
end
it "should recognize fields when adding from DSL, e.g. when calling boost_fields" do
@@ -451,7 +451,7 @@
q_name = "qPhoto#{obj_id}"
expect(connection.searches.last[:q]).to eq "(_query_:\"{!edismax qf='description_text^1.5'}keyword1\" OR _query_:\"{!join from=photo_container_id_i to=id_i v=$#{q_name}}\")"
- expect(connection.searches.last[q_name]).to eq "_query_:\"{!field f=type}Photo\"+_query_:\"{!edismax qf='description_text^1.3'}keyword1\""
+ expect(connection.searches.last[q_name]).to eq "+_query_:\"{!field f=type}Photo\"+_query_:\"{!edismax qf='description_text^1.3'}keyword1\""
end
private
diff --git a/sunspot/spec/helpers/integration_helper.rb b/sunspot/spec/helpers/integration_helper.rb
index c2a9c5cc7..bcf5db52b 100644
--- a/sunspot/spec/helpers/integration_helper.rb
+++ b/sunspot/spec/helpers/integration_helper.rb
@@ -4,9 +4,21 @@ def self.included(base)
Sunspot.config.solr.url = ENV['SOLR_URL'] || 'http://localhost:8983/solr/default'
Sunspot.config.solr.update_format = ENV['UPDATE_FORMAT'].to_sym if ENV['UPDATE_FORMAT']
Sunspot.reset!(true)
+ # Solr 7 removed index-time boosts, and Solr 9 rejects a JSON update that carries a field boost
+ Sunspot.config.solr.index_time_boosts = IntegrationHelper.solr_version.to_i < 7
end
end
+ def self.solr_version
+ @solr_version ||= Sunspot.session.send(:connection).get('admin/system')['lucene']['solr-spec-version']
+ end
+
+ # Skips the example on Solr `major` and later, for behaviour that version changed
+ def skip_from_solr(major, reason)
+ version = IntegrationHelper.solr_version
+ skip "#{reason} (Solr #{version})" if version.to_i >= major
+ end
+
def featured_for_posts(method, param, negated = false)
with_method = negated ? :without : :with
param = date_ranges[param] if param.is_a? String
diff --git a/sunspot/spec/integration/faceting_spec.rb b/sunspot/spec/integration/faceting_spec.rb
index 299a9a5e2..f58cb7911 100644
--- a/sunspot/spec/integration/faceting_spec.rb
+++ b/sunspot/spec/integration/faceting_spec.rb
@@ -516,6 +516,7 @@ def self.test_field_type(name, attribute, field, *args)
end
it 'should return time ranges' do
+ skip_from_solr 6, 'Solr 6 removed facet.date'
time = Time.utc(2009, 7, 8)
search = Sunspot.search(Post) do
facet :published_at, :time_range => time..(time + 60*60*24*2), :sort => :count
diff --git a/sunspot/spec/integration/keyword_search_spec.rb b/sunspot/spec/integration/keyword_search_spec.rb
index ac09866aa..fcffcb58c 100644
--- a/sunspot/spec/integration/keyword_search_spec.rb
+++ b/sunspot/spec/integration/keyword_search_spec.rb
@@ -124,6 +124,7 @@
end
it 'should assign a higher score to the result matching the higher-boosted field' do
+ skip_from_solr 7, 'Solr 7 removed index-time field boosts'
search = Sunspot.search(Post) { keywords 'rhinoceros' }
expect(search.hits.map { |hit| hit.primary_key }).to eq(
@posts.map { |post| post.id.to_s }
@@ -142,6 +143,7 @@
end
it 'should assign a higher score to the higher-boosted document' do
+ skip_from_solr 7, 'Solr 7 removed index-time document boosts'
search = Sunspot.search(Post) { keywords 'test' }
expect(search.hits.map { |hit| hit.primary_key }).to eq(
@posts.map { |post| post.id.to_s }
@@ -217,6 +219,7 @@
end
it 'should assign scores in order of multiple boost query match' do
+ skip_from_solr 7, 'BM25, the default similarity from luceneMatchVersion 6, scores the featured term below the rating range'
search = Sunspot.search(Post) do
keywords 'rhino' do
boost(2.0) { with(:featured, true) }
diff --git a/sunspot/spec/integration/scoped_search_spec.rb b/sunspot/spec/integration/scoped_search_spec.rb
index 75cd77bbe..e867362d0 100644
--- a/sunspot/spec/integration/scoped_search_spec.rb
+++ b/sunspot/spec/integration/scoped_search_spec.rb
@@ -594,6 +594,7 @@ def search_ordered_by_random(direction = nil)
end
it 'should work properly when combined with fulltext' do
+ skip_from_solr 6, 'Ranks @p1 first on Solr 6.6.6 and 9.10.1, under BM25 and classic similarity alike; cause not yet found'
search = Sunspot.search(Post) do
fulltext('Post Ipsum') do
boost_fields :body => 0.2
diff --git a/sunspot/spec/mocks/project.rb b/sunspot/spec/mocks/project.rb
index 5d2c4e4c6..f5d709308 100644
--- a/sunspot/spec/mocks/project.rb
+++ b/sunspot/spec/mocks/project.rb
@@ -113,7 +113,7 @@ def milestones
nested :milestones do
string(:only_here) { 'yes' }
integer(:started_at) { 1 }
- text(:notes) { 'gadget notes' }
+ text(:notes, :boost => 2) { 'gadget notes' }
end
end
diff --git a/sunspot_rails/Appraisals b/sunspot_rails/Appraisals
index d24067e7a..66f1f291a 100644
--- a/sunspot_rails/Appraisals
+++ b/sunspot_rails/Appraisals
@@ -11,6 +11,8 @@ if ruby_version < Gem::Version.new('3.0.0')
gem 'sqlite3', '~> 1.3.0'
gem 'rspec', '~> 3.7'
gem 'rspec-rails', '~> 3.7'
+ # ActiveSupport < 7.2.4 and 8.0.x pass quirks_mode to JSON.generate/parse, which json 3 rejects
+ gem 'json', '< 3'
end
end
end
@@ -21,12 +23,13 @@ if Gem::Version.new('3.0.0') <= ruby_version && ruby_version < Gem::Version.new(
gem 'sunspot', path: File.expand_path('sunspot', ENV['SUNSPOT_LIB_HOME'])
gem 'sunspot_solr', path: File.expand_path('sunspot_solr', ENV['SUNSPOT_LIB_HOME'])
gem 'rails', "~> #{rails_version}"
- gem 'json', '< 3' # Under json 3, these Rails versions' to_json raises "unknown keyword: quirks_mode"
gem 'sprockets', '~> 3.0'
gem 'progress_bar', '~> 1.0.5', require: false
gem 'sqlite3', '~> 1.4.0'
gem 'rspec', '~> 3'
gem 'rspec-rails', '~> 6'
+ # ActiveSupport < 7.2.4 and 8.0.x pass quirks_mode to JSON.generate/parse, which json 3 rejects
+ gem 'json', '< 3'
end
end
end
diff --git a/sunspot_rails/lib/sunspot/rails.rb b/sunspot_rails/lib/sunspot/rails.rb
index e10211472..734fd798a 100644
--- a/sunspot_rails/lib/sunspot/rails.rb
+++ b/sunspot_rails/lib/sunspot/rails.rb
@@ -54,6 +54,7 @@ def master_config(sunspot_rails_configuration)
config.solr.open_timeout = sunspot_rails_configuration.open_timeout
config.solr.proxy = sunspot_rails_configuration.proxy
config.solr.update_format = sunspot_rails_configuration.update_format
+ config.solr.index_time_boosts = sunspot_rails_configuration.index_time_boosts
config
end
@@ -70,6 +71,7 @@ def slave_config(sunspot_rails_configuration)
config.solr.open_timeout = sunspot_rails_configuration.open_timeout
config.solr.proxy = sunspot_rails_configuration.proxy
config.solr.update_format = sunspot_rails_configuration.update_format
+ config.solr.index_time_boosts = sunspot_rails_configuration.index_time_boosts
config
end
end
diff --git a/sunspot_rails/lib/sunspot/rails/configuration.rb b/sunspot_rails/lib/sunspot/rails/configuration.rb
index 147b773f9..1d127fbac 100644
--- a/sunspot_rails/lib/sunspot/rails/configuration.rb
+++ b/sunspot_rails/lib/sunspot/rails/configuration.rb
@@ -305,6 +305,14 @@ def update_format
@update_format ||= user_configuration_from_key('solr', 'update_format')
end
+ #
+ # Whether to send index-time field and document boosts. Defaults to true.
+ # Set it to false on Solr 7 and later, which removed them.
+ #
+ def index_time_boosts
+ @index_time_boosts ||= user_configuration_from_key('solr', 'index_time_boosts') != false
+ end
+
def proxy
@proxy ||= user_configuration_from_key('solr', 'proxy')
end
diff --git a/sunspot_rails/spec/configuration_spec.rb b/sunspot_rails/spec/configuration_spec.rb
index edb6a4514..1d1af0c22 100644
--- a/sunspot_rails/spec/configuration_spec.rb
+++ b/sunspot_rails/spec/configuration_spec.rb
@@ -61,6 +61,10 @@
expect(@config.update_format).to be_nil
end
+ it "should send index-time boosts when 'index_time_boosts' is not set" do
+ expect(@config.index_time_boosts).to eq(true)
+ end
+
it "should set 'log_level' property using Rails log level when not set" do
allow(::Rails.logger).to receive(:level){ 3 }
expect(@config.log_level).to eq('SEVERE')
@@ -171,6 +175,10 @@
expect(@config.update_format).to eq('json')
end
+ it "should handle the 'index_time_boosts' property when set" do
+ expect(@config.index_time_boosts).to eq(false)
+ end
+
it "should handle the 'proxy' property when set" do
expect(@config.proxy).to eq('http://proxy.com:12345')
end
diff --git a/sunspot_rails/spec/rails_app/config/sunspot.yml b/sunspot_rails/spec/rails_app/config/sunspot.yml
index ed1791ce9..70c68f9d7 100644
--- a/sunspot_rails/spec/rails_app/config/sunspot.yml
+++ b/sunspot_rails/spec/rails_app/config/sunspot.yml
@@ -24,6 +24,7 @@ config_test:
read_timeout: 2
open_timeout: 0.5
update_format: json
+ index_time_boosts: false
proxy: http://proxy.com:12345
auto_commit_after_request: false
auto_commit_after_delete_request: true