[AdrianAtZyte]

Fixes #1163

[codspeed]

Merging this PR will not alter performance

✅ 10 untouched benchmarks

---

_{Comparing AdrianAtZyte:detect-language-from-phrases (e3a27fc) with master (2f54bb0)}

[codecov]

Codecov Report

❌ Patch coverage is 93.93939% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.05%. Comparing base (4c4c5e5) to head (e3a27fc).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
dateparser/languages/locale.py	87.50%	1 Missing ⚠️
dateparser/search/search.py	96.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1435      +/-   ##
==========================================
- Coverage   99.09%   99.05%   -0.05%     
==========================================
  Files         239      239              
  Lines        3988     4011      +23     
==========================================
+ Hits         3952     3973      +21     
- Misses         36       38       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[serhii73]

Code review (claude code-review, at 2b73aca; compared with the PR's merge-base because the branch is 56 commits behind master). Not approving as is:

1. Scores are inflated per word. A multi-word entry now counts once per word, and every locale that has those words as separate single-word entries gets the same inflated score, while locales are ranked by that count first. detect_language('I will see you next week at noon', languages=None) returns ff on this head (nl at the merge-base; en is the right answer and scores [2,3] against ff's [3,0]), and search_dates on that text with auto-detection returns the junk match ('see', ...) plus a misdated next week. count_applicability('next week') goes from [0,0] to [2,0], and Meet me on 5 June next week ties nn and dje with en.
2. Deduplication changed. The set of individual words replaces set(words) over whole tokens, so tokens that share a word undercount (last week and next week count week once), and scores depend on which phrases happen to share words.
3. An existing expectation was changed. The Filipino sentence in tests/test_search.py flips from tl to fil because the inflated scores change which tied locale wins. That edit hides a behaviour change instead of showing it is intended.
4. Tests are too narrow. They only check text that is a single multi-word entry (next week, next month). They do not cover mixed text, repeated entries, or several competing languages, so the ff result above passes the suite.

The branch also needs a rebase; #1401 changes _count_words_present_in_the_dictionary too (a guard so padded chunks don't credit parts), so this PR may overlap with it. Counting a multi-word entry once, when all of its words are present, would keep scores comparable.