[AdrianAtZyte]

Fixes #875, fixes #1277, fixes #913

[codspeed]

Merging this PR will not alter performance

✅ 10 untouched benchmarks

---

_{Comparing AdrianAtZyte:chinese-script-variants (8a3cbcf) with master (663348c)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.03%. Comparing base (f43ba92) to head (8a3cbcf).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1407      +/-   ##
==========================================
+ Coverage   99.02%   99.03%   +0.01%     
==========================================
  Files         239      239              
  Lines        3786     3838      +52     
==========================================
+ Hits         3749     3801      +52     
  Misses         37       37              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[serhii73]

Thanks, this fixes the parse() cases from #875, #1277 and #913. With the new data, though, search_dates now raises
for common zh-Hans / zh-Hant text, including with automatic language detection:

from dateparser.search import search_dates

search_dates("最後更新於3小時前", languages=["zh-Hant"])
# master: [('3小時', ...)]           this PR: IndexError: list index out of range
search_dates("时间是下午3:30", languages=["zh-Hans"])
# master: None                       this PR: IndexError
search_dates("禮拜一在會議室開會，時間是下午3:30")
# master: [('時間', ...), ('3:30', ...)]   this PR: IndexError

The crash is in Locale.translate_search: the current_and_next_joined in dictionary branch is skipped for zh and
ja through word_joint_unsupported_languages = ["zh", "ja"], but that list is compared against the full shortname,
so zh-Hans and zh-Hant still go through it. With no word spacing, the last token joined with an empty next token is
the token itself. Now that 前, 天 and the time simplifications are in these locales, that token is in the
dictionary and original_tokens[i + 1] is read past the end. Comparing the base language
(self.shortname.split("-")[0] not in word_joint_unsupported_languages) stops all three inputs from raising, and
zh-Hans search then gives the same results as zh. The first input then returns None, as zh does for
最后更新于3小时前, because the chunk is joined without spaces as 3hourago. That may be worth a look too.

A second issue: in: 在 is copied from zh, and 在 is the ordinary word for "at/in":

search_dates("他在2020年去了北京", languages=["zh-Hans"],
             settings={"RELATIVE_BASE": datetime(2020, 6, 15, 12)})
# master: [('2020年', datetime(2020, 6, 15, 0, 0))]
# this PR: [('在2020年', datetime(4040, 6, 15, 12, 0))]
dateparser.parse("在2020年", languages=["zh-Hant"])   # master: None, this PR: 4040-06-15

zh already behaves like this, but it would be better not to spread it to zh-Hans and zh-Hant. Could in: 在 be left
out of the two new files?

[serhii73]

Code review (claude code-review, at 8f2f878), plus my own comparison against master. No regression found: with languages=['zh-Hans'] or ['zh-Hant'], 2020年6月1日 goes from None to a correct date, and 5分鐘前 in zh-Hant now parses. Behaviour for other languages is unchanged.

Non-blocking notes:

• (\d+)月(?=.*[前后]) -> \1 月 turns 6月1日前 into "6 months 1 day ago", giving 2019-12-14 for a base of 2020-06-15. Master already returns the same wrong date with auto-detection, so this is not new, but the rule now applies to explicit zh-Hans/zh-Hant too. The lookahead matches any later 前/后 in the string; requiring it directly after the number (or after 个月/個月) would be safer.
• search_dates('他2020年6月1日去了北京', languages=['zh-Hans']) returns the span 2020年6月1 without the final 日 (master finds nothing), so callers slicing the original text leave a stray 日.
• zh-Hans has no 分鐘 entry, so 5分鐘前 still returns None there while it works in zh-Hant. Several natural phrases (这个月, 上星期, 下星期, 3星期前) are still unparsed and untested.
• The split('-')[0] change in translate_search applies to every hyphenated locale (e.g. a zh-Hant-HK variant), unlike the exact-name checks elsewhere in locale.py.
• skip: ['约'] also occurs inside 大约, 约会 and 节约; a test for it would help. The new tests are positive-only; a negative test that Simplified-only forms stay out of zh-Hant would guard the split between the two scripts.