Repository navigation
Keep AM when searching for dates after a time - #1400
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1400 +/- ##
=======================================
Coverage 98.97% 98.98%
=======================================
Files 239 239
Lines 3627 3628 +1
=======================================
+ Hits 3590 3591 +1
Misses 37 37 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
serhii73
left a comment
There was a problem hiding this comment.
Thanks, this fixes all four linked issues (May 31, 8AM UTC, tomorrow 2am, 12:47 am, on Monday at 7am). Two things before merging:
1. The updated test depends on the day of the month. "June 5 am utc" gets its day from the relative-base chain, which starts at today ("May 2020" → today's day, …, "January UTC" → 2023-01-). So the expected datetime(2023, 6, 23, 5, 0) only holds on the 23rd, the day the PR was pushed. On the 30th, test_relative_base_setting_2_en fails with 2023-06-30 05:00. Suggestion:
datetime.datetime(2023, 6, min(datetime.datetime.now(tz=datetime.timezone.utc).day, 30), 5, 0, tzinfo=pytz.utc)(min(..., 30) because June has no 31st. The search clamps to June 30 on the 31st.)
2. German "am" ("on the") after a 24-hour time or a year is now kept as AM, and the date is lost:
from datetime import datetime
from dateparser.search import search_dates
rb = {"RELATIVE_BASE": datetime(2020, 6, 15, 12)}
search_dates("Das Konzert findet um 20 Uhr am 5. Mai 2021 statt.", languages=["de"], settings=rb)
# master: [('um 20 Uhr am 5. Mai 2021', datetime(2021, 5, 5, 0, 0))]
# PR: [('Mai 2021', datetime(2021, 5, 15, 0, 0))]
search_dates("Er wurde 1990 am 3. März in Berlin geboren", languages=["de"], settings=rb)
# master: [('1990 am 3. März in', datetime(1990, 3, 3, 0, 0))]
# PR: [('März in', datetime(2020, 3, 15, 0, 0))]Only keeping am after a valid 12-hour value seems to cover both. For example:
item = re.sub(r"(?<!\b(?:0?[1-9]|1[0-2])(?:[:.][0-5]\d)?\s*)\bam\b", "", item)With that, the issue examples still work, the German cases match master, and um 10 Uhr am 5. Mai 2021 even improves (2021-05-05 10:00 instead of 2021-05-10).
serhii73
left a comment
There was a problem hiding this comment.
Code review (claude code-review, at ecffb5e). No new correctness regression found. Keeping "am" after a 1-12 time is the intended change, and the German "am 5. Mai" cases in the new tests behave as asserted.
Non-blocking notes:
- The rule is a syntactic guess: any "am" after a 1-12 number or h:mm is kept, so German
<1-12> am <date>(e.g. "Seite 4 am 7. Juni") will still read the preposition as AM. Using the source language, or the translation step, would be more robust than looking at what precedes the token. "June 5 am utc"now expectsmin(now.day, 30), so its expected value depends on the real clock. A fixedRELATIVE_BASEwould make it deterministic. The"May 31"and"8am UTC"cases were also replaced, which loses the original coverage.- The test for "um 20 Uhr am 5. Mai 2021" expects midnight, so the 20:00 is dropped (the same as before this PR); the 10 Uhr case keeps its time. The "1990 am 3. März in" test also pins the stray trailing "in" in the span.
- The regex is case-sensitive and runs per candidate in
parse_item; precompiling it at module level would be cheaper.
Fixes #681, fixes #1037, fixes #894, fixes #504.