Fetchers/listen notes - #25
Conversation
Build a fresh podcast_map on each run instead of reading and aggregating data/listennotes.json, matching the behavior of apple_podcasts.py. Refs Infrasity-Labs#17 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces a new script fetchers/listen_notes.py to fetch podcasts from the Listen Notes API, utilizing a daily rotation of search queries to stay within the free tier limit. Feedback points out that overwriting data/listennotes.json on each run discards podcasts fetched on previous days, and suggests loading and merging existing data at startup instead.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| filepath = 'data/listennotes.json' | ||
| podcast_map = {} |
There was a problem hiding this comment.
Since this script only runs a subset of queries (10 per day) to stay within the free tier limit, overwriting data/listennotes.json on each run will discard all podcasts fetched on previous days. To ensure we accumulate and rotate podcasts over time as intended, we should load the existing data/listennotes.json file (if it exists) at startup and merge the new results into it.
filepath = 'data/listennotes.json'
podcast_map = {}
if os.path.exists(filepath):
try:
with open(filepath, 'r', encoding='utf-8') as f:
for item in json.load(f):
title = item.get("title")
if title:
podcast_map[title] = item
except Exception as e:
print(f"Warning: Failed to load existing podcasts from {filepath}: {e}")
I added a fetcher for ListenNotes. The free quota allows the user to make only 300 requests in a month. Given that the cron job defined in the yml file makes a call once a day, I have capped the number of requests at 10. I have defined a set of keywords and am rotating between them based on variables like the day of the year to ensure we aggregate all kinds of podcasts over time.
Please note that the maintainer must create an API key on Listen Notes and add it to the repo as a secret named LISTENNOTES_API_KEY for data to be scraped successfully.
Fixes #17