Skip to content

Commit 5703e1c

Browse files
soloturnclaude
andcommitted
RedditRipper: rate-limit v.redd.it, fix silent video-download failures
Fixes #219 (ripmeapp2/ripme): "images are downloaded but no videos". handleURL() runs synchronously per post with no delay, so a listing full of video posts hit v.redd.it's DASH manifest/video endpoints back-to-back. redgifs.com already gets a 3s delay right below this for exactly this reason ("redgifs.com rate limits us if we download too fast") -- v.redd.it never got the same treatment despite being at least as common a link on Reddit. Also fixes two silent-failure bugs found along the way: - parseRedditVideoMPD's catch block used e.printStackTrace(), which isn't routed through log4j2 at all -- failures were invisible in normal operation. - when parseRedditVideoMPD returns null, the video was silently skipped with no log output whatsoever. Now logs a warning so a skipped video is diagnosable instead of just vanishing. Verified the DASH-manifest parsing logic itself is correct: fetched the manifest from the reporter's own v.redd.it ID directly, confirmed it selects DASH_360.mp4 as the best-quality representation (matching the report exactly), and that URL downloads fine. Could not directly reproduce a rate-limit failure via manual concurrent-request testing, so this fix targets the most plausible explanation matching the established redgifs.com precedent in the same method, not a confirmed root cause. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 01c622c commit 5703e1c

1 file changed

Lines changed: 7 additions & 1 deletion

File tree

src/main/java/com/rarchives/ripme/ripper/rippers/RedditRipper.java

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -402,7 +402,7 @@ private URL parseRedditVideoMPD(String vidURL) {
402402
}
403403
return new URI(vidURL + "/" + baseURL).toURL();
404404
} catch (IOException | URISyntaxException e) {
405-
e.printStackTrace();
405+
logger.error("[!] Failed to parse DASH manifest for " + vidURL, e);
406406
}
407407
return null;
408408

@@ -436,12 +436,18 @@ private void handleURL(String theUrl, String id, String title) {
436436
savePath += id + "-" + m.group(1) + Utils.filesystemSafe(title) + ".jpg";
437437
addURLToDownload(urls.get(0), Utils.getPath(savePath));
438438
} else if (url.contains("v.redd.it")) {
439+
// v.redd.it rate limits us if we request DASH manifests/videos too fast,
440+
// same as redgifs.com below, and v.redd.it links are common in subreddit
441+
// listings with many video posts fetched back-to-back.
442+
sleep(3000);
439443
String savePath = this.workingDir + "/";
440444
savePath += id + "-" + url.split("/")[3] + Utils.filesystemSafe(title) + ".mp4";
441445
URL urlToDownload = parseRedditVideoMPD(urls.get(0).toExternalForm());
442446
if (urlToDownload != null) {
443447
logger.info("url: " + urlToDownload + " file: " + savePath);
444448
addURLToDownload(urlToDownload, Utils.getPath(savePath));
449+
} else {
450+
logger.warn("[!] Skipping video, could not resolve download URL from DASH manifest: " + url);
445451
}
446452
} else {
447453
if (url.contains("redgifs.com")) {

0 commit comments

Comments
 (0)