Fix checksum verification timing out on large OS images - #411
Conversation
| match[1], | ||
| ) | ||
| return match[1] | ||
| return match[1] |
There was a problem hiding this comment.
The return match[1] was sitting outside the if match := re.search(...) block but inside the for loop, so it executed unconditionally on the first iteration. When the first pattern didn't match — which is exactly what happens when the verify output is truncated by a timeout — match was None and this raised TypeError: 'NoneType' object is not subscriptable. The second pattern was never tried, and the intended CommandError below the loop was unreachable.
Moving the return inside the if restores the obvious intent: try each pattern, return on the first match, and fall through to CommandError if none match. No behavior change on the happy path — pattern 1 matching still returns immediately.
| return False | ||
|
|
||
| def verify_file(self, checksum, filename, hashing_algorithm="md5", file_system=None): | ||
| def verify_file(self, checksum, filename, hashing_algorithm="md5", file_system=None, read_timeout=900): |
There was a problem hiding this comment.
Is 900 a good sane default?
There was a problem hiding this comment.
The old 300s default can't verify current images: we measured verify /md5 on Cat9200 hardware hashing at under 4 MB/s, so a ~1.2 GB image already exceeds 300s, and today's 2+ GB images need 500–700s+. 600 would be marginal; 900 covers them with headroom and matches the existing FileCopyModel.timeout default.
jeffkala
left a comment
There was a problem hiding this comment.
Looks good to me, I'd like @jtdub or @gsnider2195 to review as well since they've been so in depth with this lib lately.
Fixes #410
Problem
get_remote_checksumhardcodesread_timeout=300for the checksum command(
verify /md5on IOS/ASA,run md5sumon IOS-XR), and nothing in theverify_file -> compare_file_checksum -> get_remote_checksumchain accepts atimeout. On Catalyst 9k hardware
verify /md5hashes at under 4 MB/s, so a~1.2 GB image already exceeds 300s and current 2+ GB images need 500-700+
seconds. The result:
file_copy/remote_file_copytransfers that completedsuccessfully on the device get reported as failures when the post-transfer
verification hits netmiko's absolute read timer. Retried jobs can also fail in
the pre-transfer verification, since the file is already on flash by then.