Skip to content

refactor: drop direct numpy usage - #39411

Open
ormsbee wants to merge 2 commits into
openedx:masterfrom
ormsbee:ormsbee/remove-numpy
Open

ormsbee wants to merge 2 commits into
openedx:masterfrom
ormsbee:ormsbee/remove-numpy

Conversation

@ormsbee

@ormsbee ormsbee commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Description

Removes the two direct uses of numpy in edx-platform and replaces them with the standard library. This follows #39410, which did the same for scipy in the course quality API.

Course quality API (cms/djangoapps/contentstore/api/views/course_quality.py)

np.around(np.mean(...)) and np.around(np.median(...)) become float(round(statistics.fmean(...))) and float(round(statistics.median(...))). round() rounds half to even, like np.around(). The float() keeps the serialized output unchanged (2.0, not 2).

Grade percentages (lms/djangoapps/grades/scores.py)

compute_percent() used numpy.around(x, decimals=4). It now uses round(x * 10000) / 10000, which is the same algorithm numpy uses: scale, round half to even, then unscale. I deliberately avoided round(x, 4). It works from the float's exact decimal value, so it sometimes gives a different fourth decimal place than numpy, and that could shift persisted grades.

Testing

  • Bit-for-bit comparison against numpy 1.26.4:
    • compute_percent: 3.56M inputs, including every integer earned/possible with possible <= 2000, random weighted scores, sums of fractional points, and values on and next to each half-step. There were 0 mismatches. For comparison, round(x, 4) would have differed on 2,374 of them.
    • Course quality mean and median: 200K random datasets (integer block counts and float video durations), 0 mismatches.
  • cms/djangoapps/contentstore/api/tests/test_quality.py passes. It already covers the round-half-to-even case (a mean and median of 1.5 report as 2.0).
  • lms/djangoapps/grades test suite passes.

Where numpy still comes from

numpy stays installed after this, because these dependencies require it:

  • openedx-calc: the formula evaluator maps sin, sqrt, log, and the rest onto numpy and numpy.lib.scimath. edx-platform imports calc directly in common/djangoapps/util/views.py, and capa uses it for numerical and formula responses.
  • Shapely: hard-requires numpy>=1.26. capa uses it for image response click regions.
  • chem: chem.miller uses numpy vectors to grade crystallography problems. chem also declares scipy, but nothing in it imports scipy. That declaration is now the only reason scipy is installed in the LMS/CMS environment.
  • xblocks-core: capa's responsetypes.py imports numpy, but only for numpy.complex in FormulaResponse.strip_dict(). That attribute was removed in numpy 1.24, and nothing calls the method. capa's safe_exec also lists numpy and scipy as lazy imports for problem code.
  • elasticsearch 7.9.1 (optional): its serializer imports numpy when it's present. This is why the numpy<2.0.0 constraint exists (Unpin elasticsearch and numpy in edx-platform #35126), so that constraint should stay.

The codejail sandbox (requirements/edx-sandbox) pins numpy and scipy directly, so that problem code can use them. That's intentional and unaffected by this PR.

🤖 Generated with Claude Code

@ormsbee
ormsbee force-pushed the ormsbee/remove-numpy branch from 57b68e7 to 22c75d3 Compare November 20, 2026 03:53
@ormsbee
ormsbee marked this pull request as ready for review November 20, 2026 03:55
ormsbee and others added 2 commits November 20, 2026 20:31
The course quality view only used numpy for mean, median, and rounding.
Replace them with statistics.fmean(), statistics.median(), and round(),
which rounds half to even like np.around(). The results are wrapped in
float() so the serialized values stay the same (e.g. 2.0, not 2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
compute_percent() used numpy.around(x, decimals=4). Replace it with
round(x * 10000) / 10000, which is the same algorithm numpy uses (scale,
round half to even, unscale), so the results are bit-for-bit identical.

This is deliberately not round(x, 4). Python's round() works from the
exact decimal value of the float, so for some inputs it gives a different
fourth decimal place than numpy, which could shift persisted grades.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ormsbee
ormsbee force-pushed the ormsbee/remove-numpy branch from 22c75d3 to f628a1c Compare November 21, 2026 01:31
@ormsbee

ormsbee commented Nov 22, 2026

Copy link
Copy Markdown
Contributor Author

FYI @blarghmatey and @pdpinch, as this touches grade calculation, though I don't expect it to actually cause any issues.

@ormsbee

ormsbee commented Nov 22, 2026

Copy link
Copy Markdown
Contributor Author

@feanil, @kdmccormick: May I trouble one of you folks to review this one?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant