Repository navigation
maxkernel: move adk/evaluation to maxkernel/evaluation #147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -66,7 +66,9 @@ if result.error_trace: | |
| ### CLI Usage | ||
|
|
||
| > [!NOTE] | ||
| > All CLI commands should be run from the project root directory (`MaxKernel`). | ||
| > All CLI commands should be run from the `maxkernel/` directory, with | ||
| > `maxkernel/adk` on `PYTHONPATH` (the code adapter imports `hitl_agent` | ||
| > from there), e.g. `PYTHONPATH=adk python3 -m evaluation.benchmark ...`. | ||
|
|
||
| You can also use `jax_kernel_evaluator.py` directly from the command line to evaluate a single kernel without setting up a dataset. | ||
|
|
||
|
|
@@ -162,7 +164,9 @@ harness, because inner-jit donation is dropped when traced under another jit. | |
| ### CLI Usage Examples | ||
|
|
||
| > [!NOTE] | ||
| > All CLI commands should be run from the project root directory (`MaxKernel`). | ||
| > All CLI commands should be run from the `maxkernel/` directory, with | ||
| > `maxkernel/adk` on `PYTHONPATH` (the code adapter imports `hitl_agent` | ||
| > from there), e.g. `PYTHONPATH=adk python3 -m evaluation.benchmark ...`. | ||
|
Comment on lines
+167
to
+169
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Using a relative path in |
||
|
|
||
| **Run benchmark on a remote TPU:** | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| [tool.ruff] | ||
| line-length = 80 | ||
| target-version = "py310" | ||
| indent-width = 2 | ||
| extend-exclude = ["jaxbench_adapted_dataset"] | ||
| # First-party import roots: `evaluation` is imported from `maxkernel/`, and | ||
| # `hitl_agent` (used by code_adapter) from `maxkernel/adk/`. | ||
| src = ["..", "../adk"] | ||
|
|
||
| [tool.ruff.lint] | ||
| select = ["E", "F", "I", "N", "W"] | ||
| ignore = [ | ||
| "E501", # Line too long | ||
| "N", # Naming conventions | ||
| "W", # Warnings | ||
| "E741", # Ambiguous variable name | ||
| "F821", # Undefined name (often in specialized kernels) | ||
| "F823", # Local variable referenced before assignment | ||
| "E722", # Bare except | ||
| "F841", # Local variable assigned but never used | ||
| "E731", # Lambda assignment | ||
| "E402", # Module level import not at top of file | ||
| ] | ||
|
Comment on lines
+12
to
+23
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ignoring critical linting rules like |
||
|
|
||
| [tool.ruff.lint.per-file-ignores] | ||
| "__init__.py" = ["F401"] # Unused imports are common in __init__.py for re-exporting | ||
|
|
||
| [tool.ruff.format] | ||
| quote-style = "double" | ||
| indent-style = "space" | ||
| skip-magic-trailing-comma = false | ||
| line-ending = "auto" | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using a relative path in
PYTHONPATH(likePYTHONPATH=adk) can be fragile because Python resolves relative paths insys.pathrelative to the current working directory at the time of import. If any script or library (e.g., during JAX initialization or evaluation setup) changes the working directory usingos.chdir(), subsequent imports ofhitl_agentwill fail withModuleNotFoundError.\n\nUsing an absolute path via$(pwd)/adkis much more robust.