fix test workflow - #771
Conversation
0632cb6 to
71ff316
Compare
71ff316 to
333cdb5
Compare
| import { extension } from "./extension"; | ||
|
|
||
| const APPROX_TIME_TO_OPEN_VISUAL_EDITOR = 1600; | ||
| const APPROX_TIME_TO_OPEN_VISUAL_EDITOR = 1700; |
There was a problem hiding this comment.
Wow, I believe you, but if we're cutting this short, then maybe we should bump it to a notably larger number?
I'm not sure that a 100/1600 ~ 6% increase is justifiable without a deeper understanding as to why the number would be consistently larger than 1600ms but consistently smaller than 1700.
There was a problem hiding this comment.
All I know is that I originally tried an additional await wait(APPROX_TIME_TO_OPEN_VISUAL_EDITOR after await extension().activate() and the tests passed in CI, then I removed that additional wait and the roundtripping test failed, then I adjusted this from 1600 to 1700 on the feeling that a bit of extra waiting seems to help and the tests all passed.
Altogether it is unclear what is happening. Extra waiting does seem to help. Its unclear where that waiting should be but it seems that the waiting does not have to be before switching to the visual editor.
There was a problem hiding this comment.
I expect this may take some finessing once we have seen more. I wouldn't be surprised if the tests flake on other PRs or once more tests are added or anything changes.
cscheid
left a comment
There was a problem hiding this comment.
I have a comment on those waiting numbers, but it's not important enough to hold off this extremely important PR!
Following up on discussion in #770, lets try to get quarto extension tests working in the test github workflow.
Adds a
build-vscodecommand topackage.jsonand runs it in the workflow prior to tests in order to ensure that all necessary parts of the extension are built in the workflow prior to testing ✅🙂waittiming and the test can fail depending on the timing, which is a bit sketchy, but seems ok for now.This was my first idea to try after #770 and it worked!
How the CI looks now
and heres a link to how it looked before (with two tests skipped in CI)