Skip to content

[Core] az: Add core.output_newline and default to LF line endings when run from WSL or Git Bash on Windows - #34236

Open
Phil Sphicas (philsphicas) wants to merge 1 commit into
Azure:devfrom
philsphicas:wsl-az-lf-newlines
Open

Phil Sphicas (philsphicas) wants to merge 1 commit into
Azure:devfrom
philsphicas:wsl-az-lf-newlines

Conversation

@philsphicas

@philsphicas Phil Sphicas (philsphicas) commented Oct 16, 2026 •

Copy link
Copy Markdown

🤖 PR Validation — ️✔️ All clear

Breaking Changes Tests
️✔️ None ️✔️ 130/130

Related command
az (all commands): stdout line endings of the Windows MSI install when invoked from bash (WSL, Git Bash)

Description
Fixes #33385.

On Windows, Python's text-mode sys.stdout converts every \n to \r\n when writing. When the Windows az is called from WSL or Git Bash, FOO=$(az ... -o tsv) therefore captures a trailing \r, and for x in $(az ... -o tsv) gives every item except the last a stray \r. That \r ends up in later az calls and causes misleading errors such as 400 Bad Request. Data that already contains \r\n is also corrupted to \r\r\n.

The cause is the stdout stream, not the tsv formatter, so this affects every -o format and commands that print() directly (e.g. az aks get-credentials --file -, --help, --version).

Changes:

  • New config core.output_newline (native | lf; env var AZURE_CORE_OUTPUT_NEWLINE via the usual knack mapping).
    • lf calls stdout.reconfigure(newline='\n') in place, so text is written exactly as generated, with no conversion and no \r\r\n corruption.
    • native keeps today's behaviour.
    • An empty value means the default.
    • Invalid or unreadable values log a warning and keep native behaviour instead of failing the command.
  • The MSI bash launcher (build_scripts/windows/scripts/az → CLI2\wbin\az, which WSL and Git Bash run for az) sets AZ_LAUNCHER=bash and adds it to WSLENV, so LF becomes the default for bash callers only. An explicit env var or config value always wins.
  • Applied from azure/cli/__main__.py via an EVENT_CLI_PRE_EXECUTE handler: logging is configured by then and no command output has been written yet. Library use of get_default_cli().invoke() or run_az_cmd() is unaffected.

Effect:

  • cmd/PowerShell: output is byte-identical unless the user opts in.
  • WSL / Git Bash with the MSI install: stdout is LF-only, matching az on Linux and macOS.
  • Unchanged: stderr, binary stdout writes (e.g. az storage blob download to stdout), and child processes.

Related: #33427 takes a tsv-only approach to the same issue. This change also covers json, yaml, table and direct print() output, preserves CRLF already in data, and leaves non-bash Windows callers unchanged.

Testing Guide
Unit tests (10 new in src/azure-cli-core/azure/cli/core/tests/test_output.py) cover:

  • resolution order (env > global config > launcher default > native)
  • empty values
  • LF bytes, including CRLF already in the data
  • native
  • invalid and unreadable values
  • streams without reconfigure

Commands run and results:

  • python -m unittest azure.cli.core.tests.test_output azure.cli.core.tests.test_util: passed (48 tests). This was run with the Python bundled in the Windows MSI install (2.90.0), with this branch's src/azure-cli-core and src/azure-cli placed first on sys.path. azdev test was not run: dependency installation failed locally with a TLS error.
  • flake8 --max-line-length 120 on the changed Python files: passed.
  • pylint --rcfile=pylintrc src/azure-cli-core/azure/cli/core/_output.py: passed except R0903 on AzOutputProducer, which is reported for the unmodified file too.
  • shellcheck build_scripts/windows/scripts/az: passed.
  • End-to-end from WSL (Ubuntu 24.04) against the real MSI Python running this branch's code, through the modified launcher: 15 checks passed. Covered:
    • LF by default for json, yaml, table, tsv, --help, --version and bare az
    • AZURE_CORE_OUTPUT_NEWLINE=native (added to WSLENV) → CRLF
    • invalid value → standard WARNING:, hidden by --only-show-errors
    • existing WSLENV entries (including AZ_LAUNCHER/u)
    • no launcher → native
    • error exit code preserved

Manual check from WSL or Git Bash with an MSI install that includes this change:

x=$(az account show --query id -o tsv); printf '%q\n' "$x"     # no $'...\r'
az version --query 'join(`"\r\n"`, [`"a"`, `"b"`])' -o tsv | od -c   # a \r \n b \n  (was a \r \r \n b \r \n)
AZURE_CORE_OUTPUT_NEWLINE=native WSLENV="$WSLENV:AZURE_CORE_OUTPUT_NEWLINE" az version -o tsv | od -c  # CRLF (WSL)
az config set core.output_newline=native                         # opt out persistently

From PowerShell or cmd, output is unchanged (CRLF) unless core.output_newline=lf is set.

History Notes

[Core] az: Add core.output_newline config (native | lf) and default to LF line endings on stdout when run from WSL or Git Bash on Windows


This checklist is used to make sure that common guidelines for a pull request are followed.

… bash on Windows

On Windows, Python's text-mode stdout translates every "\n" to "\r\n", so
`FOO=$(az ... -o tsv)` from WSL or Git Bash captures a trailing "\r", and
"\r\n" already present in data is written as "\r\r\n".

- New config `core.output_newline` (`native` | `lf`, env AZURE_CORE_OUTPUT_NEWLINE).
  `lf` reconfigures stdout in place with newline="\n"; `native` keeps today's behavior.
- The MSI bash launcher (wbin/az, used by WSL and Git Bash) sets AZ_LAUNCHER=bash,
  which defaults the setting to `lf`. Explicit env/config always wins.
- The launcher adds AZ_LAUNCHER to WSLENV so it reaches the Windows process under WSL.
- cmd/PowerShell behavior is unchanged unless the setting is configured.

Fixes Azure#33385

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e8fa45af-9457-40bb-b8c7-3b159937f2e4
Copilot AI balanced review requested due to automatic review settings October 16, 2026 20:01
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@microsoft-github-policy-service microsoft-github-policy-service Bot added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Oct 16, 2026
@microsoft-github-policy-service

Copy link
Copy Markdown
Contributor

Thank you for your contribution Phil Sphicas (@philsphicas)! We will review the pull request and get back to you soon.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are localized, preserve native defaults for other launchers, and have focused regression coverage with no unresolved findings.

0 open findings

What changed in this PR

Adds configurable stdout line endings to prevent stray carriage returns when Windows Azure CLI output is consumed by WSL or Git Bash.

Changes:

  • Adds core.output_newline with environment/config overrides and a launcher-dependent default.
  • Enables LF by default through the Windows bash launcher while preserving native behavior elsewhere.
  • Adds regression tests for configuration precedence, byte output, and fallback behavior.
File Description
src/​azure-cli/​azure/​cli/​__main__.py Registers newline handling before command execution.
src/​azure-cli-core/​azure/​cli/​core/​tests/​test_output.py Tests newline selection and stream behavior.
src/​azure-cli-core/​azure/​cli/​core/​_output.py Resolves configuration and reconfigures stdout.
src/​azure-cli-core/​azure/​cli/​core/​_environment.py Defines the launcher environment-variable constant.
build_scripts/​windows/​scripts/​az Marks bash invocation and forwards the marker through WSL.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act-codegen-extensibility-squad act-identity-squad act-platform-engineering-squad ARM az resource/group/lock/tag/deployment/policy/managementapp/account management-group Auto-Assign Auto assign by bot Core CLI core infrastructure customer-reported Issues that are reported by GitHub users external to the Azure organization. Installation Storage az storage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

-o tsv emits CRLF on Windows, breaking for x in $(az ... -o tsv) shell loops with misleading 400 errors

3 participants