Skip to content

File names on macOS without Unicode with another locale (iconv) - #206

Merged
JeromeMartinez merged 1 commit into
MediaArea:masterfrom
JeromeMartinez:CI_Codepages
Nov 7, 2026
Merged

JeromeMartinez merged 1 commit into
MediaArea:masterfrom
JeromeMartinez:CI_Codepages

Conversation

@JeromeMartinez

Copy link
Copy Markdown
Member

File names on macOS without Unicode, with a locale which is neither ASCII nor UTF-8.

Without Unicode, strings are the bytes of the locale encoding, and the File and Dir functions gave them as they are to the file system, which accepts only valid UTF-8 names on macOS: with e.g. a Latin-1, EUC-JP or GB18030 locale, names typed in a terminal using it could not be opened or created (on Linux and FreeBSD, any bytes are a valid name, so nothing changes there).

  • FileName_To_Native() converts the names which are not valid UTF-8 from the locale encoding to UTF-8, and FileName_From_Native() the names of the file system to the locale encoding when it can encode them and the conversion gives back the same name, as with Unicode.
  • The File and Dir functions use them also without Unicode (they used them only with Unicode).
  • The conversions use iconv of the system, because wchar_t is not Unicode with most of these locales on macOS, with the IANA name of the encoding when iconv does not know the name of the locale (e.g. eucJP). ZenLib is linked with -liconv on macOS (autotools, libzen.pc, libzen-config, CMake); the library is part of macOS, nothing to ship.

To be merged before MediaArea/MediaInfo#1336 and MediaArea/MediaInfoLib#2803, whose tests expect these conversions on macOS without Unicode.

… locale

Without Unicode, strings are the bytes of the locale encoding, and the
File and Dir functions gave them as they are to the file system, which
accepts only valid UTF-8 names on macOS: with a locale which is neither
ASCII nor UTF-8 (e.g. Latin-1, EUC-JP, GB18030), names typed in a
terminal using it could not be opened or created.

FileName_To_Native() now converts the names which are not valid UTF-8
from the locale encoding to UTF-8, and FileName_From_Native() the names
of the file system to the locale encoding when it can encode them and
the conversion gives back the same name, as with Unicode; the File and
Dir functions use them also without Unicode. The conversions use iconv
of the system (wchar_t is not Unicode with most of these locales on
macOS), so ZenLib is linked with -liconv on macOS.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant