Skip to content

Latest commit

 

History

History
487 lines (388 loc) · 24.7 KB

library.md

File metadata and controls

487 lines (388 loc) · 24.7 KB

Library Overview

XISOSharp.Core is the class library behind the CLI. It is published to NuGet as XISOSharp and contains the complete XISO read/write engine plus CCI (CCISharp) and ZAR (ZArchiveSharp) container support.

Companion projects

The companion applications use the same library surface:

  • SimpleXisoDrive mounts ISO/CISO/CCI and GOD/SVOD sources read-only through XisoImage/XisoImageSource, resolves CCI sets with CCISharp.CciFile, and opens CHD/ZAR through their dedicated readers (Dokan on Windows, FUSE 3 on Linux/macOS).
  • XISO Studio converts and explores Xbox images, including CCI validation, GOD/SVOD input, a title metadata panel and XBE logo/title/save previews through XbeAssets.

The StfsSharp, CCISharp and ZArchiveSharp packages remain usable on their own; XISOSharp consumes them directly for GOD/SVOD, CCI and ZAR support. The 1.x compatibility facades were removed in 1.9.0.

Consumer package usage

Consumers that need the standalone readers pin the packages directly instead of relying on XISOSharp's transitive dependencies:

<PackageReference Include="XISOSharp" Version="1.9.1" />
<PackageReference Include="CCISharp" Version="1.1.0" />
<PackageReference Include="StfsSharp" Version="1.0.0" />
using CCISharp;
using StfsSharp;
using StfsSharp.Models;

// GOD/SVOD header, .data folder or parent folder:
if (GodHeader.TryReadTitle(path, out GodTitleInfo? title))
    Console.WriteLine($"{title.TitleName} - disc {title.DiscNumber}/{title.DiscCount}");

// CCI split discovery and container validation:
CciError error = CciFile.Open(path, out CciFile? cci);
if (error == CciError.None && cci is not null)
{
    using (cci)
        Console.WriteLine($"{cci.Parts.Count} part(s), {cci.TotalSectors} sectors");
}

SimpleXisoDrive uses XISOSharp for mounting and XDVDFS reads and references StfsSharp.GodHeader directly for GOD/SVOD header, .data and parent-folder resolution. XISO Studio uses XISOSharp for exploring/reading the filesystem, StfsSharp.GodHeader directly for GOD title metadata, and CCISharp.CciFile directly for CCI container validation.

Packaging & compatibility

Aspect Value
Package ID XISOSharp
Target frameworks net8.0, net9.0, net10.0
Dependencies ZArchiveSharp (ZAR), CCISharp (CCI/CCIM), StfsSharp (GOD/SVOD and XBLA/XBLIG STFS)
Signing strong-named assembly
Trimming / AOT IsTrimmable, IsAotCompatible
Versioning MinVer from git tags (format v1.0.2 / 1.0.2)
Symbols snupkg via SourceLink
Docs XML documentation generated; package README; MIT license
API validation Package validation (strict) across target frameworks

Namespaces and types

Core operations live in XISOSharp; data models, records, and enums (UnpackOptions, ProcessRunResult, ExplorerNode, XisoExplorerOptions, VolumeInfo, EntryInfo, AuditResult, ProgressInfo, ExtractMode, …) live in XISOSharp.Models; IBlockDevice / IFilesystem live in XISOSharp.Interfaces. Internal data structures stay in XISOSharp.DataStructures.

Type Purpose
XisoImage / XisoImageHash Open-once disposable facade (list/read/extract/validate/title/hash, async wrappers) + full-image CRC32/MD5/SHA-1/SHA-256/SHA3-256
XisoReader Verify, extract, list, tree, rewrite, info, ls, audit, repair, salvage, hash, per-file hash sidecars, copy-out, copy-in, checksum, filetime, split/join, loose XEX/XBE info, XEX title metadata, BlockDevice overloads
XisoDiscInfo / XisoDiscLayout Redump/XGD layout inspector: XGD type/wave, video/game partitions, PFI layer break, embedded/sectors.txt security ranges and consistency warnings
XisoNameRules Xbox 42-character component limit and the Latin-1/Windows-1252 storage contract (GetComponentError, GetPathError, IsValidComponent)
HostIoDiagnostics Actionable host-I/O messages (path, operation, likely fix) for permission, long-path, sharing, existing-destination, illegal-name and full-disk errors
XisoDat Redump-style XML DAT parser/verifier: game/rom size + MD5/SHA-1/CRC32 matching with verified/ambiguous/not-in-DAT/missing reports
SymlinkMode Follow (default), Skip or Reject policy for create/pack/remap/Apply source walks
XisoSkeleton.Apply Skeleton overlay: restore original file data into a petrified skeleton, optionally verifying the .hash; the exact inverse of Petrify
XbeAssets XBE logo / $$XTIMAGE / $$XSIMAGE extraction with BCL-only PNG and DDS output
XisoImageSource / XisoImageFormat / XisoImageSourceInfo Unified detection and open (TryDetect, OpenRead, OpenBlockDevice, StripImageSuffix) for ISO, split ISO, CISO, CCI, GOD/SVOD, split variants and ZAR
XisoCci / CciBlockDevice CCI/CCIM compression, extraction, default names and random access (CCISharp-backed)
XisoZarchive (read APIs) ListZar, ExtractZar, ConvertZarToIso alongside the existing CreateZar
StfsSharp.StfsPackage / StfsEntry / StfsPackageInfo Read-only XBLA/XBLIG single-file STFS package (from the StfsSharp package): file table, hash-chain block lists, Find, ReadAllBytes, CopyTo, ExtractTo, Validate hash-tree verification and volume/title metadata
XisoExplorer / XisoAttributes In-place exploration (list/get/copy/hash/XEX/XBE, bounded OpenReadStream, keep-open VFS mode) + Windows attribute mapping
XisoWriter Create and rewrite images + build-image remap (CreateFromRemapTree)
XisoPatcher / DirectoryEntryTableWriter In-place file patching (CopyIntoImage) + single-table build/serialize primitive
Logger Configurable text output with quiet/silent modes
ISourceFilesystem / SourceEntry Read-side source store for packing without host paths (1.8)
IXisoLogger / XisoLogOptions / XisoLogScope Per-call log sinks for concurrent operations (1.8)
GlobMatcher/WaxGlob Glob matching for -X + ordered wax captures {0}/{n} for build-image
AvlTree AVL tree operations on directory entries
BoyerMoore Pattern search used for .xbe media patching
FileTimeHelper FILETIME writer
Constants All format constants (incl. 0x89D80000 hybrid, 0x48 reserved mask)
XisoValidator Conversion validation (compare two images)
XisoRanges/XisoSkeleton/XgdTables/XboxPrng/SecuritySectors Redump archival: ranges, skeleton/SHA-1, wave tables, PRNG seed, sectors.txt
XisoRedump/XisoOperations/XisoZarchive Video/filler/wipe/trim/petrify/update/ZAR/rebuild
XisoChecksum / CisoWriter/CisoReader SHA3-256 image checksum + CISO compress/decompress
BlockDevice/* IBlockDevice + File/Memory/Offset/Stream/Ciso/Cci devices (GOD/SVOD is mapped through StfsSharp.GodVolume by an internal adapter)
RemapFilesystem Ordered host→image remapping, xdvdfs.toml, GenerateSpecText, DryRunRemap
Records/enums VolumeInfo, EntryInfo, AuditResult, ValidationResult, XexInfo, XexTitleInfo, XbeInfo, XbeImage, TitleInfo/TitleKind, RemapRule, XisoLayoutSpec, XisoLayoutEntry, ProgressInfo, ExtractMode, …
Exceptions XisoFormatException, XisoEmptyException, XisoFileTooLargeException, ExtractErrorException

Container input

Use XisoImageSource when you need the same format coverage as the CLI:

if (XisoImageSource.TryDetect(path, out XisoImageSourceInfo? info) && info is not null)
{
    Console.WriteLine($"{info.Format}: {info.Length} bytes, {info.Parts.Count} part(s)");
}

using Stream image = XisoImageSource.OpenRead(@"C:\games\game.1.cci");
using IBlockDevice dev = XisoImageSource.OpenBlockDevice(@"C:\games\game.1.iso");
using IBlockDevice god = XisoImageSource.OpenBlockDevice(@"C:\games\4D5307E6"); // or its .data folder

Open-once facade

XisoImage keeps one decompressed view open, probes the volume eagerly and serializes operations internally:

using XisoImage image = XisoImage.Open(@"C:\games\game.1.cci");
Console.WriteLine($"{image.Source.Format} {image.Volume.DiscFormat} {image.Length}");

foreach (ExplorerNode node in image.ListFlat("/"))
    Console.WriteLine(node.FullPath);

byte[] readme = image.ReadFile("/readme.txt");
XisoImageHashResult hash = image.ComputeImageHash();   // CRC32/MD5/SHA1/SHA256/SHA3-256
TitleInfo? title = image.GetTitleInfo();               // default.xbe / default.xex / default_mp.xbe
AuditResult audit = image.Validate();
Format Detection Open behavior
Iso .iso/.xiso/.img or any unknown plain file FileStream / FileBlockDevice
SplitIso <base>.1.iso SplitIsoStream concatenates consecutive parts
Ciso / SplitCiso .cso/.1.cso, magic sniff fallback CisoBlockDevice
Cci / SplitCci .cci/.1.cci, magic sniff fallback CciBlockDevice
Zar .zar Not a sector image; use XisoZarchive.ExtractZar/ConvertZarToIso

OpenRead/OpenBlockDevice throw FileNotFoundException, InvalidDataException or NotSupportedException (ZAR) rather than returning a partially valid view.

XBLA/XBLIG STFS packages

A single-file XBLA or XBLIG package is an STFS file system, not an XISO sector image, so it has its own reader instead of XisoImageSource. Use the StfsSharp package directly (XISOSharp references it and re-exports no facade):

using StfsSharp;
using StfsSharp.Models;

using StfsPackage package = StfsPackage.Open(@"C:\games\4D5307E6");
Console.WriteLine($"{package.Info.TitleName} ({package.Info.Kind})");

foreach (StfsEntry entry in package.Entries)
    Console.WriteLine($"{entry.Path} {entry.Size}");

byte[] defaultXex = package.ReadAllBytes(package.Find("default.xex")!);
package.ExtractTo(@"C:\games\4D5307E6-extracted", skipExisting: true);

StfsSharp.StfsPackage follows the STFS volume descriptor and the level-0/level-1/level-2 hash-table interleave, including writable-format packages with primary and secondary tables. XisoChecksum.ComputeStfsChecksum uses the same sorted-path SHA3-256 tree hash as ISO and ZAR, so equivalent trees compare equal. The reader is read-only; STFS writing/rebuilding is out of scope. ExtractTo has an ExtractNamePolicy overload for hostile entry names (reject/sanitize/skip), and Validate verifies the SHA-1 hash tree (per-block hashes, level tables, top hash and header hash) and returns a StfsValidationReport; package signatures are not verified.

Design notes

  • Faithful port: the engine mirrors extract-xiso.c (reference build 202609111233) operation-for-operation so output is byte-identical to the reference tool (verified by the oracle-gated interop tests and the battle harness).
  • Directory layout is an AVL tree; the writer performs a three-pass layout calculation — see XISO Format.
  • Synchronous core: the engine is synchronous and thread-static buffers keep it allocation-light; async wrappers offload to the thread pool.
  • Absolute-path creation: XisoWriter.CreateXiso resolves a local source once and walks it with absolute host paths, so creating from a directory never changes the process working directory. Progress counters are per operation, and concurrent creates run independently without a process-wide lock. A caller-supplied inRoot without host paths is resolved once against the caller's CWD.
  • CWD-free extraction: XisoReader.Extract/UnpackImage resolve entry paths through a LocalFilesystem rooted at the output directory, so the process working directory never changes and concurrent extracts stay independent. UnpackImage(..., IFilesystem) targets a custom store (disk, memory, or any caller implementation).

Error handling

Exception Base When
XisoFormatException IOException Not a valid XISO, corrupt header/tree, bad offsets
XisoEmptyException ExtractErrorException Image contains no files
XisoFileTooLargeException IOException A file exceeds the ~4 GB XISO limit (FileName, FileSize)
ExtractErrorException Exception Non-fatal extraction errors with an ErrorCode

ExtractError codes: ErrEndOfSector (−5001), ErrIsoRewritten (−5002), ErrIsoNoFiles (−5003). The CLI treats ErrIsoNoFiles as success (0).

Logging

Logger is a static class with redirectable writers:

Logger.Out = TextWriter.Null;     // discard normal output
Logger.Error = Console.Error;     // keep errors
Logger.Quiet = true;              // suppress non-error output
Logger.RealQuiet = true;          // suppress everything
Logger.RouteInfoToError = true;   // send info to Error, keep Out for JSON

Progress during extraction/creation is reported through Logger and — for write operations — through the optional ProgressCallback (long currentBytes, long totalBytes) and the structured IProgress<ProgressInfo> channel (FileCount, DirCount, DirAdded, FileAdded, FinishedPacking events, plus per-chunk FileProgress while extracting — see ProgressInfo).

Per-call logging (1.8)

Logger stays the compatibility path. For concurrent operations, route one operation to an isolated sink instead:

using XISOSharp.Logging;

sealed class MySink : IXisoLogger
{
    public void Log(string message) => Console.WriteLine(message);
    public void LogWarning(string message) => Console.Error.WriteLine(message);
    public void LogError(string message) => Console.Error.WriteLine(message);
    public void LogDebug(string message) { }
}

using (XisoLogScope.Begin(new XisoLogOptions { Sink = new MySink() }))
{
    XisoWriter.PackFromDirectory("src", "game.iso");
}

Scopes use AsyncLocal, so two tasks can log to separate sinks. Static Logger.Out/Error writes are suppressed while a sink scope is active (set SuppressStaticOutput = false to keep both). UnpackOptions.Log applies a sink to extraction, and the new ISourceFilesystem writer overloads accept a XisoLogOptions argument.

Packing from a source filesystem (1.8)

ISourceFilesystem is the read-side counterpart to IFilesystem:

MemoryFilesystem source = new();
using (Stream stream = source.CreateFile("src/readme.txt"))
    stream.Write("hello"u8);

// MemoryFilesystem implements both IFilesystem (destination) and
// ISourceFilesystem (source); the image is byte-identical to a host pack.
XisoWriter.PackFromDirectory(source, "src", "game.iso", fileTime: 0);

DirectoryExists, EnumerateEntries, and OpenRead are the only required members; entries are root-relative with / separators. The writer sorts them ordinally and reads sequentially, so custom stores do not need to be thread-safe.

Sparse outputs (1.8)

Writer output, LocalFilesystem.CreateFile(path, expectedLength), XISO extraction, CISO/CCI decompression and ZAR extraction mark pre-sized files sparse where the OS supports it (NTFS FSCTL_SET_SPARSE; Unix holes are implicit) and skip all-zero chunks. The same call sites keep a dense fallback when marking fails, so behavior is unchanged on unsupported file systems.

Quick samples

using XISOSharp;
using XISOSharp.Models;             // VolumeInfo, XexInfo, XbeInfo, ProgressInfo, …
using System.IO;                    // FileShare, FileAttributes
using System.Security.Cryptography; // HashAlgorithmName (explorer hashing)

// Extract
int result = XisoReader.Extract("game.iso", "./out", llCompat: false);

// Resume an interrupted unpack (skip same-size files already on disk)
int resumed = XisoReader.UnpackImage("game.iso", "./out",
    options: new UnpackOptions { SkipExisting = true });

// Copy independent files with four workers (output is byte-identical)
int fast = XisoReader.UnpackImage("game.iso", "./out",
    options: new UnpackOptions { MaxDegreeOfParallelism = 4 });

// Keep going past bad files; the run still throws a summary at the end
try
{
    XisoReader.UnpackImage("game.iso", "./out",
        options: new UnpackOptions { ContinueOnError = true });
}
catch (ExtractFileException ex)  // one file: entry, sector, expected/actual
{
    Console.Error.WriteLine(ex.Message);
}
catch (ExtractErrorException ex)  // summary: every failure listed
{
    Console.Error.WriteLine(ex.Message);
}

// Recover a hostile image: rename unsafe entry names instead of aborting
int recovered = XisoReader.UnpackImage("damaged.iso", "./out",
    options: new UnpackOptions { NamePolicy = ExtractNamePolicy.Sanitize });

// List
XisoReader.List("game.iso", llCompat: false);

// Stream input (memory, network, embedded resource — readable + seekable, stays open)
using var image = new MemoryStream(File.ReadAllBytes("game.iso"));
XisoReader.UnpackImage(image, "game.iso", "./out");

// In-memory unpack (no disk writes; also LocalFilesystem / custom IFilesystem)
var memory = new MemoryFilesystem();
XisoReader.UnpackImage("game.iso", memory);
byte[] xbe = memory.ReadAllBytes("default.xbe");

// Extraction pre-sizes known-length destinations (LocalFilesystem: SetLength +
// SequentialScan; MemoryFilesystem: capacity) before copying (Phase 7).

// Explore an image in-process (no extraction; .iso or .cso)
var explorer = new XisoExplorer("game.iso");   // probes the volume, fails fast
foreach (var node in explorer.ListChildren("/"))
    Console.WriteLine($"{(node.IsDirectory ? "dir " : "file")} {node.FullPath} ({node.Size} B)");
explorer.CopyOut("/docs/readme.txt", "./readme.txt");
string? sha256 = explorer.ComputeHashHex("/default.xbe", HashAlgorithmName.SHA256);
XexInfo? xex = explorer.GetXexInfo("/default.xex");
XbeInfo? xbe = explorer.GetXbeInfo("/default.xbe"); // OG-Xbox cert, no reference tool parses this

// Read file bytes in place — no extraction (VFS/Dokan ReadFile hot path)
using (Stream data = explorer.OpenReadStream("/default.xex"))
{
    data.Seek(0x100, SeekOrigin.Begin);
    int n = data.Read(buffer);                 // bounded to the file's Size (.cso-aware)
}
int read = XisoReader.ReadFileBytes("game.iso", "/default.xbe", buffer, fileOffset: 0x200);

// Keep-open explorer for an app that hammers one image (mount/VFS mode):
// one held stream, serialized calls, read streams die with Dispose.
using var mounted = new XisoExplorer("game.iso",
    new XisoExplorerOptions { KeepOpen = true, Share = FileShare.ReadWrite });
VolumeInfo vol = mounted.Volume;               // IsValid, DiscFormat, FileLength, CreationTime, DescriptorSector
ExplorerNode? node = mounted.GetNode("/sub/file.bin");
using Stream nodeData = mounted.OpenReadStream(node);
FileAttributes attrs = XisoAttributes.ToWindowsFileAttributes(node.Attributes);

// Split for FATX (4 GiB default cap) and reassemble (CLI: split / join)
IReadOnlyList<string> parts = XisoReader.SplitXiso("game.iso", "game", 4L * 1024 * 1024 * 1024);
XisoReader.JoinSplitXiso(parts[0], "rejoined.iso");

// Create with exclusions (WaxGlob {0}/{n} also works for build-image)
XisoWriter.CreateXiso(
    "source_dir", "./out", null, null, out var isoPath, "game.iso", null,
    excludePatterns: ["**/*.tmp", "**/node_modules/**"]);

// Build-image ordered remapping (wax captures, negation)
var rules = new[] { new RemapRule("bin", "/"), new RemapRule("assets/**", "/assets/{1}") };
RemapFilesystem.BuildImage("./src", "out.iso", rules); // via RemapFilesystem → CreateFromRemapTree

// Deterministic image (xdvdfs stamp 0 → byte-identical output for identical input)
XisoWriter.PackFromDirectory("source_dir", "game.iso", fileTime: 0);

// Explicit sector layout + allocator (reallocation primitive for in-place patching)
SectorLayout layout = XisoReader.GetSectorLayout("game.iso");
var allocator = SectorAllocator.FromLayout(layout);
uint sector = allocator.AllocateForBytes(1_000_000); // first-fit free run, no overlaps

// Pack to a fixed layout (CLI: --layout)
XisoLayoutSpec spec = XisoLayoutSpec.FromSectorLayout(layout);
XisoWriter.PackFromDirectory("game_dir", "repacked.iso", layout: spec);

// Info
VolumeInfo info = XisoReader.GetVolumeInfo("game.iso");
Console.WriteLine($"Valid: {info.IsValid}, root sector: {info.RootDirSector}");

// Audit (flags reserved 0x48 + empty 0x0000)
AuditResult audit = XisoReader.AuditXiso("game.iso");
Console.WriteLine(audit.IsValid ? "PASS" : $"FAIL: {string.Join("; ", audit.Issues)}");

// Repair patchable issues in place (keeps .old backup; dryRun previews)
RepairResult repaired = XisoReader.Repair("game.iso");

// Rebuild reachable entries from a damaged image into a fresh audited .iso
SalvageResult salvaged = XisoReader.Salvage("game.iso"); // -> game.salvaged.iso

// Executable + disc identity (no reference tool covers XBE)
XbeInfo? cert = XisoReader.GetXbeInfo("game.iso", "/default.xbe");
Console.WriteLine($"{cert?.TitleName} [{cert?.TitleId:X8}]");
Console.WriteLine($"Layout: {XisoReader.GetVolumeInfo("game.iso").DiscFormat}");

// Block device (in-memory golden fixture)
using var dev = new MemoryBlockDevice(File.ReadAllBytes("game.iso"));
var (sector, size, lseek) = XisoReader.VerifyXiso(dev, "game.iso");

// Archival (XboxKit parity)
XisoRedump.TryExtractVideo("game.redump.iso", "game.video.iso", out _);
var filler = XisoOperations.ExtractFiller("game.iso");
XisoOperations.WipeFiller("game.iso", "wiped.iso");
XisoSkeleton.Petrify("game.iso", "skeleton.iso", "hash.txt");
int restored = XisoSkeleton.Apply("skeleton.iso", "./files", "game-restored.iso", "hash.txt");

// Redump/XGD layout inspection
XisoDiscLayout layout = XisoDiscInfo.Read("game.redump.iso");
Console.WriteLine($"{layout.XgdType} {layout.Wave}; game partition at 0x{layout.GamePartition?.ByteOffset:X}");

// Per-file hashes for any container (ISO/split/CISO/CCI/GOD/STFS/ZAR)
IReadOnlyList<FileHashEntry> hashes =
    XisoReader.ComputeFileHashSidecar("game.iso", HashAlgorithmName.SHA256);

// Name policy: check before writing
string? nameError = XisoNameRules.GetPathError("/some/very-long-component.txt");

// CISO
CisoWriter.CompressToCso("game.iso", "game.cso", level: 9);
CisoReader.DecompressToIso("game.cso", "rebuilt.iso");

// Checksum (SHA3-256, xdvdfs compat, sorted BTreeMap)
string hex = XisoChecksum.ComputeImageChecksumHex("game.iso");

// Rebuild lossless Redump
XisoRedump.RebuildRedump("game.xiso", "game.video.iso", "filler.bin", "su20076000_00000000", "rebuilt.redump.iso");

All reader paths also accept rebuilt "sector-0" XISOs (descriptor at absolute offset 0): probes report DiscLseek = 0 / DescriptorSector = 0, and explorer mounts, reads, sector ranges, ZAR packing, and in-place CopyIn work on them. RebuildRedump rejects sector-0 inputs (repack to the standard layout first).

See: CLI · Archival · xdvdfs Compat · Compression

Async and cancellation

All long-running operations accept a CancellationToken:

using var cts = new CancellationTokenSource();
(int result, string? outPath) = await XisoWriter.CreateXisoAsync(
    "source_dir", "./out", null, null, "game.iso", null, cts.Token);

Async variants: XisoReader.DecodeXisoAsync, XisoWriter.CreateXisoAsync.

Cancellation is honored per entry during extraction (an interrupted unpack throws OperationCanceledException without touching the process working directory), and extract mode reports ProgressInfoType.FileAdded per written file plus per-chunk ProgressInfoType.FileProgress while each file copies (driven by the shared XisoFileCopier core, which reuses one buffer instead of allocating per file). Outputs that collide with their inputs throw IOException before writing (input==output guard); path comparison lives in XisoPaths, resume options in UnpackOptions.

See also: XisoReader API · XisoWriter API · Utilities & Types · Building