diskimage

package module
v0.0.0-...-69ab7d0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 31, 2026 License: BSD-3-Clause Imports: 25 Imported by: 1

README

go-diskimages/diskimage

diskimage

Unified toolkit (library + CLI) for creating, converting, resizing, and reading/writing files inside VM disk images (raw / DMG-UDIF, plus QCOW2 via the CLI), across ext4/fat32/btrfs/xfs/zfs/exfat/apfs and MBR/GPT partition tables, with optional LUKS and APFS FileVault encryption. Used to prepare images for Apple Virtualization.framework VMs. GRUB-config patching lives in the sibling go-bootloaders/grub package; OCI/Tart image extraction lives in the sibling tart-oci package.

Note: the CLI command was renamed from diskimagec to diskimage.

Module

github.com/go-diskimages/diskimage

The orchestrator of the go-diskimages family: it composes dmg and qcow2 (image containers), the go-filesystems drivers (ext4, fat32, btrfs, xfs, zfs, exfat, apfs, uefi), go-fde (APFS/LUKS full-disk encryption), and go-bootloaders/grub into one CLI + library. See go.mod for exact versions.

For raw→OCI/Tart image extraction, see the sibling tart-oci package.

API

Creating images
func Create(opts CreateOptions) error

CreateOptions selects the container format (FormatRaw or FormatDmg), partition scheme (PartNone, PartMBR, PartGPT), filesystem (FSNone, FSExt4, FSFat32, FSBtrfs, FSXfs, FSZfs, FSExFAT, FSNTFS, FSApfs), volume label, and — for DMG images — the UDIF sub-format (DmgUDIFFormat) and an optional FileVault passphrase (DmgPassphrase, APFS only). QCOW2 image creation is exposed by the CLI (diskimage create qcow2) via the sibling qcow2 package directly, not through CreateOptions.

Growing, resizing, converting
func Grow(path string, sizeBytes int64) error
func ResizeImage(path string, newSizeBytes int64) error
func ConvertImageFormat(path, dstFormat string) error

ResizeImage and ConvertImageFormat are pure Go (via the dmg package's UDIF codec) and work on all platforms — see UDIF / DMG below.

File operations inside an image
func ReadFile(opts FileOptions) ([]byte, error)
func WriteFile(opts FileOptions, data []byte, perm os.FileMode) error
func Stat(opts FileOptions) (filesystem.Stat, error)
func Rename(opts FileOptions, newPath string) error
func DeleteFile(opts FileOptions) error
func DeleteDir(opts FileOptions) error
func MkDir(opts FileOptions, perm os.FileMode) error
func List(opts ListOptions) ([]ListEntry, error)

FileOptions/ListOptions name the image path, optional partition index, and (auto-detected if empty) filesystem type; ListOptions can also request populated Mode/Size/Inode via FetchStat.

Filesystem detection
func DetectFilesystem(imagePath string, partIndex int) (FilesystemType, error)

Probes magic bytes for ext4, fat32, btrfs, xfs, zfs, exfat.

UEFI variable store
func CreateUEFIVarsStore(path string, sizeBytes int64) (UEFIVarsStore, error)

UEFIVarsSizeX86_64 (512 KiB, OVMF_VARS.fd-compatible) and UEFIVarsSizeARM64 (64 MiB, QEMU_VARS.fd-compatible) give the conventional store sizes.

Volume label (ext4)
func SetExt4Label(imagePath string, partIndex int, label string) error
func Ext4Label(imagePath string, partIndex int) (string, error)

Reads or writes the ext4 s_volume_name of the partition at partIndex. Works on raw, QCOW2, and UDIF-UDRW DMG inputs via the unified OpenBlockDevice dispatcher, bridged into the ext4 driver via a small in-package ext4Adapter. Label capped at 16 bytes; the metadata-csum is refreshed using the kernel-canonical CRC-32C convention (crc32c(~0, sb[:0x3FC]), no final XOR). Use it offline — concurrent writers may produce a torn superblock.

Block device access

OpenBlockDevice opens a raw, QCOW2, or UDIF-UDRW DMG image as a uniform BlockDevice:

dev, err := diskimage.OpenBlockDevice("disk.qcow2")
if err != nil { log.Fatal(err) }
defer dev.Close()

buf := make([]byte, 512)
dev.ReadAt(buf, 0)

The BlockDevice interface exposes ReadAt, WriteAt, Size, and Close. The implementation auto-detects QCOW2 by magic bytes; UDIF DMG by the koly trailer; everything else is treated as a raw file.

UDIF support is UDRW-only in-place — Read/Write are bounded to the data fork (sector area at file offset 0), and Close refreshes the koly trailer's dataForkChecksum + masterChecksum if anything was written so the next readAllUDIFSectors open passes its master checksum verification. Compressed UDIF subformats (UDRO / UDZO / UDBZ / UDSP) are rejected with an error pointing at dmg.UnpackToTemp

  • PackFromTemp for the explicit unpack-edit-repack path.
LUKS block devices

OpenLUKSBlockDevice layers LUKS decryption on top of a raw or QCOW2 image:

dev, err := diskimage.OpenLUKSBlockDevice("disk.luks.qcow2", []byte("pass"))
if err != nil { log.Fatal(err) }
defer dev.Close()   // closes LUKS and the underlying QCOW2 device

buf := make([]byte, 4096)
dev.ReadAt(buf, 0)          // reads from the decrypted LUKS payload
dev.WriteAt(buf, 0)         // encrypts and writes back

Supported backing formats:

Format How detected
Raw file fallback (any path not recognized as QCOW2)
QCOW2 magic bytes QFI\xfb at offset 0

Filesystem detection inside a backing image (see detect.go):

Filesystem How detected
APFS NX SuperBlock magic "NXSB" at offset 32 of block 0
ext4 superblock magic 0xEF53 at offset 0x438
FAT32 OEM "MSWIN4.1" / FAT32 BPB signature in the boot sector
NTFS OEM "NTFSIMG1" (test driver) or "NTFS " (real NTFS)
exFAT OEM "EXFAT " at bytes 3-10 of the boot sector
btrfs superblock magic _BHRfS_M at offset 0x10040
XFS superblock magic XFSB at offset 0
ZFS vdev label magic across blocks 0/256K/end

For raw files the LUKS header is opened directly from the file path.
For QCOW2 files, the QCOW2 container is opened first and LUKS reads/writes through it. dev.Close() closes both layers.

dev.Size() returns the size of the decrypted payload reported by the LUKS device. For LUKS1 this is 0 (the LUKS1 format does not encode the payload length in its header); for LUKS2 it reflects the configured payload length.

Build requirements

Pure Go (CGO_ENABLED=0), cross-platform, all six supported 64-bit architectures. A handful of darwin-only integration tests (dmg_convert_resize_integration_test.go, grub_test.go) additionally cross-check the pure-Go UDIF codec against real hdiutil output and the go-bootloaders/grub patch helpers against real GRUB behavior on macOS; they build under the darwin tag but are not part of the public API and are not required to build or use the library on any platform.

Used by

  • weft (github.com/openweft) — extracts and prepares VM disk images for Apple Virtualization.framework VMs

Notes

  • ZFS images: ZFS is supported on whole-disk and partitioned images. The ZFS driver will attempt to handle fat‑ZAP directory blocks by converting them to micro‑ZAP when possible. Conversion behaviour:
    • If the ZAP header block is zeroed, a micro‑ZAP is written in-place (no new allocation).
    • Otherwise the driver allocates a new block for the micro‑ZAP and updates the directory dnode to point to it.
    • This is a pragmatic compatibility path (avoids implementing full fat‑ZAP write support) and has limitations — very large directories or long names may exceed micro‑ZAP capacity and cause write errors.
  • Partitioned images: allocation offsets are partition-aware; writes are performed at partition base + allocator offset to ensure correct physical placement inside partitioned images.
  • Debugging: the integration stress test's iteration count is configurable via the DISKIMAGE_STRESS_ITERS environment variable (see integration/RESOURCES.md).

ZFS diagnostic commands

Use these commands to inspect ZFS state and relevant files on the host:

# show pool health and errors
zpool status

# list datasets and properties
zfs list

# print a file's contents (replace <file_path> with the real path)
cat <file_path>

Example (run integration stress with ZFS/Btrfs debug enabled):

DISKIMAGE_STRESS_ITERS=1 \
	go test ./integration -run TestStress_FileOperations_AllFS_AllPartitions -v

See OpenZFS docs for low-level format details: https://openzfs.github.io/openzfs-docs/

UDIF / DMG

  • DMG is an Apple UDIF container which can be created in several forms:

    • UDRW (read-write): fixed-size at creation but resizable in-place.
    • UDSP (sparseimage / sparsebundle): grows automatically as files are added.

  • This module exposes helpers, backed by the pure-Go dmg codec, to detect, convert and resize UDIF images on any platform — no hdiutil/plutil required:

    • CreateOptions.DmgUDIFFormat: when creating a DMG with Create, set this field to UDRW or UDSP to request the underlying UDIF format (defaults to UDRW).
    • ConvertImageFormat(path, dstFormat string) error — convert an image in-place between UDIF formats (e.g. UDRW ↔ UDSP).
    • ResizeImage(path string, newSizeBytes int64) error — resize an image:
      • For UDIF images, uses the pure-Go UDIF resize (UDRW only).
      • For UDSP images the image must first be converted to UDRW.
      • For raw files the function falls back to truncation via Grow.

Notes and caveats:

  • Converting UDSP → UDRW is supported, but resizing UDSP in-place is not — convert to UDRW first if you need to resize.

Example: create a UDRW APFS DMG and then convert to sparseimage and resize

opts := diskimage.CreateOptions{
		Path: "/tmp/test.dmg",
		SizeBytes: 10 << 20,
		Format: diskimage.FormatDmg,
		Filesystem: diskimage.FSApfs,
		Label: "Example",
		DmgUDIFFormat: "UDRW",
}
if err := diskimage.Create(opts); err != nil {
		log.Fatal(err)
}

if err := diskimage.ConvertImageFormat(opts.Path, "UDSP"); err != nil {
		log.Fatal(err)
}

if err := diskimage.ConvertImageFormat(opts.Path, "UDRW"); err != nil {
		log.Fatal(err)
}

if err := diskimage.ResizeImage(opts.Path, 20*1024*1024); err != nil {
		log.Fatal(err)
}

Run the darwin-only integration tests to exercise these features locally:

GOWORK=off go test . -run TestCreateConvertAndResizeUDIF -v

Documentation

Index

Constants

View Source 
const (
	FormatRaw ImageFormat = "raw"
	FormatDmg ImageFormat = "dmg"

	PartNone PartitionScheme = "none"
	PartMBR  PartitionScheme = "mbr"
	PartGPT  PartitionScheme = "gpt"

	FSNone    FilesystemType = "none"
	FSExt4    FilesystemType = "ext4"
	FSFat32   FilesystemType = "fat32"
	FSBtrfs   FilesystemType = "btrfs"
	FSXfs     FilesystemType = "xfs"
	FSZfs     FilesystemType = "zfs"
	FSExFAT   FilesystemType = "exfat"
	FSNTFS    FilesystemType = "ntfs"
	FSApfs    FilesystemType = "apfs"
	FSHfsPlus FilesystemType = "hfsplus"
)
View Source 
const UEFIVarsSizeARM64 = filesystem_uefi.VarsSizeARM64

UEFIVarsSizeARM64 is the standard QEMU_VARS.fd size for arm64 (64 MiB).

View Source 
const UEFIVarsSizeX86_64 = filesystem_uefi.VarsSizeX86_64

UEFIVarsSizeX86_64 is the standard OVMF_VARS.fd size for x86-64 (512 KiB).

Variables

View Source 
var (
	// ValidFormats lists accepted --format values.
	ValidFormats = []string{"raw", "dmg"}
	// ValidPartSchemes lists accepted --part values.
	ValidPartSchemes = []string{"none", "mbr", "gpt"}
	// ValidFilesystems lists accepted --filesystem values.
	ValidFilesystems = []string{"none", "ext4", "fat32", "btrfs", "xfs", "zfs", "exfat", "ntfs", "apfs", "hfsplus"}
	// DmgSupportedFilesystems is the set of filesystems that may be embedded
	// in a DMG image. Only filesystems natively understood by macOS are
	// allowed: APFS (with optional FileVault FDE), HFS+, FAT32 and exFAT.
	//
	// HFS+ is what a distribution .dmg wants. It compresses far better under
	// UDZO than APFS, and the Finder metadata that gives such an image its
	// background picture and icon positions keys on an "H+" volume — the
	// alias inside the .DS_Store record names the filesystem type.
	DmgSupportedFilesystems = []string{"none", "apfs", "hfsplus", "fat32", "exfat"}
)

Functions

func ConvertImageFormat 

func ConvertImageFormat(path, dstFormat string) error

ConvertImageFormat converts the image at path to the given format (e.g. "UDZO" or "UDSP") in-place. Works on all platforms.

"UDRW" is the raw image, with no UDIF container: that is what hdiutil writes for it and the only shape macOS mounts read/write. A raw image is accepted as the source too, so the conversion goes both ways.

func Create 

func Create(opts CreateOptions) error

Create creates a disk image according to opts.

For --part none the filesystem is written directly at offset 0 of the image. For --part mbr/gpt the partition table is written first, then the filesystem is written at the partition offset (first partition, aligned to 1 MiB).

func DeleteDir 

func DeleteDir(opts FileOptions) error

DeleteDir removes a directory (recursively) inside the image.

func DeleteFile 

func DeleteFile(opts FileOptions) error

DeleteFile removes a file at the given path inside the image.

func Ext4Label 

func Ext4Label(imagePath string, partIndex int) (string, error)

Ext4Label reads the current volume label from the ext4 filesystem at imagePath/partIndex. Convenience helper for callers that want to confirm an image's label without round-tripping through e2label. Returns the empty string when no label has been set.

func Grow 

func Grow(path string, sizeBytes int64) error

Grow resizes the backing image file at path to sizeBytes. It behaves like truncate: the file is grown or shrunk to the given size.

func IsDmgSupportedFilesystem 

func IsDmgSupportedFilesystem(fs FilesystemType) bool

IsDmgSupportedFilesystem reports whether fs is allowed inside a DMG image.

func MkDir 

func MkDir(opts FileOptions, perm os.FileMode) error

MkDir creates a directory inside the image.

func ReadFile 

func ReadFile(opts FileOptions) ([]byte, error)

ReadFile reads and returns the contents of a regular file inside an image.

func Rename 

func Rename(opts FileOptions, newPath string) error

Rename moves or renames a file or directory inside the image.

func ResizeImage 

func ResizeImage(path string, newSizeBytes int64) error

ResizeImage resizes a disk image:

  • UDIF images: uses the pure-Go UDIF resize, works on all platforms, and puts the image back in the shape it was found in. A sparse image stays sparse; it used to be refused here because growing one silently rewrote it uncompressed.
  • Raw files: truncates to new size via Grow.

func SetExt4Label 

func SetExt4Label(imagePath string, partIndex int, label string) error

SetExt4Label opens the ext4 filesystem at imagePath/partIndex and writes a new s_volume_name. Works on both raw and QCOW2 images: OpenBlockDevice handles format detection, and the ext4 driver does the on-disk superblock write directly (no journal). When the filesystem has metadata_csum enabled the superblock CRC32c is recomputed automatically.

The label is capped at filesystem_ext4.MaxLabelLen (16 bytes); a longer label is rejected rather than silently truncated. Pass an empty string to clear the label.

SetExt4Label is intended for offline image manipulation (e.g. stamping a label on a cloud image before booting it under cloud- boot's `device = "LABEL=…"` plan target). Don't call it concurrently with a process actively mutating the same image — see filesystem_ext4.(*ext4FS).SetLabel for the locking story.

func Stat 

func Stat(opts FileOptions) (filesystem.Stat, error)

Stat returns filesystem metadata for the given path inside the image.

func WriteFile 

func WriteFile(opts FileOptions, data []byte, perm os.FileMode) error

WriteFile writes data to a file inside the image, creating parent directories as needed.

Types

type BlockDevice 

type BlockDevice interface {
	io.ReaderAt
	WriteAt(p []byte, off int64) (int, error)
	// Size returns the usable byte count of the device.
	Size() int64
	// Close releases all resources associated with the device.
	Close() error
}

BlockDevice is a read-write block device backed by a disk image file. It is the common interface returned by OpenBlockDevice and OpenLUKSBlockDevice.

func OpenAPFSBlockDevice 

func OpenAPFSBlockDevice(path string, passphrase []byte) (BlockDevice, error)

OpenAPFSBlockDevice opens a raw or QCOW2 disk image that contains a FileVault-encrypted APFS container. The image format is detected automatically; the APFS key bag embedded in the container is unlocked with passphrase.

The returned BlockDevice transparently decrypts / encrypts all I/O with the Volume Encryption Key recovered from the container key bag.

Supported container formats:

  • raw — the file itself is an APFS container (NX superblock at byte 0)
  • qcow2 — the virtual disk exported by the QCOW2 layer begins with an APFS container; I/O is translated through both the QCOW2 and APFS layers

func OpenBlockDevice 

func OpenBlockDevice(path string) (BlockDevice, error)

OpenBlockDevice opens a disk image as a read-write block device. The format is detected automatically from the file header:

  • QCOW2 → reads/writes go through the QCOW2 copy-on-write layer.
  • UDIF DMG whose data fork is uncompressed → reads/writes hit it in place; the koly trailer's CRC-32 checksums are refreshed on Close. Compressed and sparse images are refused — they need an unpack-pack strategy that has a surprise Close cost (see dmg.UnpackToTemp / PackFromTemp if you want it explicitly).
  • everything else → treated as a raw image; the device reads and writes the file directly.

func OpenLUKSBlockDevice 

func OpenLUKSBlockDevice(path string, passphrase []byte) (BlockDevice, error)

OpenLUKSBlockDevice opens a raw or QCOW2 disk image that contains a LUKS encrypted payload. The image format is detected automatically; the LUKS container embedded in the virtual disk is unlocked with passphrase.

The returned BlockDevice transparently encrypts/decrypts all I/O.

Supported container formats:

  • raw — the file itself is a LUKS device (LUKS header at byte 0)
  • qcow2 — the virtual disk exported by the QCOW2 layer begins with a LUKS header; I/O is translated through both the QCOW2 and LUKS layers

type CreateOptions 

type CreateOptions struct {
	// Path is the output file path.
	Path string
	// SizeBytes is the total image size in bytes.
	SizeBytes int64
	// Format is the image container format (default: raw).
	Format ImageFormat
	// Partition is the partition table scheme (default: none).
	Partition PartitionScheme
	// Filesystem is the filesystem to write in the (first) partition or bare
	// image. Use FSNone to create an image/partition without any filesystem.
	Filesystem FilesystemType
	// Label is passed to the filesystem formatter as volume label / pool name.
	Label string
	// DmgUDIFFormat selects the UDIF format the image is written in — "UDRW"
	// (raw, the default), "UDZO" (zlib), or "UDSP" (sparse, zero runs
	// elided). A distribution image wants UDZO: an HFS+ volume that is mostly
	// empty compresses to a small fraction of its size, and macOS mounts it
	// read-only, which is what a downloaded .dmg should be.
	DmgUDIFFormat string
	// DmgPassphrase, when non-empty and Format=FormatDmg with Filesystem=FSApfs,
	// requests a FileVault-encrypted (FDE) APFS DMG. The passphrase is fed to
	// pkg/go-fde/apfs which writes the APFS NX superblock and key bag using
	// PBKDF2-SHA256 + AES Key Wrap (RFC 3394) over AES-256-XTS payload. It is
	// only used in-process (never persisted in argv).
	DmgPassphrase []byte
}

CreateOptions configures the disk image to create.

type FileOptions 

type FileOptions struct {
	// Path is the path to the disk image file.
	Path string
	// Filesystem selects which filesystem driver to use. If empty, the
	// filesystem type is auto-detected from the image.
	Filesystem FilesystemType
	// PartIndex selects the 0-based partition index to open.
	PartIndex int
	// FilePath is the absolute path inside the filesystem (e.g. "/etc/hosts").
	FilePath string
	// UnlockKeys provides optional keys used to unlock encrypted APFS images.
	// For bi-key encrypted images provide two keys.
	UnlockKeys []string
}

FileOptions configures ReadFile/WriteFile operations against an image.

type FilesystemType 

type FilesystemType string

FilesystemType identifies which filesystem to write inside the image.

func DetectFilesystem 

func DetectFilesystem(imagePath string, partIndex int) (FilesystemType, error)

DetectFilesystem probes the magic bytes in a disk image and returns the filesystem type found in the partition at partIndex. It supports: ext4, fat32, btrfs, xfs, zfs, exfat.

type ImageFormat 

type ImageFormat string

ImageFormat is the container format for the disk image.

type ListEntry 

type ListEntry struct {
	Name     string
	FileType uint8  // raw filesystem-specific type byte
	Mode     uint16 // POSIX mode (0 if not fetched)
	Size     uint64 // file size in bytes (0 if not fetched)
	Inode    uint64 // inode number (0 if not fetched)
}

ListEntry is a single entry returned by List.

func List 

func List(opts ListOptions) ([]ListEntry, error)

List opens a disk image and returns the directory entries at DirPath. If opts.Filesystem is empty, the filesystem type is auto-detected.

type ListOptions 

type ListOptions struct {
	// Path is the path to the disk image file.
	Path string
	// Filesystem selects which filesystem driver to use.
	// If empty, the filesystem type is auto-detected from the image.
	Filesystem FilesystemType
	// PartIndex is the 0-based partition index to open (0 for unpartitioned images).
	PartIndex int
	// DirPath is the directory inside the filesystem to list (default "/").
	DirPath string
	// FetchStat populates Mode, Size and Inode on each ListEntry.
	FetchStat bool
	// UnlockKeys provides optional keys used to unlock encrypted APFS images.
	UnlockKeys []string
}

ListOptions controls the List operation.

type PartitionScheme 

type PartitionScheme string

PartitionScheme describes the partition table placed in the image.

type UEFIVarsStore 

type UEFIVarsStore = filesystem_uefi.Store

UEFIVarsStore is the handle returned by CreateUEFIVarsStore. It embeds the full uefi.Store interface so callers can read/write variables and enroll Secure Boot keys before handing the file to QEMU.

func CreateUEFIVarsStore 

func CreateUEFIVarsStore(path string, sizeBytes int64) (UEFIVarsStore, error)

CreateUEFIVarsStore creates a new UEFI NvVar variable store file at path with the given size in bytes.

The file is created (or truncated) and initialised with a valid VARIABLE_STORE_HEADER followed by erased-flash bytes (0xFF), matching the format expected by QEMU pflash images (OVMF_VARS.fd for x86-64, QEMU_VARS.fd for arm64).

Typical sizes: 

x86-64 : UEFIVarsSizeX86_64 (512 KiB)
arm64  : UEFIVarsSizeARM64  (64 MiB)

The returned UEFIVarsStore is open and ready for use; call Close when done.

Source Files

View all Source files

Directories

Path Synopsis
cmd
diskimage command
diskimage is a command-line tool for creating and managing VM disk images.
 diskimage is a command-line tool for creating and managing VM disk images.
diskimage/copy
diskimage/create
Package create implements the diskimage create sub-commands.
 Package create implements the diskimage create sub-commands.
diskimage/exec
diskimage/grow
Package grow implements the diskimage grow sub-command.
 Package grow implements the diskimage grow sub-command.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.