README
¶
asm
This library for assembly and disassembly of tar archives, facilitated by
github.com/vbatts/tar-split/tar/storage.
Concerns
For completely safe assembly/disassembly, there will need to be a Content
Addressable Storage (CAS) directory, that maps to a checksum in the
storage.Entity of storage.FileType.
This is due to the fact that tar archives can allow multiple records for the same path, but the last one effectively wins. Even if the prior records had a different payload.
In this way, when assembling an archive from relative paths, if the archive has multiple entries for the same path, then all payloads read in from a relative path would be identical.
Thoughts
Have a look-aside directory or storage. This way when a clobbering record is encountered from the tar stream, then the payload of the prior/existing file is stored to the CAS. This way the clobbering record's file payload can be extracted, but we'll have preserved the payload needed to reassemble a precise tar archive.
clobbered/path/to/file.[0-N]
alternatively
We could just not support tar streams that have clobbering file paths. Appending records to the archive is not incredibly common, and doesn't happen by default for most implementations. Not supporting them wouldn't be a security concern either, as if it did occur, we would reassemble an archive that doesn't validate signature/checksum, so it shouldn't be trusted anyway.
Otherwise, this will allow us to defer support for appended files as a FUTURE FEATURE.
Documentation
¶
Overview ¶
Package asm provides the API for streaming assembly and disassembly of tar archives.
Using the `github.com/vbatts/tar-split/tar/storage` for Packing/Unpacking the metadata for a stream, as well as an implementation of Getting/Putting the file entries' payload.
Index ¶
- func IterateHeaders(unpacker storage.Unpacker, handler func(hdr *tar.Header) error) error
- func NewInputTarStream(r io.Reader, p storage.Packer, fp storage.FilePutter) (io.Reader, error)deprecated
- func NewInputTarStreamWithDone(r io.Reader, p storage.Packer, fp storage.FilePutter) (io.ReadCloser, <-chan error, error)
- func NewOutputTarStream(fg storage.FileGetter, up storage.Unpacker) io.ReadCloser
- func WriteOutputTarStream(fg storage.FileGetter, up storage.Unpacker, w io.Writer) error
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func IterateHeaders ¶ added in v0.11.6
IterateHeaders calls handler for each tar header provided by Unpacker
func NewInputTarStream
deprecated
NewInputTarStream wraps the Reader stream of a tar archive and provides a Reader stream of the same.
In the middle it will pack the segments and file metadata to storage.Packer `p`.
The storage.FilePutter is where payload of files in the stream are stashed. If this stashing is not needed, you can provide a nil storage.FilePutter. Since the checksumming is still needed, then a default of NewDiscardFilePutter will be used internally
If callers need to be able to abort early and/or wait for goroutine termination, prefer NewInputTarStreamWithDone.
Deprecated: This leaves a goroutine around if the consumer aborts without consuming the whole stream, and does not allow the caller to know when r is safe to deallocate or when p has written everything. Use NewInputTarStreamWithDone instead.
func NewInputTarStreamWithDone ¶ added in v0.12.3
func NewInputTarStreamWithDone(r io.Reader, p storage.Packer, fp storage.FilePutter) (io.ReadCloser, <-chan error, error)
NewInputTarStreamWithDone wraps the Reader stream of a tar archive and provides a Reader stream of the same.
In the middle it will pack the segments and file metadata to storage.Packer `p`.
It also returns a done channel that will receive exactly one error value (nil on success) when the internal goroutine has fully completed parsing the tar stream (including the final paddingChunk draining loop) and has finished writing all entries to `p`.
The returned reader is an io.ReadCloser so callers can stop early; closing it aborts the pipe so the internal goroutine can terminate promptly (rather than hanging on a blocked pipe write).
The caller is expected to consume the returned reader fully until EOF (not just the tar EOF marker); closing the returned reader earlier will cause the done channel to return a failure.
func NewOutputTarStream ¶
func NewOutputTarStream(fg storage.FileGetter, up storage.Unpacker) io.ReadCloser
NewOutputTarStream returns an io.ReadCloser that is an assembled tar archive stream.
It takes a storage.FileGetter, for mapping the file payloads that are to be read in, and a storage.Unpacker, which has access to the rawbytes and file order metadata. With the combination of these two items, a precise assembled Tar archive is possible.
func WriteOutputTarStream ¶ added in v0.9.11
WriteOutputTarStream writes assembled tar archive to a writer.
Types ¶
This section is empty.
Source Files