data

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 16, 2021 License: Apache-2.0 Imports: 7 Imported by: 0

README

Data Transformation

This repository contains various data extraction, transformation processing and visualization tools. Currently it contains the following:

  • data.Table provides you with a way to ingest, transform and process data tables in comma-separated value format and output in CSV, ASCII and SQL formats;
  • data.DOM provides a document object model which can read and write the XML format in addition to validating the XML;
  • data.Canvas provides a drawing canvas on which graphics primitives such as lines, circles, text and rectangles can be placed. Additionally transformation, grouping and stylizing of primitives can be applied. Canvases can currently be written in SVG format, the intention is to also allow rendering using OpenGL later.

Documentation

I have published the documention at data.mutablelogic.com. You can also see the following useful sources of information:

Project Status

This module is currently in development but is mostly feature-complete.

Contributing and Filing Issues

  • File an issue or question on github.
  • Feel free to fork this repository. Any pull requests are gratefully received. Licensed under Apache 2.0, please read that license about using, distribution and forking. Licensed works, modifications, and larger works may be distributed under different terms and without source code.

License

This repository is released under the Apache license:

Copyright 2021 David Thorpe and all other authors of the software

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Documentation

Index

Constants

View Source
const (
	XmlNamespaceSVG   = "http://www.w3.org/2000/svg"   // Scalable Vector Graphics
	XmlNamespaceXLink = "http://www.w3.org/1999/xlink" // XLink
	XmlNamespaceXHTML = "http://www.w3.org/1999/xhtml" // XHTML
)
View Source
const (
	BorderDefault = "+++++++++|-"
	BorderLines   = "┌┬┐├┼┤└┴┘│─"
)

Variables

View Source
var (
	ZeroSize  = Size{0, 0}
	ZeroPoint = Point{0, 0}
	NilPoint  = Point{float32(math.NaN()), float32(math.NaN())}
	NilSize   = Size{float32(math.NaN()), float32(math.NaN())}
)
View Source
var (
	// Paper sizes in mm
	A4PortraitSize      = Size{210, 297}
	A4LandscapeSize     = Size{297, 210}
	LetterPortraitSize  = Size{215.9, 279.4}
	LetterLandscapeSize = Size{279.4, 215.9}
	LegalPortraitSize   = Size{215.9, 355.6}
	LegalLandscapeSize  = Size{355.6, 215.9}
)

Functions

func Float32

func Float32(v interface{}) float32

Float32 returns a float value from untyped or returns NaN otherwise

Types

type Canvas

type Canvas interface {

	// Get and set canvas viewbox
	Origin() Point
	Size() Size
	SetViewBox(Point, Size) error

	// Set canvas properties
	Title(string) Canvas
	Version(string) Canvas

	// Write to data stream
	Write(Writer, io.Writer) error
}

type CanvasElement

type CanvasElement interface {
	Id(string) CanvasElement
	Class(string) CanvasElement
	Style(...CanvasStyle) CanvasElement
	Transform(...CanvasTransform) CanvasElement
}

type CanvasGroup

type CanvasGroup interface {
	CanvasElement

	Desc(string) CanvasGroup
	Group(...CanvasElement) CanvasGroup
}

type CanvasPath

type CanvasPath interface {
	CanvasElement

	MoveTo(Point) CanvasPath
	LineTo(Point) CanvasPath
	QuadraticTo(pt, c Point) CanvasPath
	CubicTo(pt, c1, c2 Point) CanvasPath
	ClosePath() CanvasPath
}

type CanvasStyle

type CanvasStyle interface{}

type CanvasText

type CanvasText interface{}

type CanvasTransform

type CanvasTransform interface{}

type Color

type Color struct {
	R, G, B uint8
}

Color represents an 24-bit RGB colour without opacity

type CompareFunc

type CompareFunc func(a, b []interface{}) bool

type DOMOption

type DOMOption uint8
const (
	DOMWriteDirective    DOMOption = (1 << iota) // Write <?xml?> at top
	DOMWriteIndentTab                            // Indent output with tabs
	DOMWriteIndentSpace2                         // Indent output with two spaces
)

func (DOMOption) Is

func (o DOMOption) Is(f DOMOption) bool

type DOMValidateNodeFunc

type DOMValidateNodeFunc func(Node) error

type Document

type Document interface {
	Node

	// Return nodes
	GetElementById(string) Node
	GetElementByIdNS(string, string) Node

	// Create nodes
	CreateElement(string) Node
	CreateElementNS(string, string) Node
	CreateText(string) Node
	CreateComment(string) Node

	// Write XML
	Write(io.Writer) error
}

type Error

type Error uint
const (
	ErrNone Error = iota
	ErrBadParameter
	ErrDuplicateEntry
	ErrInternalAppError
	ErrSkipTransform
	ErrNotImplemented
	ErrNotFound
)

func (Error) Error

func (e Error) Error() string

func (Error) WithPrefix

func (e Error) WithPrefix(args ...interface{}) error

type FillRule

type FillRule int
const (
	NonZero FillRule = iota
	EvenOdd
)

func (FillRule) String

func (r FillRule) String() string

type IteratorFunc

type IteratorFunc func(int, []interface{}) error

type Labels

type Labels interface {
	Set

	// Append label to the set
	Append(string)
}

type LineCap

type LineCap int
const (
	CapButt LineCap = iota
	CapRound
	CapSquare
)

func (LineCap) String

func (c LineCap) String() string

type LineJoin

type LineJoin int
const (
	JoinMiter LineJoin = iota
	JoinMiterClip
	JoinArcs
	JoinRound
	JoinBevel
)

func (LineJoin) String

func (j LineJoin) String() string

type Node

type Node interface {
	Name() xml.Name
	Cdata() string
	Parent() Node
	GetElementsByTagName(string) []Node
	GetElementsByTagNameNS(string, string) []Node

	Children() []Node
	AddChild(Node) error
	RemoveChild(Node) error
	FirstChild() Node
	LastChild() Node
	InsertChildBefore(Node, Node) error
	RemoveAllChildren() error

	Attrs() []xml.Attr
	Attr(string) (xml.Attr, bool)
	AttrNS(string, string) (xml.Attr, bool)
	SetAttr(string, string) error
	SetAttrNS(string, string, string) error
}

type Point

type Point struct {
	X, Y float32
}

type Points

type Points interface {
	Set

	// Append point to the set
	Append(Point)
}

type Scale

type Scale interface {
	// Return name associated with the scale
	Name() string

	// Return minimum represented value on scale
	Min() float32

	// Return maximum represented value on scale
	Max() float32

	// Write scale to canvas
	WritePath(Canvas) CanvasGroup
}

Scale is an X or Y scale (currently linear) which can represent all values

type Series

type Series interface {
	// Read series from a table, the iterator function can return
	// ErrSkipTransform if a returned values can be discarded. The
	// values returned should be float32, data.Point or string
	// which respectively appends Values, Points and Labels
	Read(Table, SeriesIteratorFunc) error

	// Sets returns sets contained with the series
	Sets() []Set
}

type SeriesIteratorFunc

type SeriesIteratorFunc func(int, []interface{}) ([]interface{}, error)

type Set

type Set interface {
	// Return name associated with the set
	Name() string

	// Len returns the length of the set
	Len() int
}

type Size

type Size struct {
	W, H float32
}

func (Size) UnitString

func (s Size) UnitString(u Unit) (string, string)

type Table

type Table interface {
	// Read CSV data with table options
	Read(io.Reader, ...TableOpt) error

	// Write data with table options
	Write(io.Writer, ...TableOpt) error

	// Append a row to the table
	Append(...interface{})

	// Len returns the number of rows
	Len() int

	// Row returns values for a zero-indexed table row
	// or nil if the row does not exist
	Row(int) []interface{}

	// Col returns column information for a zero-indexed table column
	// or nil if the column doesn't exist
	Col(int) TableCol

	// Sort sorts the rows using a comparison function, which should return
	// true if the first argument is less than the second argument
	Sort(CompareFunc)

	// OptHeader used on Read to indicate there is a CSV header and
	// with Write to output header in addition to data. Ignored for
	// ForMap and ForArray
	OptHeader() TableOpt

	// OptTransform used on Read, or Write transforms a value.
	// Several transform functions can be used in series on a value.
	// Transformation functions are called in series until nil or
	// error returned. If ErrSkipTransform is returned, the next
	// transformation is tried
	OptTransform(...TransformFunc) TableOpt

	// OptType used on Read to indicate transformation of values into
	// native types. If a type cannot be transformed, string is used.
	// Use "DefaultTypes" for all types, and "Nil" to interpret empty strings
	// as nil values. For example:
	//
	//   t := data.NewTable(data.ZeroSize)
	//   t.Read(os.Stdin,data.OptType(data.DefaultTypes|data.Nil))
	//
	OptType(Type) TableOpt

	// OptAscii used on Write to output ASCII table instead of CSV. Option
	// is ignored for Read. Arguments are maximum table width and the border
	// characters (data.BorderDefault or data.BorderLines). Setting width
	// to zero makes unconstrained width, setting string to empty sets default
	// table line output
	OptAscii(uint, string) TableOpt

	// OptCsv used to Read or Write CSV files, with delimiter character, or if
	// zero then comma is used
	OptCsv(rune) TableOpt

	// OptSql used to Write SQL format with the provided table name. The output
	// can be directly ingested by SQLite. Including OptHeader() option will also
	// include a statement to create the table
	OptSql(string) TableOpt

	// OptDuration used on Read to interpret values into durations (h,m,s,ms,ns)
	// and truncate to the provided duration
	OptDuration(time.Duration) TableOpt

	// OptTimezone used on Read to set timezone for dates and times which do not
	// include timezone explicitly. If timezone is nil, current local timezone is used.
	OptTimezone(tz *time.Location) TableOpt

	// OptRowIterator called on read before appending row to table. If error returned
	// by iterator function is ErrSkipTransform then the row is not appended to the table
	OptRowIterator(IteratorFunc) TableOpt
}

type TableCellFlag

type TableCellFlag uint
const (
	Bold TableCellFlag = (1 << iota)
)

type TableCol

type TableCol interface {
	// Name returns the column name
	Name() string

	// Type returns the types that the column represents
	Type() Type

	// Min returns the minimum value of all column numbers or zero
	Min() float64

	// Max returns the maximum value of all column numbers or zero
	Max() float64

	// Sum returns the sum of all column numbers or zero
	Sum() float64

	// Count returns the count of all column numbers
	Count() uint64

	// Mean returns the mean average value of all column numbers
	// or +Inf if no numbers in the column
	Mean() float64
}

TableCol represents information about a table column

type TableOpt

type TableOpt func(Table)

type TextAlign

type TextAlign int
const (
	Start TextAlign = iota
	Middle
	End
)

func (TextAlign) String

func (ta TextAlign) String() string

type TransformFunc

type TransformFunc func(int, int, interface{}) (interface{}, error)

type Type

type Type uint
const (
	Nil Type = (1 << iota)
	String
	Uint
	Int
	Float
	Duration
	Date
	Datetime
	Bool
	Other
	NumberTypes  = Uint | Int | Float
	DefaultTypes = NumberTypes | Duration | Date | Datetime | Bool
	TypeMin      = Nil
	TypeMax      = Other
)

func (Type) FlagString

func (t Type) FlagString() string

func (Type) Is

func (t Type) Is(f Type) bool

Is returns true if all types are represented. For example.

t.Is(Nil) returns true if type contains nil
t.Is(Nil|Int) returns true if type is Int or Nil

func (Type) String

func (t Type) String() string

func (Type) Type

func (t Type) Type() (Type, bool)

Type returns a single type which can represent a set of types, and also returns true if the columns includes nil values

type Unit

type Unit int
const (
	None Unit = iota
	PX
	CM
	MM
	IN
	PC
	PT
	EX
	EM
)

func (Unit) String

func (u Unit) String() string

type Values

type Values interface {
	Set

	// Append value to the set
	Append(float32)

	// Calculate a scale which can represent all values
	Scale() Scale
}

type Viz

type Viz interface {
	// Draw a grid on X axis with major & minor divisions,
	// use zero for no major and/or minor
	XGrid(int, int) VizGroup

	// Draw a grid on Y axis with major & minor divisions
	YGrid(int, int) VizGroup
}

type VizGroup

type VizGroup interface {
	CanvasGroup
}

type Writer

type Writer int
const (
	SVG Writer = iota
)

Directories

Path Synopsis
cmd
colorswatch command
csvreader command
tiger command
pkg
dom
f32
geom
Geometry functions
Geometry functions
viz

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL