walker

package
v1.0.24 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 20, 2026 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ScoreHelmChartDefinition = 100
	ScoreKustomization       = 90
	ScoreDockerCompose       = 80
	ScoreMesheryDesign       = 70
	ScoreChartArchive        = 60
	ScoreGenericYAML         = 20
	ScoreGenericJSON         = 10
)

Relevance scores assigned to a candidate from its path alone. A larger score ranks earlier. They are exported so that callers rendering a picker can group or threshold candidates the same way the walker ranks them.

Variables

View Source 
var (
	ErrInvalidSizeFileCode = "meshkit-11241"
	ErrCloningRepoCode     = "meshkit-11242"
	ErrResolvingGitRefCode = "meshkit-11328"
	ErrFetchingGitTreeCode = "meshkit-11329"
	ErrFetchingGitBlobCode = "meshkit-11330"
	ErrInvalidBaseURLCode  = "meshkit-11331"

	ErrNoFileInterceptorCode = "meshkit-11332"
	ErrRootNotFoundCode      = "meshkit-11333"
	ErrSymlinkedRootCode     = "meshkit-11334"
)

Functions

func ErrCloningRepo 

func ErrCloningRepo(err error) error

func ErrFetchingGitBlob added in v1.0.24 

func ErrFetchingGitBlob(err error, path string) error

ErrFetchingGitBlob is returned when a selected blob could not be downloaded.

func ErrFetchingGitTree added in v1.0.24 

func ErrFetchingGitTree(err error, ref string) error

ErrFetchingGitTree is returned when the recursive Git Trees API call fails.

func ErrInvalidBaseURL added in v1.0.24 

func ErrInvalidBaseURL(err error, baseURL string) error

ErrInvalidBaseURL is returned when the configured base URL cannot be used for the GitHub API, either because it does not parse or because it names a host other than github.com.

func ErrInvalidSizeFile 

func ErrInvalidSizeFile(err error) error

func ErrNoFileInterceptor added in v1.0.24 

func ErrNoFileInterceptor() error

ErrNoFileInterceptor is returned when files are fetched with no interceptor registered to receive them.

func ErrResolvingGitRef added in v1.0.24 

func ErrResolvingGitRef(err error, ref string) error

ErrResolvingGitRef is returned when a branch, tag or reference name could not be resolved to a commit SHA through the GitHub API.

func ErrRootNotFound added in v1.0.24 

func ErrRootNotFound(root, ref string) error

ErrRootNotFound is returned when the configured root names no file or directory in the repository at the reference being walked.

func ErrSymlinkedRoot added in v1.0.24 

func ErrSymlinkedRoot(root, ref string) error

ErrSymlinkedRoot is returned when the configured root is a symbolic link and the listing route, which has no clone to read it through, cannot serve it.

Types

type CandidateFile added in v1.0.24 

type CandidateFile struct {
	// Path is the repository-relative path of the file.
	Path string `json:"path,omitempty"`
	// Name is the base name of Path.
	Name string `json:"name,omitempty"`
	// SHA is the blob SHA, suitable for a later blob fetch.
	SHA string `json:"sha,omitempty"`
	// Size is the blob size in bytes as reported by the tree.
	Size int64 `json:"size,omitempty"`
	// Kind is the IaC file type inferred from the path. It is empty when the
	// path is interesting but not distinctive enough to name a type.
	Kind core.IaCFileTypes `json:"kind,omitempty"`
	// Score is the relevance score used for the ranking.
	Score int `json:"score,omitempty"`
}

CandidateFile is a single interesting file described from Git tree metadata alone. No blob has been downloaded to produce it, so Kind is inferred from the path and is a hint, not an identification: the authoritative answer comes from files.IdentifyFile once the contents are in hand.

type DirInterceptor 

type DirInterceptor func(Directory) error

type Directory 

type Directory struct {
	Name string `json:"name,omitempty"`
	Path string `json:"path,omitempty"`
}

type File 

type File struct {
	Name    string `json:"name,omitempty"`
	Content string `json:"content,omitempty"`
	Path    string `json:"path,omitempty"`
}

func WalkLocalDirectory 

func WalkLocalDirectory(path string) ([]*File, error)

type FileInterceptor 

type FileInterceptor func(File) error

type Git 

type Git struct {
	// contains filtered or unexported fields
}

Git represents the Git Walker

func NewGit 

func NewGit() *Git

NewGit returns a pointer to an instance of Git

func (*Git) BaseURL 

func (g *Git) BaseURL(baseurl string) *Git

BaseURL sets git repository base URL and returns a pointer to the same Git instance

func (*Git) Branch 

func (g *Git) Branch(branch string) *Git

Branch sets git repository branch which will be cloned and returns a pointer to the same Git instance

func (*Git) FetchCandidates added in v1.0.24 

func (g *Git) FetchCandidates(ctx context.Context, candidates []CandidateFile) error

FetchCandidates downloads the blob behind each candidate and hands it to the registered file interceptor, in the order given. Candidates usually come from ListInterestingFiles, filtered down to whatever the user selected.

Downloads overlap, up to blobFetchConcurrency at a time, but the interceptor is still called once at a time and in order, so it does not have to be safe for concurrent use.

File.Path on the intercepted file is the repository-relative path, not a local filesystem path: nothing is written to disk on this route.

The repository must be on github.com, for the reason ListInterestingFiles gives, and a file interceptor must be registered: a fetch with nowhere to deliver to is refused rather than reported as an import of no files.

func (*Git) ListInterestingFiles added in v1.0.24 

func (g *Git) ListInterestingFiles(ctx context.Context) (InterestingFiles, error)

ListInterestingFiles resolves the configured reference to a commit, lists that commit's tree recursively in a single request, and returns the ranked candidate files. No blob is downloaded, so the result is cheap enough to render an import picker from.

The listing is always recursive, because a picker is asking what the repository holds: with no Root configured that is the whole repository, and a Root narrows which subtree is listed, never how deep. Walk keeps its own historical meaning for Root - the named directory only, unless the caller asked for "/**" - and is unaffected by this.

A truncated listing is reported through InterestingFiles.Truncated rather than as an error: the candidates returned are still usable, they are just not the whole repository.

The repository must be on github.com. There is no clone to fall back to on this route, and the access token must never travel to a host the caller did not configure.

func (*Git) MaxFileSize 

func (g *Git) MaxFileSize(size int64) *Git

BaseURL sets git repository base URL and returns a pointer to the same Git instance

func (*Git) Owner 

func (g *Git) Owner(owner string) *Git

Owner sets git repository owner and returns a pointer to the same Git instance

func (*Git) ReferenceName 

func (g *Git) ReferenceName(refName string) *Git

func (*Git) RegisterDirInterceptor 

func (g *Git) RegisterDirInterceptor(i DirInterceptor) *Git

func (*Git) RegisterFileInterceptor 

func (g *Git) RegisterFileInterceptor(i FileInterceptor) *Git

func (*Git) RegisterProgressHook added in v1.0.24 

func (g *Git) RegisterProgressHook(h ProgressHook) *Git

RegisterProgressHook registers a callback invoked as the walk advances through its stages. The hook is called synchronously, so it should return promptly.

func (*Git) Repo 

func (g *Git) Repo(repo string) *Git

Repo sets github repository and returns a pointer to the same Git instance

func (*Git) Root 

func (g *Git) Root(root string) *Git

Root sets git repository root node from where Git walker needs to start traversing and returns a pointer to the same Git instance

If the root parameter ends with a "/**" then github walker will run in "traversal" mode, ie. it will look into each sub directory of the root node If path will be prefixed with "/" if not already.

A root of "" or "/" scopes nothing: a walk reads the repository root exactly as it always has, and ListInterestingFiles reads it as the whole repository.

func (*Git) ShowLogs 

func (g *Git) ShowLogs() *Git

ShowLogs enable the logs and returns a pointer to the same Git instance

func (*Git) Timeout added in v1.0.24 

func (g *Git) Timeout(d time.Duration) *Git

Timeout bounds the whole traversal. A zero duration, the default, leaves the traversal bounded only by the context passed to WalkContext.

func (*Git) Token added in v1.0.24 

func (g *Git) Token(token string) *Git

Token sets the GitHub App or OAuth access token used to authenticate against the GitHub API and against the go-git clone, so private repositories are reachable and the authenticated rate limit applies.

The token is never logged, never placed in an error message and never reported through the progress hook.

func (*Git) UseGithubAPI added in v1.0.24 

func (g *Git) UseGithubAPI() *Git

UseGithubAPI opts the walker into the hybrid GitHub crawl: Trees API plus selective blob downloads, with a go-git clone as the fallback. It is off by default so existing callers keep the clone-and-filter behaviour.

func (*Git) Walk 

func (g *Git) Walk() error

Walk will initiate traversal process.

It is equivalent to WalkContext with a background context and is retained for callers that predate context support.

func (*Git) WalkContext added in v1.0.24 

func (g *Git) WalkContext(ctx context.Context) error

WalkContext initiates the traversal process under ctx.

When UseGithubAPI has been enabled and the configured base URL points at github.com, the traversal runs over the Git Trees API: the reference is resolved to a commit, the tree is listed once recursively, the entries are filtered and ranked, and only the surviving blobs are downloaded. Every other case - a non-github.com host, a truncated tree, or the API path not being enabled - walks a go-git clone exactly as Walk always has.

type Github 

type Github struct {
	// contains filtered or unexported fields
}

Github represents the Github Walker

func NewGithub 

func NewGithub() *Github

NewGithub returns a pointer to an instance of Github

func (*Github) Branch 

func (g *Github) Branch(branch string) *Github

Branch sets github repository branch which will be traversed and returns a pointer to the same Github instance

func (*Github) Owner 

func (g *Github) Owner(owner string) *Github

Owner sets github repository owner and returns a pointer to the same Github instance

func (*Github) RegisterDirInterceptor 

func (g *Github) RegisterDirInterceptor(i GithubDirInterceptor) *Github

RegisterFileInterceptor takes in a directory interceptor which will be invoked on each "directory" node and it returns pointer to the same github instance

Github Walker walks the nodes concurrently so if the interceptor is reading or writing to any variable from a higher namespace then those operations should be done in thread safe manner in order to avoid data races

func (*Github) RegisterFileInterceptor 

func (g *Github) RegisterFileInterceptor(i GithubFileInterceptor) *Github

RegisterFileInterceptor takes in a file interceptor which will be invoked on each "file" node and it returns pointer to the same github instance

Github Walker walks the nodes concurrently so if the interceptor is reading or writing to any variable from a higher namespace then those operations should be done in thread safe manner in order to avoid data races

func (*Github) RegisterProgressHook added in v1.0.24 

func (g *Github) RegisterProgressHook(h ProgressHook) *Github

RegisterProgressHook registers a callback invoked as the walk advances. The hook is called from the goroutine walking the node, so an implementation that touches shared state must do so in a thread safe manner.

func (*Github) Repo 

func (g *Github) Repo(repo string) *Github

Repo sets github repository and returns a pointer to the same Github instance

func (*Github) Root 

func (g *Github) Root(root string) *Github

Root sets github repository root node from where Github walker needs to start traversing and returns a pointer to the same Github instance

If the root parameter ends with a "/**" then github walker will run in "traversal" mode, ie. it will look into each sub directory of the root node

If the root node ends with an extension, then that file will be returned and github walker will not traverse deeper

func (*Github) Token added in v1.0.24 

func (g *Github) Token(token string) *Github

Token sets the GitHub App or OAuth access token used to authenticate the Contents API calls, so private repositories are reachable and the authenticated rate limit applies.

The token is never logged and never placed in an error message.

func (*Github) Walk 

func (g *Github) Walk() error

Walk will initiate traversal process.

It is equivalent to WalkContext with a background context and is retained for callers that predate context support.

func (*Github) WalkContext added in v1.0.24 

func (g *Github) WalkContext(ctx context.Context) error

WalkContext initiates the traversal process under ctx. Cancelling ctx aborts the in-flight and subsequent Contents API requests.

type GithubContentAPI 

type GithubContentAPI struct {
	Name        string `json:"name,omitempty"`
	Path        string `json:"path,omitempty"`
	SHA         string `json:"sha,omitempty"`
	Size        int64  `json:"size,omitempty"`
	URL         string `json:"url,omitempty"`
	HTMLURL     string `json:"html_url,omitempty"`
	GitURL      string `json:"git_url,omitempty"`
	DownloadURL string `json:"download_url,omitempty"`
	Type        string `json:"type,omitempty"`
	// Content will be empty when the path is of a directory
	Content string `json:"content,omitempty"`
	// Encoding will be empty when the path is of a directory
	Encoding string `json:"encoding,omitempty"`
}

GithubContentAPI represents Github API v3 response to /repos/{owner}/{repo}/contents/{path}?ref={branch}

type GithubDirInterceptor 

type GithubDirInterceptor func(GithubDirectoryContentAPI) error

GithubDirInterceptor represents function signature which will be used on "dir" nodes when the github walker traverses the paths

type GithubDirectoryContentAPI 

type GithubDirectoryContentAPI []GithubContentAPI

GithubDirectoryContentAPI represents Github API v3 response to /repos/{owner}/{repo}/contents/{path}?ref={branch} when "path" is of a directory

type GithubFileInterceptor 

type GithubFileInterceptor func(GithubContentAPI) error

GithubFileInterceptor represents function signature which will be used on "file" nodes when the github walker traverses the paths

type InterestingFiles added in v1.0.24 

type InterestingFiles struct {
	// CommitSHA is the commit the tree was listed from.
	CommitSHA string `json:"commitSha,omitempty"`
	// Truncated reports that GitHub could not return the whole tree in one
	// response, so Candidates is incomplete and a clone is needed to see
	// every file.
	Truncated bool `json:"truncated,omitempty"`
	// Candidates is ranked most interesting first.
	Candidates []CandidateFile `json:"candidates,omitempty"`
}

InterestingFiles is the ranked candidate listing for one commit.

type ProgressHook added in v1.0.24 

type ProgressHook func(ProgressUpdate)

ProgressHook receives ProgressUpdate values as a walk advances. How it is delivered differs per walker and is documented on each RegisterProgressHook: Git calls it one update at a time, while Github calls it from every goroutine its fan-out walks a node with. An implementation that touches shared state must therefore be safe for concurrent use.

type ProgressStage added in v1.0.24 

type ProgressStage string

ProgressStage names the phase of a walk a ProgressUpdate describes. 

const (
	ProgressStageResolveRef ProgressStage = "resolve-ref"
	ProgressStageListTree   ProgressStage = "list-tree"
	ProgressStageRank       ProgressStage = "rank"
	ProgressStageFetchBlob  ProgressStage = "fetch-blob"
	ProgressStageClone      ProgressStage = "clone"
)

Progress stages reported through a ProgressHook.

type ProgressUpdate added in v1.0.24 

type ProgressUpdate struct {
	Stage   ProgressStage `json:"stage,omitempty"`
	Message string        `json:"message,omitempty"`
	Current int           `json:"current,omitempty"`
	Total   int           `json:"total,omitempty"`
}

ProgressUpdate describes how far a walk has advanced. Current and Total are only meaningful for stages that iterate over a known number of items, such as blob downloads; they are zero elsewhere.

An update never carries the access token.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.