app

package
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 8, 2026 License: Apache-2.0 Imports: 26 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type App 

type App struct {
	// contains filtered or unexported fields
}

App represents the core application logic

func NewApp 

func NewApp(st *store.Store, emitter EventEmitter) *App

NewApp creates a new App instance with dependencies injected

func (*App) CheckForUpdate 

func (a *App) CheckForUpdate() (*types.UpdateInfo, error)

CheckForUpdate checks if a new version is available

func (*App) CheckHTTPAccess 

func (a *App) CheckHTTPAccess(targetURL string) (map[string]types.BotAccess, error)

CheckHTTPAccess tests HTTP access for each bot

func (*App) CheckRobotsTxt 

func (a *App) CheckRobotsTxt(targetURL string) (map[string]types.BotAccess, error)

CheckRobotsTxt checks robots.txt for bot access

func (*App) CheckSSR 

func (a *App) CheckSSR(targetURL string) (*types.ContentVisibilityResult, string, error)

CheckSSR performs SSR check and returns score with screenshot

func (*App) CheckSystemHealth 

func (a *App) CheckSystemHealth() *types.SystemHealthCheck

CheckSystemHealth checks if all required dependencies are available

func (*App) ClearCrawlQueue 

func (a *App) ClearCrawlQueue(projectID uint) error

ClearCrawlQueue removes all URLs from the crawl queue for a project

func (*App) DeleteCrawlByID 

func (a *App) DeleteCrawlByID(crawlID uint) error

DeleteCrawlByID deletes a specific crawl

func (*App) DeleteProjectByID 

func (a *App) DeleteProjectByID(projectID uint) error

DeleteProjectByID deletes a project and all its crawls

func (*App) DownloadAndInstallUpdate 

func (a *App) DownloadAndInstallUpdate() error

DownloadAndInstallUpdate downloads and installs the update

func (*App) GetAICrawlerData 

func (a *App) GetAICrawlerData(projectURL string) (*types.AICrawlerResponse, error)

GetAICrawlerData retrieves AI crawler data for a project

func (*App) GetActiveCrawlStats 

func (a *App) GetActiveCrawlStats(projectID uint) (*types.ActiveCrawlStats, error)

GetActiveCrawlStats returns statistics for an active crawl (no URL list, just counts). For incremental crawling, stats are aggregated across all crawls in the run.

func (*App) GetActiveCrawls 

func (a *App) GetActiveCrawls() []types.CrawlProgress

GetActiveCrawls returns the progress of all active crawls

func (*App) GetConfigForDomain 

func (a *App) GetConfigForDomain(urlStr string) (*types.ConfigResponse, error)

GetConfigForDomain retrieves the configuration for a specific domain

func (*App) GetCrawlStats 

func (a *App) GetCrawlStats(crawlID uint) (*types.ActiveCrawlStats, error)

GetCrawlStats returns statistics for any crawl (active or completed). For incremental crawling, stats are aggregated across all crawls in the run.

func (*App) GetCrawlWithResultsPaginated 

func (a *App) GetCrawlWithResultsPaginated(crawlID uint, limit int, cursor uint, contentTypeFilter string) (*types.CrawlResultPaginated, error)

GetCrawlWithResultsPaginated returns a specific crawl with paginated results. For incremental crawling, results are aggregated across all crawls in the run.

func (*App) GetCrawls 

func (a *App) GetCrawls(projectID uint) ([]types.CrawlInfo, error)

GetCrawls returns deduplicated crawl history for a project. For incremental crawling, runs are aggregated into single entries. For standalone crawls, each crawl is returned as-is.

func (*App) GetFaviconData 

func (a *App) GetFaviconData(faviconPath string) (string, error)

GetFaviconData reads a favicon file and returns it as a base64 data URL

func (*App) GetPageContent 

func (a *App) GetPageContent(crawlID uint, pageURL string) (string, error)

GetPageContent retrieves the saved text content for a specific URL in a crawl Content is read from ~/.bluesnake/<domain>/<crawlid>/<sanitized-url>.txt

func (*App) GetPageLinksForURL 

func (a *App) GetPageLinksForURL(crawlID uint, pageURL string) (*types.PageLinksResponse, error)

GetPageLinksForURL retrieves inbound and outbound links for a specific URL in a crawl

func (*App) GetProjects 

func (a *App) GetProjects() ([]types.ProjectInfo, error)

GetProjects returns all projects from the database with their latest crawl info

func (*App) GetQueueStatus 

func (a *App) GetQueueStatus(projectID uint) (*types.QueueStatus, error)

GetQueueStatus returns the status of the crawl queue for a project

func (*App) GetSSRCSRScreenshots 

func (a *App) GetSSRCSRScreenshots(targetURL string) (jsScreenshot, noJSScreenshot string, err error)

GetSSRCSRScreenshots captures screenshots with and without JavaScript

func (*App) GetVersion 

func (a *App) GetVersion() string

GetVersion returns the current version of the application

func (*App) ResumeCrawl 

func (a *App) ResumeCrawl(projectID uint) (*types.ProjectInfo, error)

ResumeCrawl continues a paused run for a project

func (*App) RunAICrawlerChecks 

func (a *App) RunAICrawlerChecks(projectURL string) error

RunAICrawlerChecks runs all AI crawler checks and saves the results

func (*App) SaveAICrawlerData 

func (a *App) SaveAICrawlerData(projectURL string, data *types.AICrawlerData, ssrScreenshot, jsScreenshot, noJSScreenshot string) error

SaveAICrawlerData saves AI crawler data for a project

func (*App) SearchCrawlResultsPaginated 

func (a *App) SearchCrawlResultsPaginated(crawlID uint, query string, contentTypeFilter string, limit int, cursor uint) (*types.CrawlResultPaginated, error)

SearchCrawlResultsPaginated searches and filters crawl results with pagination. For incremental crawling, results are aggregated across all crawls in the run.

func (*App) StartCrawl 

func (a *App) StartCrawl(urlStr string) (*types.ProjectInfo, error)

StartCrawl initiates a crawl for the given URL and returns the project info. The returned ProjectInfo contains the canonical domain (after following redirects), which may differ from the input URL (e.g., amahahealth.com -> www.amahahealth.com).

func (*App) Startup 

func (a *App) Startup(ctx context.Context)

Startup initializes the app with a context

func (*App) StopCrawl 

func (a *App) StopCrawl(projectID uint) error

StopCrawl stops an active crawl for a specific project

func (*App) UpdateConfigForDomain 

func (a *App) UpdateConfigForDomain(
	urlStr string,
	jsRendering bool,
	initialWaitMs, scrollWaitMs, finalWaitMs int,
	parallelism int,
	userAgent string,
	includeSubdomains bool,
	spiderEnabled bool,
	sitemapEnabled bool,
	sitemapURLs []string,
	checkExternalResources bool,
	robotsTxtMode string,
	followInternalNofollow bool,
	followExternalNofollow bool,
	respectMetaRobotsNoindex bool,
	respectNoindex bool,
) error

UpdateConfigForDomain updates the configuration for a specific domain

func (*App) UpdateIncrementalConfigForDomain 

func (a *App) UpdateIncrementalConfigForDomain(urlStr string, enabled bool, budget int) error

UpdateIncrementalConfigForDomain updates the incremental crawling settings for a domain

type Bot 

type Bot struct {
	Name            string   `json:"name"`
	Version         string   `json:"version"`
	InfoURL         string   `json:"info_url"`
	UserAgentPrefix string   `json:"user_agent_prefix"`
	Type            []string `json:"type"`
	Brand           string   `json:"brand"`
	Domain          string   `json:"domain"`
}

Bot represents a bot configuration

type BotsConfig 

type BotsConfig struct {
	Bots []Bot `json:"bots"`
}

BotsConfig represents the bots configuration file

type EventEmitter 

type EventEmitter interface {
	Emit(eventType EventType, data interface{})
}

EventEmitter is the interface for emitting events Each transport layer implements this differently: - Desktop (Wails): runtime.EventsEmit - HTTP: WebSocket/SSE broadcasting - MCP: MCP notifications

type EventType 

type EventType string

EventType represents the type of event 

const (
	EventCrawlStarted   EventType = "crawl:started"
	EventCrawlCompleted EventType = "crawl:completed"
	EventCrawlStopped   EventType = "crawl:stopped"
)

type NoOpEmitter 

type NoOpEmitter struct{}

NoOpEmitter is a default implementation that does nothing Useful for testing or when events aren't needed

func (*NoOpEmitter) Emit 

func (n *NoOpEmitter) Emit(eventType EventType, data interface{})

Emit does nothing

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.