Documentation
¶
Overview ¶
Copyright 2012 The Walk Authors. All rights reserved. Use of this source code is governed by a BSD-style license that can be found in the LICENSE file.
[spider frame (golang)] Pholcus (Ghost Spider) is a high-concurrency, distributed, heavyweight crawler written in pure Go. It supports standalone, server, and client modes with Web, GUI, and CLI interfaces; simple flexible rules; batch task concurrency; rich output formats (mysql/mongodb/csv/excel etc.); and shared demos. It also supports horizontal and vertical crawling, simulated login, and advanced features like pause/cancel. (Official QQ group: Go Big Data 42731170) GUI package.
Index ¶
- Constants
- Variables
- func Init()
- func Run()
- func SetTaskConf()
- func SpiderNames() (names []string)
- func SpiderPrepare()
- type InputConfig
- type KV
- type LogView
- func (lv *LogView) AppendText(value string)
- func (lv *LogView) Clean()
- func (*LogView) LayoutFlags() walk.LayoutFlags
- func (*LogView) MinSizeHint() walk.Size
- func (lv *LogView) PostAppendText(value string)
- func (*LogView) SizeHint() walk.Size
- func (lv *LogView) WndProc(hwnd win.HWND, msg uint32, wParam, lParam uintptr) uintptr
- func (lv *LogView) Write(p []byte) (int, error)
Constants ¶
const TEM_APPENDTEXT = win.WM_USER + 6
Variables ¶
var GuiOpt = struct { Mode []*KV Pausetime []*KV ProxyMinute []*KV }{ Mode: []*KV{ {Key: "Standalone", Int: status.OFFLINE}, {Key: "Server", Int: status.SERVER}, {Key: "Client", Int: status.CLIENT}, }, Pausetime: []*KV{ {Key: "No pause", Int64: 0}, {Key: "0.1 sec", Int64: 100}, {Key: "0.3 sec", Int64: 300}, {Key: "0.5 sec", Int64: 500}, {Key: "1 sec", Int64: 1000}, {Key: "3 sec", Int64: 3000}, {Key: "5 sec", Int64: 5000}, {Key: "10 sec", Int64: 10000}, {Key: "15 sec", Int64: 15000}, {Key: "20 sec", Int64: 20000}, {Key: "30 sec", Int64: 30000}, {Key: "60 sec", Int64: 60000}, }, ProxyMinute: []*KV{ {Key: "No proxy", Int64: 0}, {Key: "1 min", Int64: 1}, {Key: "3 min", Int64: 3}, {Key: "5 min", Int64: 5}, {Key: "10 min", Int64: 10}, {Key: "15 min", Int64: 15}, {Key: "20 min", Int64: 20}, {Key: "30 min", Int64: 30}, {Key: "45 min", Int64: 45}, {Key: "60 min", Int64: 60}, {Key: "120 min", Int64: 120}, {Key: "180 min", Int64: 180}, }, }
GuiOpt holds pause time and run mode options.
var Input = &InputConfig{ AppConf: cache.Task, Pausetime: cache.Task.Pausetime, ProxyMinute: cache.Task.ProxyMinute, }
Functions ¶
func SetTaskConf ¶
func SetTaskConf()
func SpiderNames ¶
func SpiderNames() (names []string)
func SpiderPrepare ¶
func SpiderPrepare()
Types ¶
type InputConfig ¶ added in v1.4.0
type InputConfig struct {
Spiders []*model.GUISpider
*cache.AppConf
Pausetime int64
ProxyMinute int64
}
InputConfig holds GUI input.
type LogView ¶
type LogView struct {
walk.WidgetBase
// contains filtered or unexported fields
}
func (*LogView) AppendText ¶
func (*LogView) LayoutFlags ¶
func (*LogView) LayoutFlags() walk.LayoutFlags