Directories
¶
| Path | Synopsis |
|---|---|
|
Package crawl runs the Baidu mirror: it drains the queue, fetches each item (a Baike lemma or a web search page) through the baidu client, stores the raw bytes and a parsed record, expands the related lemmas it discovers, and marks every row's outcome.
|
Package crawl runs the Baidu mirror: it drains the queue, fetches each item (a Baike lemma or a web search page) through the baidu client, stores the raw bytes and a parsed record, expands the related lemmas it discovers, and marks every row's outcome. |
|
Package store is the persistence layer for the Baidu mirror: a crawl queue, a visited set, the harvested records (Baike articles, search results, suggest queries), the per-run job log, and the raw artifact files captured verbatim.
|
Package store is the persistence layer for the Baidu mirror: a crawl queue, a visited set, the harvested records (Baike articles, search results, suggest queries), the per-run job log, and the raw artifact files captured verbatim. |
Click to show internal directories.
Click to hide internal directories.