README
¶
hero-scrape
Find the hero (main) image of an URL
Demo
See a demo on https://hero-scrape.viktor-braun.de
Description
hero-scrape extracts the main image of a webpage. It use different strategies to find the main images (OpenGraph HTML Tags and heuristic search). You can use the existing strategies or implement your own.
To find the "biggest" image it is necessary to download it. fastimage is the perfect choice for that job.
Installation
go get github.com/v-braun/hero-scrape
Usage
With pre configured strategies
pageUrl, _ := url.Parse("https://github.com/v-braun/hero-scrape")
res, _ := http.Get(pageUrl.String())
defer res.Body.Close()
result, _ := heroscrape.Scrape(pageUrl, res.Body)
fmt.Println(result.Image)
With cusom strategies
pageUrl, _ := url.Parse("https://github.com/v-braun/hero-scrape")
res, _ := http.Get(pageUrl.String())
defer res.Body.Close()
result, _ := heroscrape.ScrapeWithStrategy(pageUrl, res.Body, , NewOgStrategy(), NewHeuristicStrategy(), YourOwnStrategy())
fmt.Println(result.Image)
Related Projects
- hero-scrape Demo for this lib
- fastimage Finds the type and/or size of a remote image given its uri, by fetching as little as needed.
- goquery A little like that j-thing, only in Go.
Known Issues
If you discover any bugs, feel free to create an issue on GitHub fork and send me a pull request.
Authors
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
License
See LICENSE.
Documentation
¶
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNotComplete = errors.New("Not complete")
ErrNotComplete will be returned if the Scrape was not completley done
Logger instance for the entire module
Functions ¶
Types ¶
type ImageLocation ¶
type ImageLocation string
type SearchResult ¶
SearchResult represents the scrape result
func Scrape ¶
Scrape the given url
Example ¶
package main
import (
"fmt"
"net/http"
"net/url"
heroscrape "github.com/v-braun/hero-scrape"
)
func main() {
pageUrl, _ := url.Parse("https://github.com/v-braun/hero-scrape")
res, _ := http.Get(pageUrl.String())
defer res.Body.Close()
result, _ := heroscrape.Scrape(pageUrl, res.Body)
fmt.Println(result.Image)
}
Output:
func ScrapeWithStrategy ¶
func ScrapeWithStrategy(srcURL *url.URL, html io.Reader, strategies ...Strategy) (*SearchResult, error)
ScrapeWithStrategy scrapes the given url with the given strategy
func (*SearchResult) Complete ¶
func (sr *SearchResult) Complete() bool
Complete returns true if the SearchResult has found everything
type Strategy ¶
Strategy interface represents an interface for scraping an website
func NewHeuristicStrategy ¶
func NewHeuristicStrategy() Strategy
func NewOgStrategy ¶
func NewOgStrategy() Strategy
NewOgStrategy returns a new Strategy that search for OG meta tags
Source Files