crow

crow

Overview

4chan API client + archiver

Install

Archiver

go install github.com/fiwippi/crow@latest

API

go get github.com/fiwippi/crow/pkg/api

Usage

Archiver

$ ./crow --help
Usage:
  ./crow po 570368
  ./crow po/thread/570368
  ./crow https://boards.4channel.org/po/thread/570368

  -dst string
        Destination dir (default "./")
  -files-only
        Whether to archive only the files and not the html page of the thread
  -interval duration
        How often to check if the thread updated (default 5m0s)
  -overwrite
        Whether to overwrite files which already exist
  -run-once
        Download the thread once and exit without checking for updates
  -validate-md5
        Whether to validate the MD5 hash of files (default true)

API

To download all files in a thead, errors ignored for brevity:

package main

import (
	"io"
	"os"

	"github.com/fiwippi/crow/pkg/api"
)

func main() {
    // Create the client and retrieve the thread
    c := api.DefaultClient()
    t, _, _ := c.GetThread("po", 570368)
    
    // Download all files from the thread
    for _, p := range t.Posts {
        if p.HasFile {
            // Download the file
            m, _ := c.GetFile(p)
    
            // Create file on fileystem
            out, _ := os.Create(m.Filename + m.Ext)
            defer out.Close()
    
            // Copy the contents to the file
            _, _ = io.Copy(out, m.Body)
        }
    }
}

Notes

4chan API Rules

1. Do not make more than one request per second.
2. Thread updating should be set to a minimum of 10 seconds, preferably higher.
3. Use If-Modified-Since when doing your requests.
4. Make API requests using the same protocol as the app. Only use SSL when a user is accessing your app over HTTPS.

License

Type

BSD-3-Clause

Disclaimer

1. The content of this website is for mature audiences only and may not be suitable for minors. If you are a minor or it is illegal for you to access mature images and language, do not proceed.
2. You agree not to hold the Author responsible for any damages from your use of the website, and you understand that the content posted is not owned or generated by 4chan, but rather by 4chan's users.