mediateq

package module
v0.0.0-...-c99d64b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 11, 2023 License: Apache-2.0 Imports: 4 Imported by: 0

README

mediateq

mediateq is a file storage REST API microservice that allows users to upload and download files, as well as resize images on the fly.

CI

Prerequisites

  • libvips 8.3+
  • PostgreSQL 14+

Installation

To install mediateq, clone the project repository:

git clone https://github.com/behouba/mediateq.git

Mediateq depend on libvips.

Install libvips on Debian based Linux distributions:

sudo apt install -y libvips-dev

or run the following script from bimg as sudo (supports OSX, Debian/Ubuntu, Redhat, Fedora, Amazon Linux):

curl -s https://raw.githubusercontent.com/h2non/bimg/master/preinstall.sh | sudo bash -

The install script requires curl and pkg-config

If the above script is not working, please follow libvips installation instructions:

https://libvips.github.io/libvips/install.html

You can then build mediateq by running the build.sh script:

./build.sh

You will also need to setup a database. Currently mediateq only support postgreSQL.

Read the instructions to setup a postgreSQL database here

To run mediateq, use the following command:

./bin/mediateq -config=mediateq.yaml

You can also run tests for mediateq by using the run_tests.sh script:

./run_tests.sh

API specification

The API specification of mediateq can be found here

All endpoints should be prefixed with: /mediateq/{version}

Note: mediateq uses base64 hash of files as filename to avoid duplications.

/info (GET)

Retrieves information about the server.

Example Request

GET /info

Example Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "version": "v0",
    "domain": "http://localhost:8080",
    "port": 8080,
    "database": "postgres",
    "storage": "localdisk",
    "allowedContentTypes": [
    "image/jpeg",
    "image/png",
    "image/bimg",
    "image/webp"
    ],
    "uptime": 35
}
/upload (POST)

Uploads a new media file to the server.

Example Request

curl -X POST -H "Content-Type: multipart/form-data" -F "file=@herman.jpeg" http://localhost:8080/mediateq/v0/upload

Example Response

Copy code
HTTP/1.1 200 OK
Content-Type: application/json

{
    "media": {
        "id": "2K2dScz76xXsc0D3u29M3w1iKcw",
        "url": "http://localhost:8080/mediateq/v0/download/2K2dScz76xXsc0D3u29M3w1iKcw",
        "origin": "::1",
        "contentType": "image/jpeg",
        "sizeBytes": 103417,
        "tmestamp": 1673176612,
        "base64Hash": "zKZqFTxVdiIWGf-4otBjjBS46aPqs-Q6W0mefJmVhmo"
    }
}
/download/{base64Hash} (GET)

Downloads a media file from the server.

Example Request

GET /download/2K2dScz76xXsc0D3u29M3w1iKcw?width=400
Example Response

Copy code
HTTP/1.1 200 OK
Content-Type: video/mp4
[binary data]
/thumbnail/{base64Hash} (GET)

Downloads thumbnail version of images files.

The width and height of the image are required queries paramters.

Example Request


GET /download/2K2dScz76xXsc0D3u29M3w1iKcw?width=640&height=480
Example Response

Copy code
HTTP/1.1 200 OK
Content-Type: video/mp4
[binary data]
/media (GET)

Retrieves a paginated list of all media files on the server.

Use query parameters limit and offset to paginate the results.

Example Request

GET /media?limit=4&offset=1

Example Response


HTTP/1.1 200 OK
Content-Type: application/json

{
    "mediaList": [
        {
            "id": "2JVvl82araY9MfG9O2keI1dSZDy",
            "url": "http://localhost:8080/mediateq/v0/download/2JVvl82araY9MfG9O2keI1dSZDy",
            "origin": "::1",
            "contentType": "image/jpeg",
            "sizeBytes": 103417,
            "tmestamp": 1672176212,
            "base64Hash": "zKZqFTxVdiIWGf-4otBjjBS46aPqs-Q6W0mefJmVhmo"
        },
        {
            "id": "2K2dScz76xXsc0D3u29M3w1iKcw",
            "url": "http://localhost:8080/mediateq/v0/download/2K2dScz76xXsc0D3u29M3w1iKcw",
            "origin": "::1",
            "contentType": "image/jpeg",
            "sizeBytes": 103417,
            "tmestamp": 1673176612,
            "base64Hash": "zKZqFTxVdiIWGf-4otBjjBS46aPqs-Q6W0mefJmVhmo"
        }
    ]
}
/media/{base64Hash} (GET)

Retrieves information about a specific media file.

Example Request


GET /media/2K2dScz76xXsc0D3u29M3w1iKcw

Example Response


HTTP/1.1 200 OK
Content-Type: application/json

{
    "media": {
        "id": "2K2dScz76xXsc0D3u29M3w1iKcw",
        "url": "http://localhost:8080/mediateq/v0/download/2K2dScz76xXsc0D3u29M3w1iKcw",
        "origin": "::1",
        "contentType": "image/jpeg",
        "sizeBytes": 103417,
        "tmestamp": 1673176612,
        "base64Hash": "zKZqFTxVdiIWGf-4otBjjBS46aPqs-Q6W0mefJmVhmo"
    }
}
/media/{base64Hash} (DELETE)

Delete a specific media file by it base64 hash identifier.

Example Request


DELETE /media/4siOxL16rCSeWvEeGxBAtqMmF04HffW_qg8zWuOh2MY

Example Response


HTTP/1.1 200 OK
Content-Type: application/json

{
    "message": "media 4siOxL16rCSeWvEeGxBAtqMmF04HffW_qg8zWuOh2MY deleted"
}
Docker container

[TODO]

Documentation

Index

Constants

View Source 
const (
	ContentTypeFormData ContentType = "multipart/form-data"

	// Supported content types for images
	ContentTypeJPEG ContentType = "image/jpeg"
	ContentTypePNG  ContentType = "image/png"
	ContentTypeGIF  ContentType = "image/gif"
	ContentTypeBIMP ContentType = "image/bimg"
	ContentTypeWEBP ContentType = "image/webp"

	// Storage type options
	StorageTypeLocalDisk StorageType = "localdisk"
	StorageTypeS3        StorageType = "s3"

	// Database options
	DBTypePostgres DBType = "postgres"
	DBTypeSQLite   DBType = "sqlite"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type ContentType 

type ContentType string

ContentType represents HTTP MIME types sent in Content-type header

type DBType 

type DBType string

DB type represents the type of database used by the server

type ImageProcessor 

type ImageProcessor interface {
	Resize(buff []byte, width, height int) ([]byte, error)
	Rotage(buff []byte, degree int) ([]byte, error)
	Grayscale(buff []byte) ([]byte, error)
}

ImageProcessor interface provides image processing methods

type Media 

type Media struct {
	ID          string      `json:"id"`          // Numeric id (db primary key)
	URL         string      `json:"url"`         // url to access the file over internet
	Origin      string      `json:"origin"`      // Origin domain of the file
	ContentType ContentType `json:"contentType"` //
	SizeBytes   int64       `json:"sizeBytes"`   // Size of the file in bytes
	Timestamp   int64       `json:"tmestamp"`    // Media creation timestamp
	Base64Hash  string      `json:"base64Hash"`  // Base64 hash of the file used as a unique string identifier
}

Media is a representation of mediateq file.

func (Media) GetFilePath 

func (m Media) GetFilePath(uploadPath string) (string, error)

GetFilePath return the path to a media file 2 subdirectories are created for more manageable browsing and use the remainder as the file name. For example, if Base64Hash is 'qwerty' and content type is 'image/png' the path will be 'q/w/erty'.

func (Media) IsImage 

func (m Media) IsImage() bool

IsImage check if a media file is an image base on it content type

type Storage 

type Storage interface {
	// Write method write file buffer to storage and
	// return the relative path and an error if the write operating fail
	Write(ctx context.Context, buff []byte, filePath string) error
	// Read method read a file from storage and return
	// the relative path and an error if the read operation fail
	Read(ctx context.Context, filePath string) ([]byte, error)
	// Remove method should remove a file from
	// the storage given the path to the target file
	Remove(ctx context.Context, path string) error
}

Storage is an abstration of place where files are stored

type StorageType 

type StorageType string

StorageType represents type of storage where media are stored

type Thumbnail 

type Thumbnail struct {
	Media
	ThumbnailSize
}

type ThumbnailSize 

type ThumbnailSize struct {
	Width  int  `yaml:"width"`
	Height int  `yaml:"height"`
	Crop   bool `yaml:"method"`
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
mediateq command
postgres
schema
pkg
config
fsutils
localdisk

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.