retriever

package
v0.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 19, 2025 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source
const (
	GATEWAY_GENTOKEN_URL      = "/gateway/gentoken"
	GATEWAY_UPLOADFILE_URL    = "/gateway/upload/file"
	GATEWAY_PARTUPLOAD_URL    = "/gateway/part-upload"
	GATEWAY_UPLOADPART_URL    = "/gateway/upload/part"
	GATEWAY_GETFILE_URL       = "/gateway/download"
	RETRIEVER_QUERYDATA_URL   = "/querydata"
	RETRIEVER_FETCHDATA_URL   = "/cache-fetch"
	RETRIEVER_NODESTATUS_URL  = "/status"
	RETRIEVER_GETCAPACITY_URL = "/capacity"

	DEFAULT_PART_SIZE = 32 * 1024 * 1024
)

Variables

This section is empty.

Functions

func CreateStorageOrder

func CreateStorageOrder(cli *chain.Client, info FileInfo, caller *signature.KeyringPair, event any) (string, error)

CreateStorageOrder creates a storage order on the blockchain for a file. Parameters:

cli - The blockchain client instance for submitting the order.
info - Struct containing file details (fragments, owner, name, territory, size).
caller - The keyring pair for signing the transaction.
event - Event handler for processing transaction events.

Returns:

string - The transaction block hash.
error - An error if the order creation fails, including encoding errors or transaction submission failures.

func DownloadData

func DownloadData(baseUrl, fid, segment, fpath string, capsule, rk, pkX []byte) error

DownloadData retrieves data from the gateway and writes it to local file system. Handles HTTP headers for decryption when re-encryption key (rk) and public key (pkX) are provided.

Parameters:

baseUrl  - Base URL of the gateway service
fid      - File id
segment  - Segment id
fpath    - Local path to save downloaded file
capsule  - Proxy re-encryption capsule
rk       - Re-encryption key bytes, generated by the re-encrypt method
pkX      - Public key bytes for encryption, generated by the re-encrypt method

Returns:

error - Possible errors include:
          - URL construction failure
          - HTTP request failure
          - File creation/write failure

func GenGatewayAccessToken

func GenGatewayAccessToken(baseUrl, message, account string, sign []byte) (string, error)

GenGatewayAccessToken generates an access token for the gateway using the provided base URL, message, account, and signature. It sends a POST request to the "/gateway/gentoken" endpoint with the necessary parameters. Message must be the current Unix timestamp (seconds) and expire after one minute to prevent the signature from being reused maliciously. Parameters:

baseUrl - The base URL of the gateway.
message - The message to be signed.
account - The account associated with the token.
sign - The signature of the message.

Returns:

string - The generated access token.
error - An error if the request fails or the response is invalid.

func GenReEncryptionKey

func GenReEncryptionKey(mnemonic string, pkB []byte) ([]byte, []byte, error)

GenReEncryptionKey generates a re-encryption key and corresponding public key using Schnorrkel scheme. This implements proxy re-encryption mechanism for decentralized storage systems.

Parameters:

mnemonic - User's mnemonic phrase for key derivation
pkB      - Recipient's public key bytes (32-byte expected)

Returns:

[]byte - Marshaled re-encryption key (rk)
[]byte - Encoded public key bytes for encryption (pkX)
error  - Possible errors include:
           - Invalid mnemonic phrase
           - Public key deserialization failure
           - Re-encryption key generation failure
           - Key serialization failure

func GetPreCapsuleAndGatewayPubkey

func GetPreCapsuleAndGatewayPubkey(baseUrl, fid string) ([]byte, []byte, error)

GetPreCapsuleAndGatewayPubkey retrieves pre-encapsulation capsule data and gateway public key for a given file by sending HTTP GET request to the gateway service. This is essential for subsequent encryption operations.

Parameters:

baseUrl - Base URL of the gateway service
fid     - Unique file identifier

Returns:

[]byte - Proxy re-encryption capsule
[]byte - Gateway's public key
error  - Possible errors include:
           - URL construction failure
           - HTTP request failure
           - JSON unmarshalling failure

func QueryDealMap

func QueryDealMap(cli *chain.Client, fid string) (map[int]struct{}, error)

QueryDealMap queries the completed segment indices of a file's storage deals from the blockchain. Parameters:

cli - The blockchain client instance for interacting with the chain.
fid - The unique file identifier string.

Returns:

map[int]struct{} - A set of completed segment indices.
error - An error if the query fails, including underlying blockchain query errors.

func SendHttpRequest

func SendHttpRequest(method, url string, headers map[string]string, dataReader *bytes.Buffer) ([]byte, error)

func SignedSR25519WithMnemonic

func SignedSR25519WithMnemonic(mnemonic string, msg []byte) ([]byte, error)

SignedSR25519WithMnemonic signs a message using the SR25519 scheme with a given mnemonic. It appends "<Bytes>" and "</Bytes>" to the message before signing. Parameters:

mnemonic - The mnemonic phrase used for signing.
msg - The message to be signed.

Returns:

[]byte - The signed message.
error - An error if the mnemonic is invalid or signing fails.

func UploadFile

func UploadFile(baseUrl, token, territory, filename string, file io.Reader, encrypt bool) (string, error)

UploadFile uploads a file to the specified territory. It sends a POST request to the "/gateway/upload/file" endpoint with the necessary parameters. Parameters:

	baseUrl - The base URL of the gateway.
	token - The access token for authentication.
	territory - The territory to which the file will be uploaded.
	filename - The name of the file to be uploaded.
	file - The file content to be uploaded.
    encrypt - Whether the data needs to be encrypted (using proxy re-encryption technology)

Returns:

string - The file identifier (FID) if the upload is successful.
error - An error if the upload fails.

func UploadFileParts

func UploadFileParts(baseUrl, token, fpath string, info *PartsInfo, encrypt bool) (string, error)

UploadFileParts uploads file parts to the specified territory. It sends a POST request to the "/gateway/upload/part" endpoint with the necessary parameters. Parameters:

baseUrl - The base URL of the gateway.
token - The access token for authentication.
fpath - The path of the file to be uploaded.
info - Information about the file parts to be uploaded.
encrypt - Whether the data needs to be encrypted (using proxy re-encryption technology)

Returns:

string - The file identifier (FID) if the upload is successful.
error - An error if the upload fails.

Types

type FileInfo

type FileInfo struct {
	Fid       string     `json:"fid"`
	FileName  string     `json:"file_name"`
	BaseDir   string     `json:"base_dir"`
	FileSize  int64      `json:"file_size"`
	Owner     []byte     `json:"owner"`
	Territory string     `json:"territory"`
	Segments  []string   `json:"segments"`
	Fragments [][]string `json:"fragments"`
}

func AsyncUploadFile

func AsyncUploadFile(baseUrl, token, territory, filename string, file io.Reader, noProxy, encrypt bool) (FileInfo, error)

AsyncUploadFile uploads a file asynchronously to the specified territory. It sends a POST request to the "/gateway/upload/file" endpoint with the necessary parameters. Parameters:

	baseUrl - The base URL of the gateway.
	token - The access token for authentication.
	territory - The territory to which the file will be uploaded.
	filename - The name of the file to be uploaded.
	file - The file content to be uploaded.
	noProxy - Whether to not create file orders through OSS proxy.
    encrypt - Whether the data needs to be encrypted (using proxy re-encryption technology)

Returns:

FileInfo - Information about the uploaded file.
error - An error if the upload fails.

func AsyncUploadFileParts

func AsyncUploadFileParts(baseUrl, token, fpath string, info *PartsInfo, noProxy, encrypt bool) (FileInfo, error)

AsyncUploadFileParts uploads file parts asynchronously to the specified territory. It sends a POST request to the "/gateway/upload/part" endpoint with the necessary parameters. Parameters:

baseUrl - The base URL of the gateway.
token - The access token for authentication.
fpath - The path of the file to be uploaded.
info - Information about the file parts to be uploaded.
noProxy - Whether to not create file orders through OSS proxy.
encrypt - Whether the data needs to be encrypted (using proxy re-encryption technology)

Returns:

FileInfo - Information about the uploaded file.
error - An error if the upload fails.

type PartsInfo

type PartsInfo struct {
	ShadowHash string    `json:"shadow_hash,omitempty"`
	FileName   string    `json:"file_name,omitempty"`
	DirName    string    `json:"dir_name,omitempty"`
	Archive    string    `json:"archive,omitempty"`
	Territory  string    `json:"territory,omitempty"`
	Parts      []string  `json:"parts,omitempty"`
	PartsCount int       `json:"parts_count,omitempty"`
	TotalParts int       `json:"total_parts,omitempty"`
	PartSize   int64     `json:"-"`
	TotalSize  int64     `json:"total_size,omitempty"`
	UpdateDate time.Time `json:"update_date,omitempty"`
}

func CreatePartsInfoForDir

func CreatePartsInfoForDir(fpath, dirname, archive string) (PartsInfo, error)

CreatePartsInfoForDir creates parts information for a directory. It calculates the hash of each file in the directory and the shadow hash of the directory. Parameters:

fpath - The path of the directory.
dirname - The name of the directory.
archive - The name of the archive.

Returns:

PartsInfo - Information about the directory parts.
error - An error if the directory cannot be opened or read.

func CreatePartsInfoForFile

func CreatePartsInfoForFile(fpath, filename string, fileSize, partSize int64) (PartsInfo, error)

CreatePartsInfoForFile creates parts information for a file. It calculates the hash of each part and the shadow hash of the file. Parameters:

fpath - The path of the file.
filename - The name of the file.
fileSize - The size of the file.
partSize - The size of each part.

Returns:

PartsInfo - Information about the file parts.
error - An error if the file cannot be opened or read.

func RequestToUploadParts

func RequestToUploadParts(baseUrl, fpath, token, territory, filename, achive string, partSize int64) (PartsInfo, error)

RequestToUploadParts requests to upload file parts to the specified territory. It sends a POST request to the "/gateway/part-upload" endpoint with the necessary parameters. Parameters:

baseUrl - The base URL of the gateway.
token - The access token for authentication.
fpath - The path of the file to be uploaded.
territory - The territory to which the file will be uploaded.
filename - The name of the file to be uploaded.
achive - The archive name of the file to be uploaded.
partSize - The size of each part to be uploaded.

Returns:

PartsInfo - Information about the file parts to be uploaded.
error - An error if the request fails.

type Response

type Response struct {
	Code int    `json:"code"`
	Msg  string `json:"msg"`
	Data any    `json:"data"`
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL