process

package
v0.6.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 12, 2024 License: Apache-2.0 Imports: 20 Imported by: 2

Documentation

Index

Constants

View Source 
const (
	CANS_PROTO_FLAG           = "CANS_PROTO_"
	CANS_ARCHIVE_FORMAT_ZIP   = "zip"
	CANS_ARCHIVE_FORMAT_TAR   = "tar"
	CANS_ARCHIVE_FORMAT_TARGZ = "tar.gz"
)

Variables

This section is empty.

Functions

func AddCansProtoRequestHeader added in v0.6.0 

func AddCansProtoRequestHeader(fname, cipher string, chunksNum, chunksId int, totalSize int64, isSplit bool, archiveFormat string) func(req *http.Request)

AddCansProtoRequestHeader is used to set the request header of CANS PROTOCOL based on file block upload

Receive parameter:

  • fname: the name of the file.
  • cipher: symmetric encryption key, used to encrypt data using AES
  • chunksNum: total number of file chunks.
  • chunksId: index of the current chunk to be uploaded ([0,chunksNum)).
  • totalSize: chunks total size (byte), can be obtained from the first return value of SplitFile
  • isSplit: indicate whether the file can be split into different cans.
  • archiveFormat: Specifies the compression format of the file. If it is "", no compression is performed. Currently supported formats are: "zip", "tar", and "tar.gz"

Return parameter:

  • handleFunc: function to set request headers.

func AddFileRequestHeader added in v0.6.0 

func AddFileRequestHeader(territory, bucket, account, message, sig, contentType string) func(req *http.Request)

AddUploadFileRequestHeader is used to set the request header for file upload

Receive parameter:

  • bucket: the territory to which the file will be uploaded, formerly known as bucket
  • territory: the territory(a space block) in which you would like your data to be stored
  • account: CESS account to which the territory belongs
  • message: message to sign
  • sig: signature of the above message using the above CESS account
  • contentType: chunks total size (byte), can be obtained from the first return value of SplitFile

Return parameter:

  • handleFunc: function to set request headers.

func AddUploadChunkRequestHeader added in v0.6.0 

func AddUploadChunkRequestHeader(fname, cipher string, chunksNum, chunksId int, totalSize int64) func(req *http.Request)

AddUploadChunkRequestHeader is used to set the request header for file chunk upload request

Receive parameter:

  • fname: the name of the file.
  • cipher: symmetric encryption key, used to encrypt data using AES
  • chunksNum: total number of file chunks.
  • chunksId: index of the current chunk to be uploaded ([0,chunksNum)).
  • totalSize: chunks total size (byte), can be obtained from the first return value of SplitFile

Return parameter:

  • handleFunc: function to set request headers.

func BatchEncryptWithAES added in v0.6.0 

func BatchEncryptWithAES(files []string, cipher string, saveDir string) ([]string, error)

BatchEncryptWithAES encrypt a batch of files with AES

Receive parameter:

  • files: file list
  • cipher: encryption and decryption cipher
  • saveDir: encrypted files save directory

Return parameter:

  • string: encrypted files
  • error: error message

func DownloadCanFile added in v0.6.0 

func DownloadCanFile(url, mnemonic, savepath, fid, filename, cipher string, sid int) error

DownloadCanFile download files in the directory from the gateway with CANS PROTOCOL,

Receive parameter:

  • url: the address of the gateway.
  • mnemonic: user's CESS account mnemonic.
  • savepath: file path to store downloaded file.
  • fid: file's FID on chain metadata.
  • filename: name of sub file in cans, if it is an empty string, the specified segment(can) is downloaded.
  • cipher: symmetric encryption key, used to encrypt data using AES.
  • sid: segment hash, if filename is an empty string, the specified segment(can) is downloaded.

Return parameter:

  • response: file(if successful).
  • error: error message.

func EncryptWithAES added in v0.6.0 

func EncryptWithAES(file string, cipher string, saveDir string) (string, error)

EncryptWithAES encrypt a file with AES

Receive parameter:

  • file: file
  • cipher: encryption and decryption cipher
  • saveDir: encrypted file save directory

Return parameter:

  • string: encrypted file
  • error: error message

func ExtractSegmenthash 

func ExtractSegmenthash(segment []string) []string

func FillAndCut added in v0.6.0 

func FillAndCut(file string, saveDir string) ([]string, error)

FillAndCut fill and cut files

Receive parameter:

  • file: the file to be processed
  • saveDir: segment save directory

Return parameter:

  • []string: segment list
  • error: error message

func FillAndCutWithAESEncryption added in v0.6.0 

func FillAndCutWithAESEncryption(file string, cipher string, saveDir string) ([]string, error)

FillAndCutWithAESEncryption fill and cut files, then encrypt using AES algorithm

Receive parameter:

  • file: the file to be processed
  • cipher: encryption and decryption cipher
  • saveDir: segment save directory

Return parameter:

  • []string: segment list
  • error: error message

func FullProcessing added in v0.6.0 

func FullProcessing(file string, cipher string, savedir string) ([]chain.SegmentDataInfo, string, error)

FullProcessing perform full process processing on the file

Receive parameter:

  • file: the file to be processed
  • cipher: encryption and decryption cipher
  • saveDir: saved directory after processing

Return parameter:

  • []segmentDataInfo: segment and fragment information of the file
  • string: [fid] unique identifier for the file
  • error: error message

func Redundancy added in v0.6.0 

func Redundancy(segment []string, saveDir string) ([]chain.SegmentDataInfo, string, error)

Redundancy calculate redundancy for files

Receive parameter:

  • segment: the file to be processed
  • saveDir: fragment save directory

Return parameter:

  • []chain.SegmentDataInfo: segment info
  • string: fid
  • error: error message

func RetrieveFile added in v0.6.0 

func RetrieveFile(url, fid, mnemonic, savepath string) error

RetrieveFile downloads files from the gateway

  • url: gateway url
  • fid: fid
  • mnemonic: polkadot account mnemonic
  • savepath: file save path

Return:

  • string: fid
  • error: error message

func RetrieveObject added in v0.6.0 

func RetrieveObject(url, fid, mnemonic string) (io.ReadCloser, error)

RetrieveObject gets the object from the gateway

  • url: gateway url
  • fid: fid
  • mnemonic: polkadot account mnemonic

Return:

  • io.ReadCloser: object
  • error: error message

func SplitFile added in v0.6.0 

func SplitFile(fpath, chunksDir string, chunkSize int64, filling bool) (int64, int, error)

Split File into Chunks.

Receive parameter:

  • fpath: the path of the file to be split.
  • chunksDir: directory path to store file chunks, please do not mix it elsewhere.
  • chunkSize: the size of each chunk, it does not exceed the file size

Return parameter:

  • int64: chunks total size (byte).
  • int: number of file chunks.
  • error: error message.

func SplitFileWithstandardSize added in v0.6.0 

func SplitFileWithstandardSize(fpath, chunksDir string) (int64, int, error)

Split File into Chunks with standard size. It split file into chunks of the default size and fills the last chunk that does not meet the size.

Receive parameter:

  • fpath: the path of the file to be split.
  • chunksDir: directory path to store file chunks, please do not mix it elsewhere.

Return parameter:

  • int64: chunks total size (byte).
  • int: number of file chunks.
  • error: error message.

func StoreFile added in v0.6.0 

func StoreFile(url, file, bucket, territory, mnemonic string) (string, error)

StoreFile stores files to the gateway

Receive parameter:

  • url: gateway url
  • file: stored file
  • bucket: bucket for storing file, it will be created automatically
  • territory: territory name
  • mnemonic: polkadot account mnemonic

Return parameter:

  • string: [fid] unique identifier for the file.
  • error: error message.

Preconditions:

  1. Account requires purchasing space, refer to [BuySpace] interface.
  2. Authorize the space usage rights of the account to the gateway account, refer to the [AuthorizeSpace] interface.
  3. Make sure the name of the bucket is legal, use the [CheckBucketName] method to check.

Explanation:

  • Account refers to the account where you configured mnemonic when creating an SDK.
  • CESS public gateway address: [https://deoss-pub-gateway.cess.network/]
  • CESS public gateway account: [cXhwBytXqrZLr1qM5NHJhCzEMckSTzNKw17ci2aHft6ETSQm9]

func StoreObject added in v0.6.0 

func StoreObject(url string, bucket, territory, mnemonic string, reader io.Reader) (string, error)

StoreObject stores object to the gateway

Receive parameter:

  • url: gateway url
  • bucket: the bucket for storing object, it will be created automatically
  • territory: territory name
  • mnemonic: polkadot account mnemonic
  • reader: strings, byte data, file streams, network streams, etc

Return parameter:

  • string: [fid] unique identifier for the file
  • error: error message

Preconditions:

  1. Account requires purchasing space, refer to [BuySpace] interface.
  2. Authorize the space usage rights of the account to the gateway account, refer to the [AuthorizeSpace] interface.
  3. Make sure the name of the bucket is legal, use the [CheckBucketName] method to check.

Explanation:

  • Account refers to the account where you configured mnemonic when creating an SDK.
  • CESS public gateway address: [https://deoss-pub-gateway.cess.network/]
  • CESS public gateway account: [cXhwBytXqrZLr1qM5NHJhCzEMckSTzNKw17ci2aHft6ETSQm9]

func UploadFileChunk added in v0.6.0 

func UploadFileChunk(url, mnemonic, file, territory, bucket string, addExtendHeader func(*http.Request)) (string, error)

UploadFileChunk upload chunk of file to the gateway

Receive parameter:

  • url: the address of the gateway.
  • mnemonic: the space owner's CESS account mnemonic
  • file: file path to store file chunks.
  • territory: the territory(a space block) in which you would like your data to be stored
  • bucket: the bucket name to store user data.
  • fname: the name of the file.
  • chunksNum: total number of file chunks.
  • chunksId: index of the current chunk to be uploaded ([0,chunksNum)).
  • totalSize: chunks total size (byte), can be obtained from the first return value of SplitFile

Return parameter:

  • response: chunk ID or file's FID(if all chunks are uploaded successfully).
  • error: error message.

func UploadFileChunks added in v0.6.0 

func UploadFileChunks(url, mnemonic, chunksDir, territory, bucket, fname, cipher string, chunksNum int, totalSize int64) (string, error)

UploadFileChunks upload file chunks in the directory to the gateway as much as possible, chunks will be removed after being uploaded, if the chunks are transferred successfuly.

Receive parameter:

  • url: the address of the gateway.
  • mnemonic: the space owner's CESS account mnemonic
  • chunksDir: directory path to store file chunks, please do not mix it elsewhere.
  • territory: the territory(a space block) in which you would like your data to be stored
  • bucket: the bucket name to store user data.
  • fname: the name of the file.
  • cipher: symmetric encryption key, used to encrypt data using AES
  • chunksNum: total number of file chunks.
  • totalSize: chunks total size (byte), can be obtained from the first return value of SplitFile

Return parameter:

  • response: file's FID(if all chunks are uploaded successfully).
  • error: error message.

func UploadFilesWithCansProto added in v0.6.0 

func UploadFilesWithCansProto(url, mnemonic, filesDir, territory, bucket, archiveFormat, cipher string, isSplit bool) (string, error)

UploadFilesWithCansProto upload files in the directory to the gateway with CANS PROTOCOL,

Receive parameter:

  • url: the address of the gateway.
  • mnemonic: the space owner's CESS account mnemonic
  • filesDir: directory path to store file chunks, please do not mix it elsewhere.
  • territory: the territory(a space block) in which you would like your data to be stored
  • bucket: the bucket name to store user data.
  • archiveFormat: Specifies the compression format of the file. If it is "", no compression is performed. Currently supported formats are: "zip", "tar", and "tar.gz"
  • cipher: symmetric encryption key, used to encrypt data using AES
  • isSplit: indicate whether the file can be split into different cans.

Return parameter:

  • response: file's FID(if all chunks are uploaded successfully).
  • error: error message.

Types

type CansRequestParams added in v0.6.0 

type CansRequestParams struct {
	SegmentIndex int    `json:"segment_index"`
	SubFile      string `json:"sub_file"`
	Cipher       string `json:"cipher"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.