README
¶
Curio Storage - 储存
Documentation
For detailed documentation and usage instructions, please refer to the official documentation.
Overview
Curio Storage is an advanced platform designed to simplify the setup and operation of storage providers within the Filecoin ecosystem. Building on the foundation of lotus-miner, Curio Storage offers enhanced redundancy, scalability, and fault tolerance, ensuring efficient and reliable data storage solutions.
Key Features
- Redundancy: Curio Storage employs multiple daemons, worker types, and database nodes to eliminate single points of failure and ensure maximum uptime.
- Simplicity: Consolidates all functions into a single binary, reducing the complexity of operations and minimizing the need for separate components.
- Scalability: Utilizes a greedy task management system for superior scaling capabilities, allowing operations to grow from 1 machine to thousands seamlessly.
- Fault Tolerance: Features robust mechanisms for handling disruptions, including remote failure recovery, automatic work balancing, and fallback-capable components.
- Flexible Configuration: Offers fine-grained configuration options to tailor the system to specific needs, enabling various subsystems like sealing and storage.
- Multiple Miner IDs: Efficiently manages multiple Miner IDs on the same hardware, optimizing resource utilization.
- User Interface (GUI): Includes a comprehensive web-based dashboard for real-time monitoring and management of mining operations.
Key Benefits for Storage Providers
Curio Storage is designed to deliver significant improvements and new features for storage providers, including:
- High Availability / Zero-Downtime Capable: Ensures continuous operation without downtime.
- Efficient Resource Utilization: Better use of machines by stacking work tightly, reducing waste.
- Scalability: No single point of stress, allowing for seamless expansion from a single machine to thousands.
- Multi-Miner-ID Support: Saves hardware resources by enabling multi-proving.
- Rich Filtering: Allows detailed filtering at storage locations by ID, purpose, etc.
- Better Hardware Efficiency: Tasks pack tightly onto any machine that has that work layer enabled.
- Debian/Ubuntu APT Repo: Automatic updates & no building (for the common case).
- Simple Migrator: Switch from Lotus Miner to Curio in minutes.
Future Developments
Since release, Curio's team has added:
- New Snap Sectors & Unsealing: Completing the feature set of lotus-miner.
- Curio SupraSeal (beta) integration: 10x more efficient Sealing than through other means (Read docs).
- Disk Garbage Collection: Know you're storing only what you need.
- Binary size reduction: All this functionality is now in a binary under 100MiB.
Curio Storage is continuously evolving, with future plans including:
- Built-In Markets (like Boost) with Automation: Enhancing market capabilities with automation.
- Staking, Resource Sharing, & Deals 2.0: Introducing advanced features for resource management.
- Efficiency Gains: Achieving significant cost savings for storage providers of all sizes.
- Non-Interactive Proof of Replication (NI-PoRep): Unlocking the potential for non-sealing storage providers.
- Two-Hour-Per-Month Admin Goal: Streamlining operations to require minimal administrative effort.
General Availability
Curio Storage is now GA. We encourage you to migrate your lotus-miner to Curio. For testing, you can deploy or migrate your miner on the calibrationnet. Your feedback is invaluable and will help us improve Curio.
Getting Started
We recommend reading docs.CurioStorage.org where you may have a pre-built binary for your system.
One way to get started with Curio Storage, follow these steps:
-
Clone the Repository:
git clone https://github.com/filecoin-project/curio.git cd curio -
Build the Project:
make clean all -
Run the Guided Setup:
curio guided-setup
Community and Support
Join our community discussions and seek support via:
- Slack: #fil-curio-help
- GitHub Issues: Submit an Issue
License
Dual-licensed under MIT + Apache 2.0
Contributing
We welcome contributions from the community! Feel free to open PRs. Reach out to the team on Slack if you need any help.
Maintainers cutting a release should follow the Curio Release Process.
Flow EVM Integration
Curio participates in the Flow ecosystem via Flow EVM (Chain ID 747). This enables contributors to use Flow EVM endpoints and receive rewards through the ReWTF program.
Flow EVM RPC configuration:
- RPC URL: https://mainnet.evm.nodes.onflow.org
- Chain ID: 747
- Currency Symbol: FLOW
- Block Explorer: https://evm.flowscan.io/
Curio Storage is a testament to our commitment to enhancing the Filecoin ecosystem, providing a robust and user-friendly platform for all storage providers. Join us on this journey and contribute to the future of decentralized storage. 💙
Directories
¶
| Path | Synopsis |
|---|---|
|
Package alertmanager provides a framework for monitoring and alerting within the Curio project.
|
Package alertmanager provides a framework for monitoring and alerting within the Curio project. |
|
docgen-openrpc/cmd
command
|
|
|
gen/api
command
|
|
|
Run from curio root.
|
Run from curio root. |
|
cmd
|
|
|
curio
command
|
|
|
curio/guidedsetup
guidedSetup for migration from lotus-miner to Curio
|
guidedSetup for migration from lotus-miner to Curio |
|
curio/internal/translations
Usage: To UPDATE translations:
|
Usage: To UPDATE translations: |
|
curio/rpc
Package rpc provides all direct access to this node.
|
Package rpc provides all direct access to this node. |
|
curio/tasks
Package tasks contains tasks that can be run by the curio command.
|
Package tasks contains tasks that can be run by the curio command. |
|
pdptool
command
|
|
|
skiff
command
|
|
|
sptool
command
|
|
|
servicedeps
Package servicedeps holds shared HTTP service dependency handles without pulling in the full cuhttp server / deal-market dependency graph.
|
Package servicedeps holds shared HTTP service dependency handles without pulling in the full cuhttp server / deal-market dependency graph. |
|
Package deps provides the dependencies for the curio node.
|
Package deps provides the dependencies for the curio node. |
|
config/cfgdocgen
command
|
|
|
harmony
|
|
|
harmonytask
Package harmonytask implements a pure (no task logic), distributed task manager.
|
Package harmonytask implements a pure (no task logic), distributed task manager. |
|
harmonytask/internal/acceptcache
Package acceptcache stores the result of a CanAccept() call so the scheduler can reuse it on the next considerWork cycle without paying the (potentially expensive) cost of re-evaluating CanAccept.
|
Package acceptcache stores the result of a CanAccept() call so the scheduler can reuse it on the next considerWork cycle without paying the (potentially expensive) cost of re-evaluating CanAccept. |
|
harmonytask/internal/peerregistry
Package peerregistry manages the in-memory routing table from task-type names to peer connections.
|
Package peerregistry manages the in-memory routing table from task-type names to peer connections. |
|
harmonytask/internal/preemptbids
Package preemptbids brokers cross-node preempt-cost responses for a time-sensitive scheduling decision.
|
Package preemptbids brokers cross-node preempt-cost responses for a time-sensitive scheduling decision. |
|
harmonytask/internal/runnowflags
Package runnowflags is a small, concurrent registry of per-task-name "run now" flags.
|
Package runnowflags is a small, concurrent registry of per-task-name "run now" flags. |
|
harmonytask/internal/runregistry
Package runregistry tracks the set of currently-executing tasks on a single node.
|
Package runregistry tracks the set of currently-executing tasks on a single node. |
|
harmonytask/pipetest
Package pipetest provides in-memory PeerConnectorInterface / PeerConnection implementations for integration testing.
|
Package pipetest provides in-memory PeerConnectorInterface / PeerConnection implementations for integration testing. |
|
resources/ffigpu
Package ffigpu provides Curio's filecoin-ffi-backed GPU probe as a resources.ResourceInspector.
|
Package ffigpu provides Curio's filecoin-ffi-backed GPU probe as a resources.ResourceInspector. |
|
resources/miniopencl
Package cl was borrowed from the go-opencl library which is more complex and doesn't compile well for our needs.
|
Package cl was borrowed from the go-opencl library which is more complex and doesn't compile well for our needs. |
|
itests
|
|
|
lib
|
|
|
apiconn
Package apiconn parses Lotus-compatible FULLNODE_API_INFO strings without importing github.com/filecoin-project/lotus.
|
Package apiconn parses Lotus-compatible FULLNODE_API_INFO strings without importing github.com/filecoin-project/lotus. |
|
ffiselect/ffidirect
This is a wrapper around the FFI functions that allows them to be called by reflection.
|
This is a wrapper around the FFI functions that allows them to be called by reflection. |
|
harmony_peer_http
Package harmonypeerhttp implements the HTTP transport for harmonytask's peer-to-peer messaging protocol.
|
Package harmonypeerhttp implements the HTTP transport for harmonytask's peer-to-peer messaging protocol. |
|
lazy
Package lazy provides a tiny once-only initializer used by pdpnode and deps.
|
Package lazy provides a tiny once-only initializer used by pdpnode and deps. |
|
parkpiece
Package parkpiece provides race-safe upsert helpers for the parked_pieces table.
|
Package parkpiece provides race-safe upsert helpers for the parked_pieces table. |
|
paths/mocks
Package mocks is a generated GoMock package.
|
Package mocks is a generated GoMock package. |
|
proof/sn-comp-sector-seal
command
|
|
|
robusthttp
Package robusthttp implements automatic retry logic for HTTP transfers that can recover from TCP streams that become "stuck" at extremely low throughput.
|
Package robusthttp implements automatic retry logic for HTTP transfers that can recover from TCP streams that become "stuck" at extremely low throughput. |
|
market
|
|
|
mk12
Package market
|
Package market |
|
mk20/http
Package http Code generated by swaggo/swag.
|
Package http Code generated by swaggo/swag. |
|
Package pdpnode runs PDP workflows without the curio sealing/PoRep dependency graph.
|
Package pdpnode runs PDP workflows without the curio sealing/PoRep dependency graph. |
|
scripts
|
|
|
cbongen-guidedsetup
command
Generates CBOR marshal/unmarshal code for guidedsetup.SectorInfo.
|
Generates CBOR marshal/unmarshal code for guidedsetup.SectorInfo. |
|
docgen/cmd
command
|
|
|
fiximports
command
|
|
|
metricsdocgen
command
Package main generates Prometheus metrics documentation from source code.
|
Package main generates Prometheus metrics documentation from source code. |
|
replaceTranslations
command
|
|
|
translationcheck
command
|
|
|
tasks
|
|
|
message/replace_mock
Package replace_mock is a generated GoMock package.
|
Package replace_mock is a generated GoMock package. |
|
tasknames
Package tasknames holds harmony task name strings used in harmony_task.name and alert filters.
|
Package tasknames holds harmony task name strings used in harmony_task.name and alert filters. |
|
Package web defines the HTTP web server for static files and endpoints.
|
Package web defines the HTTP web server for static files and endpoints. |
|
api
Package api provides the HTTP API for the lotus curio web gui.
|
Package api provides the HTTP API for the lotus curio web gui. |
|
devsrv
command
|
Click to show internal directories.
Click to hide internal directories.