Crust Wiki

Crust Wiki

  • 文档
  • Shadow
  • 贡献
  • Languages icon中文
    • English
    • Help Translate

›Node Guide

概览

  • Crust 概述
  • Crust Grants
  • Crust 生态成长计划
  • CRU 认领
  • 锁定的CRU 认领
  • 锁定的CRU 解锁
  • Bridge

    • Ethereum Bridge
    • Elrond Bridge
  • Crust 钱包
  • Crust 术语
  • CRU18 担保
  • 参数表
  • 贡献

学习

  • 账户
  • Crust 通证
  • 新增绑定
  • 担保人
  • 验证人
  • GPoS
  • sWorker

    • 概览
    • 入网
    • 工作量
  • 去中心化存储市场
  • 存储商户
  • 链上身份
  • 链上治理指南

构建

  • Builder's Portal
  • Crust Storage 101
  • Basics

    • Developer faucet
    • Crust Rocky Network
    • 代码示例:使用Crust存储文件
    • Store file with Crust IPFS Pinning Service API

    Crosschain Storage Solution

    • 基于XCMP的跨链存储解决方案
    • 基于平行链的跨链存储解决方案
    • 基于原生IPFS的跨链存储解决方案
    • 基于智能合约的跨链存储解决方案

    Integration Guide

    • DApp的部署和运行
    • NFT数据存储
    • 内容存储与分发

    Node Guide

    • Crust 节点
    • Crust Storage Manager

    Toolkits

    • Crust Pinner Github Action
    • Crust Pinner NPM Package
    • IPFS W3Auth Gateway
    • IPFS W3Auth Pinning Service

基于EVM构建

  • Overview
  • Build With EVM 101
  • Chains

    • Ethereum
    • Optimism
    • Arbitrum
    • zkSync

    Toolkits

    • SDK

基于Algorand构建

  • Overview
  • Build With Algorand 101
  • Algorand applications

基于TON构建

  • Overview
  • Build With TON 101
  • TON applications

节点

  • 节点概要
  • 节点硬件指南
  • Owner 节点
  • Member 节点
  • Isolation 节点
  • 验证人指南
  • 担保人指南
  • sWorker 版本
  • 节点权益
  • 配置 QoS

存储

  • 概览
  • 存储用户指南
  • 存储商户指南
  • 存储订单清算指南
  • 存储市场权益
  • 使用Crust Apps 存储的问题

Q&A

  • 基础知识
  • 验证人和候选人
  • 担保人
  • 奖励和惩罚
  • 节点基本问题
  • Member节点相关
  • 组相关
  • 修复不稳定链
  • 应用
  • EPID & ECDSA
  • 其它
Translate

Crust Storage Manager

sManager (Storage Manager) is a file picking bot which continuously picking and handling files from Crust Network. Node operators can customize this module to implement their own file handling strategy.

Main Flow

sManager consists of a bunch of recurring jobs exchanging data via local database and application context. The main workflow is illustrated as below:

smanager-flow

  • Chain Event Indexer subscribes chain event, and indexes latest file records to local database's File Records Queue

  • Chain Database Indexer parses file records from Crust Network on-chain database, and indexes historical file records to local database's File Records Queue

  • Pull Scheduler periodically picks file records from File Records Queue based on its picking strategy, pulls the file from IPFS network by pinning it to local IPFS node, and then seals the file by calling sWorker's seal API.

Usage

  1. Clone repo
git clone https://github.com/crustio/crust-smanager.git
  1. Installing

It's recommended to use volta as the node version manager. Please follow the volta docs to install it.

cd crust-smanager && npm i
  1. Debug
npm run dev
  1. Run in Prod
npm run build
npm start

It's recommended to run sManager using Docker with the restart=always restart policy.

A daemon guard should be configured if you want to run sManager natively without docker. Tools like pm2 and nodemon could be used.

Configuration

sManager could be configured with a json format file, whose name is specified with environment variable SMANAGER_CONFIG. If not set, default file name smanager-config.json will be used.

You could check smanager-config.example.json as a sample config file:

{
  "chain": {
    "account": "crustaccount",
    "endPoint": "ws://localhost:19944"
  },
  "sworker": {
    "endPoint": "http://localhost:12222"
  },
  "ipfs": {
    "endPoint": "http://localhost:5001"
  },
  "node": {
    "role": "member"
  },
  "telemetry": {
    "endPoint": "https://sm-submit.crust.network"
  },
  "dataDir": "data",
  "scheduler": {
    "minSrdRatio": 30,
    "sealFailedRetryCount": 3,
    "sealFailedRetryInterval": 3,
    "strategy": {
      "dbFilesWeight": 0,
      "newFilesWeight": 100
    }
  }
}

Those config items will be loaded in the sManager configuration setup process. The meaning of each item is as follows:

  • chain.account: your member account

  • chain.endPoint: your chain endpoint

  • sworker.endPoint: your sWorker endpoint

  • ipfs.endPoint: your IPFS endpoint

  • dataDir: the directory of the database of sManager

  • scheduler.minSrdRatio: a minimum ratio of SRD that one node can start to accept storage orders.

    For example, if the ratio is 30, your node will start to accept storage order once the ratio of SRD capacity is higher than 30%

  • scheduler.sealFailedRetryCount: retry count of failed seal tasks

  • scheduler.sealFailedRetryInterval: retry interval of failed seal tasks, unit in hours

  • scheduler.strategy.dbFilesWeight: how much bandwidth of this node will be used to fetch and store the history storage orders (Storage orders in the past four months).

  • scheduler.strategy.newFilesWeight: how much bandwidth of this node will be used to fetch and store the newest storage orders.

Components

sManager is designed to have several tasks running independently. Tasks are either scheduled by the block event or by configured intervals. Each task plays as an actor which consumes/produces some information and communicate with other tasks through the local database or application context.

sManager follows the Fails Early principle which means it will shutdown on any unexpected error. To support this principle, tasks are designed to be recoverable after application restarts.

Local Database

The local database stores below information:

  1. File Records: The files metadata (tips, size, replicas count, expire time, etc) on Crust Network.

  2. File and Owner Relationship: sManager also maintains the relationship between a file and an on-chain account. This information will help making better pulling decision.

  3. Chain Metadata: E.g. the block and time on chain.

  4. Pin Records: The pin history of files.

  5. Cleanup Records: The files needs to removed from local file system, normally this is triggered when a file expires on Crust Network.

Checkout Db Schema for the schema details.

Indexers

Indexers extract information into the local database from various data sources. Currently sManager has implemented below indexers:

  1. Chain Database Indexer: Index file records from the Crust Network on-chain database.

  2. Chain Event Indexer: Index file records by listening latest chain event.

  3. Chain Time Indexer: A simple indexer which pushes the latest block height and it's timestamp to the config table.

Simple Tasks

Simple tasks are specialized tasks which runs periodically. Currently sManager has implemented below tasks:

  1. Group Info Updater: Update sworker identity information from sworker api.

  2. Ipfs Gc: Schedule ipfs gc periodically.

  3. Telemetry Reporting: Report smanager statistics information to the telemetry server.

  4. Pull Scheduler: Schedule file pulling based on configured strategy.

  5. Seal Status Updater: Update sealing status periodically.

  6. File Retry Task: Retry pulling if possible.

  7. File Cleanup Task: Cleanup deleted files from local file system.

Resources

  • crust-smanager: https://github.com/crustio/crust-smanager
  • crust.js: https://github.com/crustio/crust.js
  • crust: https://github.com/crustio/crust
  • crust-sworker: https://github.com/crustio/crust-sworker
  • js-ipfs: https://github.com/ipfs/js-ipfs
← Crust 节点下一篇 →
  • Main Flow
  • Usage
  • Configuration
  • Components
    • Local Database
    • Indexers
    • Simple Tasks
  • Resources
Docs
Getting StartedCRU ClaimsWebsite Hosting with CrustNFT Data Storage with Crust
Community
DiscordTwitterTelegram
More
CooperationGitHub
Copyright © 2025 Crust Network