Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@ dincli system register-wallet --account <account_id>

### Two layers: protocol contracts vs. per-model task contracts

- **Platform-level contracts** (deployed once by the DIN-Representative): `DinCoordinator` (ETH<->DIN exchange, slasher registry admin), `DinToken` (ERC20, minted on ETH deposit), `DinValidatorStake` (validator staking/slashing), `DinModelRegistry` (model registration, open-source vs proprietary fee). See `Documentation/public/workflows/din-workflow.md`.
- **Platform-level contracts** (seven, deployed once by the DIN-Representative behind transparent proxies via `foundry/script/DeployPlatform.s.sol`): `DinCoordinator` (ETH<->DIN exchange, slasher registry admin), `DinToken` (ERC20, minted on ETH deposit), `DinValidatorStake` (validator staking/slashing, auditor encryption keys), `DinModelRegistry` (model registration requests and approvals, open-source vs proprietary fee), `DinTreasury` (custody), `DinFeeRouter` (splits swept fees), `DinEmission` (per-GI reward subsidy on a decay schedule). See `Documentation/public/workflows/din-workflow.md`.
- **Task-level contracts** (deployed per model by the model owner): `DINTaskCoordinator` and `DINTaskAuditor`. These must be authorized as "slashers" on `DinValidatorStake` (via `DinCoordinator`, only callable by the DIN-Representative) *before* the model owner can register the model in `DinModelRegistry`. See `Documentation/public/workflows/model-workflow.md` for the full step-by-step (deploy → slasher auth request → genesis model → register → global iterations).

A model's lifecycle runs in **Global Iterations (GI)**: aggregator/auditor registration → Local Model Submission (LMS) by clients → auditor evaluation/scoring → two-tier aggregation (T1 sub-batches, T2 combines T1 into the new global model) → slashing of misbehaving validators → GI end. Each phase is opened/closed explicitly by the model owner via `dincli model-owner ...` subcommands, and other roles act only within an open phase.
A model's lifecycle runs in **Global Iterations (GI)**: the model owner funds the GI's reward pool (`gi start` refuses an unfunded GI) → aggregator/auditor registration (auditors also register an X25519 encryption key) → Local Model Submission (LMS) by clients → auditor batch seed lock → auditor batches → per-batch encrypted test data → auditor evaluation (commit, then reveal) → T1/T2 batch seed lock → two-tier aggregation (T1 sub-batches, T2 combines T1 into the new global model; each tier commit, then reveal) and the T2 score → slashing of misbehaving validators → GI end, which settles rewards → participants claim and the model owner releases the GI's registration slots. Each phase is opened/closed explicitly by the model owner via `dincli model-owner ...` subcommands, the seed locks are permissionless, and other roles act only within an open phase. Test-data and aggregation disputes are bonded and answered by the model owner (`dincli model-owner disputes ...`). The state names and ordinals are in `Documentation/technical/contracts/DINShared.md` §2.

### CLI structure (`dincli/`)

Expand All @@ -63,7 +63,7 @@ A model's lifecycle runs in **Global Iterations (GI)**: aggregator/auditor regis

### Manifest-driven, pluggable model services

Each model owner supplies their own Python service files (not part of this repo) implementing model-specific logic: `model.py` (architecture), `modelowner.py` (genesis model, scoring, audit test data), `client.py` (local training/DP), `auditor.py` (scoring), `aggregator.py` (T1/T2 aggregation). These are uploaded to IPFS and referenced by CID in a per-model `manifest.json` at `<root_dir>/tasks/<network>/task_<coordinator_address>/manifest.json`. `dincli` dynamically loads and calls these functions at runtime (`DinContext.load_custom_fn`), fetching them from IPFS on demand and caching by CID. `cache_model_0/` and `cid_services/` in this repo are example/reference service implementations (MNIST-style), not framework code.
Each model owner supplies their own Python service files (not part of this repo) implementing model-specific logic: `model.py` (architecture), `modelowner.py` (genesis model, scoring, audit test data), `client.py` (local training/DP), `auditor.py` (scoring), `aggregator.py` (T1/T2 aggregation). These are uploaded to IPFS and referenced by CID in a per-model `manifest.json` at `<root_dir>/tasks/<network>/<coordinator_address>/manifest.json`. `dincli` dynamically loads and calls these functions at runtime (`DinContext.load_custom_fn`), fetching them from IPFS on demand and caching by CID. `cache_model_0/` and `cid_services/` in this repo are example/reference service implementations (MNIST-style), not framework code.

Differential privacy is opt-in per model via a nested `dp` block in the manifest; when enabled, `client.py`'s training function applies the configured DP mechanism.

Expand Down
67 changes: 54 additions & 13 deletions Documentation/public/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,7 @@ The current devnet primarily focuses on:
* Validation workflows
* Ethereum-enforced protocol state transitions

Staking and slashing concepts exist within the protocol architecture.

Economic and reward distribution mechanisms are still under active development and are not yet the primary focus of the current devnet phase.
Staking, slashing and rewards are part of every Global Iteration: validators stake DIN to register, misbehaving or absent validators are slashed, and each GI pays clients, auditors and aggregators from a reward pool the model owner funds before starting it. After a GI ends, claim your share with `dincli rewards claim <model_id> --gi <gi>` and `dincli rewards withdraw <model_id>`. Reward splits, bond sizes and other economic parameters are not final and may change between devnet phases.

---

Expand Down Expand Up @@ -349,12 +347,18 @@ dincli task gi show-state 0

### Step 5: Check your Aggregation Batch (if state = `T1nT2Bcreated`)

If the state is still `LMSevaluationClosed`, lock the batch-assignment seed first (this moves it to `AggSeedLocked`; the model owner then creates the batches):

```bash
dincli aggregator lock-seed 0
```

```bash
# Check T1 batch assigned to you
dincli model-owner aggregation show-t1-batches 0 --detailed
dincli aggregator show-t1-batches 0 --detailed

# Check T2 batch assigned to you
dincli model-owner aggregation show-t2-batches 0 --detailed
dincli aggregator show-t2-batches 0 --detailed
```

---
Expand Down Expand Up @@ -395,6 +399,14 @@ dincli aggregator reveal-t2 0

> ⚠️ Aggregation is commit-then-reveal: a commit that is never revealed is slashed like a missed submission. Reveal from the same machine you committed from; the salt is cached locally.

### Step 9: Claim your reward (once state = `GIended`)

```bash
# credit your share of GI <gi> (the GI you took part in), then withdraw it
dincli rewards claim 0 --gi <gi>
dincli rewards withdraw 0
```

> 💡 T1 and T2 batch assignments depend on the current Global Iteration state and protocol allocation logic. A registered aggregator may not receive a batch in every iteration.

---
Expand Down Expand Up @@ -433,6 +445,9 @@ dincli auditor dintoken read-stake

# Register as auditor
dincli auditor register 0

# Once per wallet: register the encryption key your batch's test data is encrypted to
dincli auditor register-encryption-key
```

---
Expand All @@ -443,13 +458,19 @@ dincli auditor register 0
dincli task gi show-state 0
```

### Step 5: Check your Auditor Batch (if state = `AuditorsBatchesCreated`)
### Step 5: Check your Auditor Batch (if state = `AuditorsBatchesCreated` or `AuditTestDataAssigned`)

If the state is still `LMSclosed`, lock the batch-assignment seed first (this moves it to `AuditSeedLocked`; the model owner then creates the batches):

```bash
dincli auditor lock-seed 0
```

```bash
dincli auditor lms-evaluation show-batch 0
```

If a batch is shown, you will soon be required to audit it.
If a batch is shown, you will soon be required to audit it. The model owner then assigns each batch its test data (`AuditTestDataAssigned`) and opens evaluation.

---

Expand All @@ -459,14 +480,27 @@ If a batch is shown, you will soon be required to audit it.
dincli task gi show-state 0
```

### Step 7: Audit your assigned batch (if state = `LMSevaluationStarted`)
### Step 7: Audit your assigned batch (commit if state = `LMSevaluationStarted`, reveal if `LMSevaluationRevealStarted`)

```bash
# check your assigned batch
dincli auditor lms-evaluation show-batch 0

# audit your batch (scripts run automatically)
# audit your batch (scripts run automatically) and commit a hidden hash of each score
dincli auditor lms-evaluation evaluate 0 --submit

# once the model owner opens the reveal window: reveal your scores
dincli auditor lms-evaluation reveal 0
```

> ⚠️ Scoring is commit-then-reveal: a commit that is never revealed is slashed as a missed vote. Reveal from the same machine you committed from; the salt is cached locally.

### Step 8: Claim your reward (once state = `GIended`)

```bash
# credit your share of GI <gi> (the GI you took part in), then withdraw it
dincli rewards claim 0 --gi <gi>
dincli rewards withdraw 0
```
> 💡 Auditor batch assignments depend on the current Global Iteration state and protocol allocation logic. A registered auditor may not receive a batch in every iteration.

Expand Down Expand Up @@ -583,17 +617,24 @@ dincli system register-wallet --account 0
# Check ETH balance
dincli system --eth-balance

# Optional (Recommended Step) to ensure training is running fine locally
# Train locally without submitting
# Train a local model (nothing is sent on-chain)
dincli client train-lms 0

# Train and submit local model
dincli client train-lms 0 --submit
# Upload the trained model and submit it
dincli client submit-lm 0

# Show submitted models
dincli client lms show-models 0
```

### Step 4: Claim your reward (once state = `GIended`)

```bash
# credit your share of GI <gi> (the GI you took part in), then withdraw it
dincli rewards claim 0 --gi <gi>
dincli rewards withdraw 0
```

---

# 🧠 Final Notes
Expand Down
4 changes: 2 additions & 2 deletions Documentation/public/guides/fashion-mnist-cnn.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,11 +119,11 @@ From your project root (the directory where `dincli` stores tasks):
```bash
# deploy DINTaskCoordinator
dincli model-owner deploy task-coordinator \
--artifact "hardhat/artifacts/contracts/DINTaskCoordinator.sol/DINTaskCoordinator.json"
--artifact "foundry/out/DINTaskCoordinator.sol/DINTaskCoordinator.json"

# deploy DINTaskAuditor
dincli model-owner deploy task-auditor \
--artifact "hardhat/artifacts/contracts/DINTaskAuditor.sol/DINTaskAuditor.json"
--artifact "foundry/out/DINTaskAuditor.sol/DINTaskAuditor.json"
```

Both contract addresses are written to your local `.env` file. Request
Expand Down
4 changes: 2 additions & 2 deletions Documentation/public/guides/keystore-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ To select a named wallet at runtime, add `DIN_WALLET_NAME` to the `.env` used by
`docker-compose`, or pass `--wallet` with each command:

```bash
docker compose exec din-node dincli --wallet validator client train-lms 0 --submit
docker compose exec din-node dincli --wallet validator client submit-lm 0
```

### Host install
Expand All @@ -108,7 +108,7 @@ DIN_WALLET_NAME=validator
Or use the top-level flag:

```bash
dincli --wallet validator client train-lms 0 --submit
dincli --wallet validator client submit-lm 0
```

---
Expand Down
16 changes: 10 additions & 6 deletions Documentation/public/manifest.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,16 @@ The manifest is a JSON file containing the metadata for your model and task. It
The manifest file must be placed at:

```
<root_dir>/tasks/<network>/task_<coordinator_address>/manifest.json
<root_dir>/tasks/<network>/<coordinator_address>/manifest.json
```

For example:
```
<root_dir>/tasks/sepolia_op_devnet/task_0x1e31...4b133/manifest.json
<root_dir>/tasks/sepolia_op_devnet/0x1e31...4b133/manifest.json
```

> [!NOTE]
> If the manifest file is absent when the genesis setup (`dincli task model-owner create-genesis`) runs, it is automatically created with default values from the default manifest CID (`bafybeibbhrefnamky7vmgcrhfx4ptxb3bdemh5dtnvwfkpualoganvnuve`).
> `dincli model-owner model create-genesis-model` needs the manifest in the task directory. To start from the reference MNIST model, run `dincli model-owner task create-task-dir` and then `dincli model-owner task cache-default-artifacts`, which fetches the default manifest, services, ABIs and requirements (their CIDs are the `default_*` entries in `dincli/config/din_info.json`).

---

Expand All @@ -39,7 +39,7 @@ For example:

| Field | Required | Description |
|---|---|---|
| `Genesis_Model_CID` | Yes | IPFS CID of the genesis model. Set after running `dincli model-owner model create-genesis` |
| `Genesis_Model_CID` | Yes | IPFS CID of the genesis model. Set after running `dincli model-owner model create-genesis-model` |
| `DINTaskCoordinator_Contract` | Yes | Address of the deployed TaskCoordinator contract |
| `DINTaskAuditor_Contract` | Yes | Address of the deployed TaskAuditor contract |
| `dp` | No | Nested Differential Privacy configuration block used by services at runtime. See [Differential Privacy Parameters](#differential-privacy-parameters) below. |
Expand Down Expand Up @@ -152,8 +152,12 @@ Please find the example/template manifest file at [cache_model_0/manifest.json](

## Updating the Manifest

Once the model is registered and assigned a model ID, the manifest file should be updated with the `Model ID` field and re-uploaded to IPFS. The following dincli command can be used to update the model CID in the DINModelRegistry contract:
Once the model is registered and assigned a model ID, the manifest file can be updated (for example, with the `Model ID` field or new service CIDs) and re-uploaded. Changing the manifest CID in `DINModelRegistry` is a request, with its own fee, that the DIN-Representative approves:

```bash
dincli task model-owner update-manifest <model_id> [--modelCID <model_cid>] [--manifestpath <manifest_path>]
# model owner: upload the manifest and submit the update request
dincli task model-owner update-manifest-request <model_id> [--manifestpath <manifest_path> | --manifestCID <manifest_cid>]

# DIN-Representative
dincli dinrep registry approve-manifest-update <request_id>
```
43 changes: 42 additions & 1 deletion Documentation/public/roles/aggregators.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ dincli aggregator register <model_id> [--gi <gi_index>]
| `--gi <gi_index>` | No | The Global Iteration index. Defaults to the current GI if omitted |

> [!IMPORTANT]
> Aggregators Registration must be open for the current GI before to successfully register as an aggregator.
> Aggregator registration must be open for the current GI before you can register. A wallet registered as an aggregator can't also register as an auditor in the same GI.

---

Expand All @@ -59,6 +59,14 @@ Aggregators operate in two tiers:
- **Tier 1 (T1)** — Intermediate aggregation: combine local model batches into sub-global model updates.
- **Tier 2 (T2)** — Final aggregation: combine sub-global model updates into a single global model.

### Lock the Batch-Assignment Seed (optional, recommended)

```bash
dincli aggregator lock-seed <model_id> [--gi <gi_index>]
```

When evaluation closes, the T1/T2 batch assignment is drawn from a seed anchored to a future block. Anyone can lock it once that block is mined; locking it yourself stops the Model Owner from waiting out a draw they dislike. The command waits for the seed block, then locks (or skips if someone already has), and the GI moves to `AggSeedLocked`. `show-t1-batches` / `show-t2-batches` do the same automatically when run before batches exist. Only valid while the GI is in `LMSevaluationClosed`.

### Tier 1 Aggregation

**Lock the batch-assignment seed (optional, recommended):**
Expand Down Expand Up @@ -138,6 +146,38 @@ dincli aggregator aggregate-t2 <model_id> [--gi <gi_index>] [--batch <batch_id>]
dincli aggregator reveal-t2 <model_id> [--gi <gi_index>]
```

There is a single T2 batch, so `reveal-t2` takes no `--batch`.

---

## 4. Rewards

After the Model Owner ends the GI, credit your share of its reward pool and withdraw it:

```bash
# Credit your share for an ended GI
dincli rewards claim <model_id> --gi <gi_index>

# Transfer your whole claimable balance to your wallet
dincli rewards withdraw <model_id>
```

---

## 5. Disputing an Aggregation

Any active validator can dispute a finalized T1 or T2 batch's CID within the dispute window after it was finalized. It posts the coordinator's DIN dispute bond, and each batch can be disputed only once:

```bash
dincli aggregator dispute <model_id> --tier t1|t2 --batch <batch_id> [--gi <gi_index>]
```

The Model Owner either rejects the dispute (the bond is forfeited) or upholds it, which assigns a fresh subgroup of 3 aggregators to recompute the batch. If the recomputation confirms the dispute, or the fresh subgroup misses its deadline, your bond is credited back. Withdraw credited bonds with:

```bash
dincli aggregator claim-dispute-bond <model_id>
```

---

## Workflow
Expand All @@ -149,3 +189,4 @@ dincli aggregator reveal-t2 <model_id> [--gi <gi_index>]
5. **Reveal T1** — Run `reveal-t1` once the Model Owner opens the T1 reveal window (`T1AggregationRevealStarted`).
6. **Aggregate and commit T2** — If assigned to the final tier, run `aggregate-t2 --submit` when the Tier 2 phase opens (`T2AggregationStarted`).
7. **Reveal T2** — Run `reveal-t2` once the T2 reveal window opens (`T2AggregationRevealStarted`).
8. **Claim** — After the GI ends, run `rewards claim` and `rewards withdraw`.
Loading
Loading