Clarify what can be trained on DIN and how to bring a dataset #157

Activity

1. Santiagocetran

   added

   documentationImprovements or additions to documentationImprovements or additions to documentation

   on Sep 21, 2026
2. umeradl commented on Sep 24, 2026

   

   umeradl

   on Sep 24, 2026

   Last edited by umeradl

   Member

   More actions

   Thanks @Santiagocetran, agreed: this needs its own doc. The core question, "what can be trained on DIN?", can already be answered, so here is the answer plus an outline for the doc.

   Owners / reviewers: this doc should be written and reviewed together by @abrahamnash, @umeradl and @Santiagocetran, alongside the partner-introduction doc for #158. The two should link to each other.

   Short answer: what can be trained on DIN

   In theory, any model can be trained on DIN. Models from a few parameters up to MBs or GBs all work. What makes this possible is the per-model custom services (model.py, modelowner.py, client.py, auditor.py, aggregator.py). The model owner defines how local models are trained, how they are audited and scored, and how local and sub-global models are aggregated (T1 → T2). The protocol itself doesn't care about model type. On-chain it only coordinates phases, stakes, and IPFS CIDs, so gas costs don't grow with model size.

   The practical limit is hardware and availability. Model size and training cost run into the hardware of the clients, auditors, and aggregators, and into how many of them are online during each Global Iteration.

   • Model families: linear/logistic regression, classical ML, ANNs, deep learning, CNNs, MobileNets and other on-device architectures, RNNs/LSTMs, transformers.
   • LLMs: fine-tuned with federated parameter-efficient methods such as FedLoRA / FedQLoRA. Only the small adapter weights move through the network, not the full base model.
   • Frameworks: PyTorch, TensorFlow/Keras, scikit-learn, JAX, or anything else Python can call. Each model declares its own requirements.txt in the manifest.
   • FL algorithms: any algorithm that fits the train → audit → aggregate loop. Examples: FedAvg, FedProx, FedAdam/FedYogi/FedAdagrad (FedOpt), SCAFFOLD, FedNova, q-FFL. Byzantine-robust aggregators (Krum, trimmed mean, coordinate-wise median) are also an option, and they work alongside the auditor scoring layer.
   • Types of FL: we focus mainly on horizontal FL, where clients hold different samples with the same features. That is the default path and what the MNIST reference implements. Other setups are also possible by customizing the training, audit, and aggregation services:
     • Vertical FL (clients hold different features for the same entities). This needs more custom work: entity alignment and split-model logic in the services.
     • Device/tier heterogeneity-aware FL, e.g. HeteroFL or FjORD-style sub-models sized to each client's hardware.
     • Personalized FL, e.g. per-client heads or Per-FedAvg-style personalization.
     • Federated transfer learning, where a pre-trained backbone is frozen and only the head is trained.
   • Privacy add-ons: differential privacy is already opt-in per model through the manifest dp block. Update compression and quantization, and secure-aggregation-style schemes, can be added inside the services.

   What to be honest about in the doc

   • Proven today vs. possible. The only end-to-end tested example is the MNIST reference in cache_model_0/services/. Anything else is supported by design but needs the model owner, possibly with our help, to write and test the services.
   • Rounds are synchronous. Each GI phase is opened and closed by the model owner, so fully asynchronous FL doesn't fit natively.
   • Bandwidth and storage. Every client, auditor, and aggregator pulls and pushes model artifacts through IPFS in each GI. For large models this cost usually matters more than compute.
   • Audit data. Auditors score submissions against model-owner-provided test data, so the model owner needs a held-out evaluation set and a scoring function that makes sense for the task.
   • Trust in service code. Participants run the model owner's Python services on their own machines. This is why DINModelRegistry puts an approval gate on both steps. A model is only registered after the DIN DAO approves its registration request (requestModelRegistration → approveModel / rejectModel). Any later manifest change, including changed service CIDs, only takes effect after the DIN DAO approves the update request (requestManifestUpdate → approveManifestUpdate / rejectManifestUpdate). A model owner can't quietly swap in new code for participants to run. The doc should explain this review step and what participants can check themselves, e.g. that the manifest CID they fetch matches the approved on-chain record.
   • Network/version. The doc should say which network and version it covers. Following the Documentation/ convention, that means DevNet 2.0 on develop, not the live DevNet 1.0 Sepolia deployment.

   Proposed doc outline

   Proposed location: Documentation/public/what-can-be-trained.md, linked from getting-started.md.

   1. Can I train my model on DIN today? A short yes/no-with-conditions answer, and which network/version it applies to.
   2. What can be trained: model families, frameworks, model sizes, LLM fine-tuning via FedLoRA.
   3. Supported FL algorithms and FL types: horizontal (default), vertical, heterogeneity-aware, personalized, transfer. What each one needs changed in the services.
   4. Working today vs. customization vs. planned: a table separating the MNIST reference, "supported by writing your own services", and roadmap items.
   5. Practical limits: hardware requirements per role, participant availability, IPFS bandwidth, synchronous GI rounds.
   6. Bringing your own model and dataset: data stays on clients; what the model owner provides (genesis model, held-out test data, scoring); the five service files; manifest; requirements.txt. Links to services.md and manifest.md.
   7. How results are evaluated: auditor scoring, BlockFLow-based scoring, global model score per GI.
   8. Who can start a training task and what it costs: model owner role, registration fees (open-source vs. proprietary), slasher authorization, staking for validators. Links to model-workflow.md.
   9. Finding participants: clients, auditors, and aggregators. How they discover and join a model. (Open question for the team.)
   10. Privacy and security: DP via the manifest; trust in service code via DAO-approved model registration and manifest updates; links to existing security guidance.
   11. Worked examples: MNIST (tested), plus sketches, e.g. a FedLoRA LLM fine-tune and a tabular regression model. Example-specific steps are clearly labeled as such.
   12. Open questions needing a team decision: listed explicitly, per the expected outcome above.

   Next step is to draft the doc from this outline. Additions and corrections are welcome here.
3. umeradl

   added a parent issue on Sep 25, 2026

   • Testnet readiness: tracking what must be in place before testnet opens #166
4. umeradl

   assigned

   umeradl

   ,

   Santiagocetran

   and

   abrahamnash

   on Sep 25, 2026
5. Santiagocetran commented on Sep 28, 2026

   

   Santiagocetran

   on Sep 28, 2026

   CollaboratorAuthor

   More actions

   I opened a first draft of the new-user guide in draft PR #189. It answers what can be trained on DIN, separates the tested MNIST example from models that need custom work, and gives newcomers a practical path to try their own task. It is linked from Getting Started and the documentation index.

   @umeradl @abrahamnash — could you review the capability claims and DevNet 2.0 scope, especially the tested-versus-possible distinction, participant setup, and privacy wording? I would also appreciate a check that it aligns with the partner-introduction document planned in #158.

   All relative links in Documentation/ resolve (168 checked), and I checked the new page's heading fragments separately.
6. umeradl commented on Oct 7, 2026

   

   umeradl

   on Oct 7, 2026

   Member

   More actions

   Done by PR No. 189, merged into develop as e02db6d with no follow-up commit (outcome). I'm closing this by hand: a merge into develop doesn't trigger closing keywords, because main is the default branch.

   Checked on origin/develop @ e02db6d. The new page is Documentation/public/what-can-be-trained.md ("What can I train on DIN?"), linked from Documentation/README.md and getting-started.md:

   • What's supported today, with a working example: "The short answer" names MNIST/Model_0 as the only end-to-end tested example and has a fit table by kind of idea.
   • What needs customization or more development: the same table, plus the frameworks paragraph. Other frameworks, LLM adapter fine-tuning, personalized FL and vertical FL are labeled as untested possibilities, and asynchronous rounds as needing further development.
   • Steps to bring a model and dataset, including data preparation, sharing and evaluation: "What would I need to bring?" (five items, covering services and manifest), "How does a training round work?", and the data-and-privacy check, which notes that DP is off by default.
   • Who can start a task, finding participants, costs and setup: "What should I check before starting?" (People, Costs) and "How do I try it?". The latter covers the model-owner guide, slasher authorization by the DIN-Representative, and registry approval of models and manifest updates.
   • Network/version is stated, and example-specific instructions are labeled: the version callout (DevNet 2.0 on develop vs. the Model_0 Sepolia guide), and the Model_0-only hardware and format caveats.
   • Open team decisions identified explicitly: the settled-vs-undecided table in the companion page partner-introduction.md (issue No. 158) covers fees, rewards, slashing, distribution and resets. The two pages link to each other.
   • Verification: check_doc_links.py Documentation resolves all 182 links on the merged tree; the PR's CI is green. Abraham signed off on the policy wording on the PR.

   My earlier outline here was longer than what shipped: it had an FL-algorithm catalogue (FedProx, SCAFFOLD, Krum and others) and worked sketches for a FedLoRA LLM and a tabular model. The merged page deliberately keeps to the tested-vs-possible framing. Those deeper sections aren't tracked in an issue yet. Adding support for new kinds of training was out of scope for this issue, as the body says.
7. umeradl

   closed this as completedon Oct 7, 2026
8. umeradl commented on Oct 7, 2026

   

   umeradl

   on Oct 7, 2026

   Member

   More actions

   Follow-up: the outline sections that did not ship (FL-algorithm catalogue, FL setups, worked FedLoRA and tabular sketches) are now tracked in issue No. 240.