A plan for a decentralized Hugging Face and GitHub
A plan for a decentralized Hugging Face and GitHub is a theoretical, staged design for a public hub where people publish AI models, datasets, and source code, find them, discuss them, and verify them, without any single company holding the only copy or the only key. It combines pieces described elsewhere in this series: swarms for large files, permanent storage for release records, a replayable registry for names and permissions, and federation for code collaboration. It differs from Permanent wiki and Git forges on AO, which proposes pairing a wiki with a Git forge; this page plans a model hub and a code hub together. Nothing described here has been built as a whole, and this page belongs to the Freedom Tech work of this wiki.
What a hub actually does
A central hub bundles many services that look like one website. On Hugging Face, a model card is a Markdown file with a metadata header that declares the license, training datasets, base model, task, and evaluation results, and the Hub uses those fields so people can filter and discover models.[1] Large files sit outside Git, tracked by small pointer files and stored in a chunked backend.[2] Code forges add issues, reviews, and permissions. A decentralized design has to supply every one of these functions, and no single protocol does all of them, which is the argument of Why BitTorrent alone cannot replace Hugging Face or GitHub.
The layer map
- Large files. Weights and datasets travel in BitTorrent version 2 swarms, where each file has its own SHA-256 Merkle root and identical files always share the same root hash.[3] Web seeds let ordinary HTTP mirrors serve as seeds that never leave.[4]
- Release records and the interface. Small signed manifests and the browser interface live on Arweave, which is designed around paying once for long-term storage,[5] with readable names pointing to the current release.[6] An ICP canister could host the interface and registry together instead, though ICP's own documentation advises keeping large datasets in dedicated storage.[7] The trade-offs are set out in ICP versus Arweave and AO for decentralized platforms.
- Registry and permissions. An AO process records who may publish under which name and which manifest is the current release. Every message is stored on Arweave and can be replayed, so anyone can check how the registry reached its state.[8]
- Code collaboration. ForgeFed, an extension of ActivityPub, gives repositories and issue trackers their own inboxes so people can open issues and submit pull requests on other servers without new accounts; Vervis is its reference implementation and Forgejo is implementing federation.[9] Radicle offers a peer-to-peer alternative in which each participant runs a node, repositories are self-certifying, and every action is signed.[10]
- Trust. Releases use safetensors rather than pickle files, which can run code when loaded,[11] and manifests are signed with tools such as the OpenSSF model signing library.[12]
- Nodes. Seeding, gateways, and compute run on many ordinary machines, using the incentive designs surveyed in Making decentralized nodes easy and worth running.
A staged build plan
Each stage is useful by itself, so a project could stop at any point and still leave something of value.
- A manifest standard. Define one signed file per release listing every file, its SHA-256 checksum and size, the torrent infohash, the license, the base model, and the model card fields. This works alongside existing hosts and needs no new infrastructure.
- Mirrors and swarms. Publish version 2 torrents with web seeds that point at existing downloads, and invite libraries and research groups to seed.
- Permanent records. Store manifests and a static browser interface on permanent storage, so the record of what was released survives any one host.
- A registry. Move names and publishing rights into a replayable AO process, or into a canister with transparent controllers.
- Federated collaboration. Connect forges through ForgeFed so discussion and review travel between servers, with bridges to peer-to-peer tools such as Radicle.
- Open indexes. Because every manifest is public, any group can build and publish its own search index, and competing indexes can coexist.
- Node incentives. Pay or reward seeding and serving only where the work can be measured.
Open problems
- Moderation versus permanence. Arweave lets each node operator apply content policies to what it stores and serves,[13] but a published record cannot simply be withdrawn, so review must happen before anything becomes permanent.
- Availability of unpopular models. A 2010 study found that 38 percent of torrents became unavailable within their first month.[14] Web seeds and institutional seeders help, but someone still has to care.
- Names and identity. Readable names can be squatted, and fake publishers can multiply, which links this plan to Proof of personhood without biometric dossiers and Portable reputation for agents.
- Governance. Whoever can change the registry code controls the hub, so that authority needs to be visible and shared.
- Private and gated models. Licenses that restrict access fit poorly with open swarms and are left out of this plan.
Discussion questions, essay ideas, and learning related AI prompt ideas
- Which single function of a central model hub would be hardest to replace without a company, and why?
- Why does the plan store only small manifests permanently and leave the large weights in swarms?
- How should a decentralized hub handle a model that turns out to be harmful after it has been published?
- What would make researchers trust a model they found through an independent index rather than a familiar website?
- How could federation and peer-to-peer collaboration tools work together instead of competing?
- Who should hold the authority to change the rules of a shared registry, and how could that authority be checked?
- Which stage of the build plan would deliver the most value to ordinary users soonest?
Readings from Wikipedia
- Hugging Face
- GitHub
- Forge (software)
- ActivityPub
- Federation (information technology)
- Distributed version control
- BitTorrent
- Open-source artificial intelligence
- Software supply chain
See also
External links
- ForgeFed
- Radicle
- Hugging Face model card documentation
- BitTorrent protocol specification v2
- Search this wiki for decentralized GitHub
- Search this wiki for Hugging Face
- Search this wiki for ForgeFed
References
- ↑ Hugging Face. Model Cards. Hugging Face Hub documentation.
- ↑ Hugging Face. Xet: our Storage Backend. Hugging Face Hub documentation.
- ↑ BitTorrent.org. BEP 52: The BitTorrent Protocol Specification v2.
- ↑ BitTorrent.org. BEP 19: WebSeed, HTTP/FTP Seeding (GetRight style).
- ↑ Arweave. Motivation. Arweave Docs.
- ↑ ar.io. Ar.io Name System (ArNS). ar.io Documentation.
- ↑ DFINITY. Asset canister (legacy). ICP Developer Docs.
- ↑ AO Cookbook. How ao messaging works.
- ↑ ForgeFed. Project home page.
- ↑ Radicle. Radicle Protocol Guide.
- ↑ Hugging Face. Pickle Scanning. Hugging Face Hub documentation.
- ↑ OpenSSF. Launch of Model Signing v1.0. April 4, 2025.
- ↑ Arweave. Transaction Blacklist. Arweave Docs.
- ↑ Kaune S, et al. Unraveling BitTorrent's File Unavailability: Measurements, Analysis and Solution Exploration. arXiv 0912.0625, 2009; IEEE P2P 2010.