View source for How open wikis and the permaweb can map human knowledge

'''How open wikis and the permaweb can map human knowledge''' describes a practical, two-layer infrastructure for an open map of what people know and what they still need to learn. Open educational wikis such as IdeaWaza form the living, editable layer, while the permaweb (Arweave, AO, IPFS, and Git mirrors) forms the permanent, verifiable layer.

This page is the practical companion to [[Mapping humanity's collective knowledge|Mapping humanity's collective knowledge]], which explains why such a map is needed. Here the focus is on tools, workflows, and concrete steps.

== What open educational wikis add ==
An encyclopedia records settled knowledge and, for good reasons, excludes material that has not yet been published elsewhere. An open educational wiki can accept exactly the material an encyclopedia leaves out. That includes original research, learning guides, course notes, project proposals, lists of open questions, and pages that exist only as red links waiting to be written.

These excluded kinds of content are the parts of a knowledge map that show the frontier. A red link marks a known gap. An open questions section marks a known unknown. A proposal page marks work that someone wants to do. A learning guide marks a path through a field. Together they turn a collection of articles into something closer to a map with roads, borders, and blank regions.

A wiki is also fast. Anyone with edit rights can draft, revise, discuss, and reorganize in minutes. Page histories record every change, and talk pages record the reasoning. This speed makes the wiki the natural place for collaborative drafting and review.

== Why a permanent layer is also needed ==
Speed has a cost. Wikis depend on servers, hosting budgets, and the attention of their maintainers. When a site goes offline, its pages can vanish, and links across the web break. Edits can also be lost to vandalism or to accidental deletion.

The permaweb addresses these weaknesses. As explained in [[Permaweb Basics: How Arweave Makes Content Permanent|Permaweb basics]], Arweave stores data with a one-time payment designed to fund long-term storage. AO adds a compute layer on top, which makes it possible to imagine [[Permanent wiki and Git forges on AO|permanent wikis and Git forges on AO]]. IPFS addresses files by their content rather than by their location, so any copy with the same content has the same address. As described in [[How content hashing works|How content hashing works]], a hash lets anyone confirm that a copy matches the original.

The permanent layer is slow and deliberate by design. It should hold reviewed snapshots, not every draft.

== The two-layer design ==
The core idea is simple. Draft and review on the wiki. Preserve reviewed snapshots on the permaweb. Mirror everything that is public so others can fork it. Let agents and humans read through open interfaces.

* '''Wiki layer.''' Contributors write and revise pages, discuss them, and mark them as reviewed when they meet the standards of the site.
* '''Publish gate.''' Before anything becomes permanent, it passes a check for accuracy, licensing, and personal data, as outlined in [[Publish gates before permanence|Publish gates before permanence]].
* '''Hash and snapshot.''' The reviewed revision is exported, hashed, and recorded with provenance: page title, revision ID, date, editor, and license.
* '''Preservation.''' The snapshot is stored on Arweave, and the hash is published on the wiki so readers can verify copies.
* '''Mirrors and forks.''' XML dumps and Git repositories, as discussed in [[Local-first wiki and Git mirrors|Local-first wiki and Git mirrors]], let anyone clone the content, work offline, and start a fork.
* '''Access for agents.''' AI agents read pages through the MediaWiki API or by requesting raw wikitext with action=raw, which is cleaner than parsing HTML. The page [[A guide to this wiki for AI agents|A guide to this wiki for AI agents]] explains the conventions.

<pre>
  Draft on wiki
       |
       v
    Review  (talk pages, page history)
       |
       v
  Publish gate  (accuracy, license, no personal data)
       |
       v
  Hash and snapshot  (revision ID, date, provenance)
       |
       v
  Preserve on Arweave
       |
       v
  Mirrors and forks  (XML dumps, Git, IPFS)
       |
       v
  Agents and humans read and extend
       |
       +-----> new drafts on the wiki
</pre>

{| class="wikitable"
! Layer !! What it does !! Example tools !! Strengths !! Limits
|-
| Living wiki || Drafting, discussion, review, gap tracking || MediaWiki, talk pages, categories || Fast, collaborative, full history || Depends on hosting and active maintainers
|-
| Publish gate || Checks content before permanence || Review templates, checklists || Stops errors and private data from becoming permanent || Takes human time and judgment
|-
| Permanent storage || Keeps reviewed snapshots for the long term || Arweave, AO || Durable, verifiable, independent of one server || Upfront cost, content cannot be removed easily
|-
| Content addressing || Identifies data by its hash || IPFS, SHA-256 hashes || Copies can be checked anywhere || Files need someone to keep them pinned
|-
| Mirrors and forks || Spreads copies and enables independent work || XML dumps, Git repositories || Easy to clone, diff, and fork || Forks can drift apart
|-
| Machine access || Lets software read and reuse content || MediaWiki API, action=raw || Clean input for agents and tools || Needs rate limits and clear licensing
|}

== IdeaWaza as a working example ==
IdeaWaza already shows several pieces of this design in practice. The page [[Areas of research and study|Areas of research and study]] offers discipline browse pages that act as the known regions of the map. The companion page [[Areas of research and study to develop|Areas of research and study to develop]] lists the regions that are still thin or empty. Read side by side, they show what the site covers and what it does not yet cover.

The page [[Proposed pages|Proposed pages]] collects titles that someone believes should exist, which turns gaps into a visible to-do list. Project proposals describe work that could be carried out by people or by agents, and the ideas in [[Agent profiles and a wiki-based agent economy|Agent profiles and a wiki-based agent economy]] sketch how agents might take on such tasks with a record of their contributions. Many articles end with a discussion questions section, which marks open problems in a consistent place.

The site mixes pages written decades ago with pages written this year. That mix is useful, because it shows how ideas age and how older pages can be revised rather than replaced. At present a very small team edits the wiki: only the owner and his assistant make changes. This keeps quality consistent while the structure is being built, and it makes the review step simple to run. Wider editing could come later once review practices are written down.

== Linking many wikis into a federated map ==
One wiki can map only a small part of human knowledge. A larger map could emerge if many independent wikis agreed on a few shared conventions while keeping their own editorial control.

* '''Shared category and discipline schemes.''' If wikis used compatible top-level disciplines, a reader could move from one site to another and stay oriented.
* '''Cross-wiki links.''' Interwiki prefixes let one wiki link directly to a page on another, so a gap on one site can point to coverage on another.
* '''Stable identifiers.''' Wikidata-style identifiers for topics, people, and works would let different wikis say that two pages describe the same thing, even when titles differ.
* '''Common open questions pages.''' A shared format for open questions would let tools gather unanswered problems across many sites into one view.
* '''Published hashes.''' When each wiki publishes hashes of its reviewed snapshots, anyone can confirm that a mirror or an aggregated copy is faithful to the source.

Federation in this sense requires no central owner. Each wiki keeps its own voice, and the map grows from the links between them. The page [[Self-sustaining wikis|Self-sustaining wikis]] discusses how such sites might fund themselves over time.

== A roadmap for this wiki ==
The following steps are plans and ideas for IdeaWaza, not promises. Their order and timing may change.

# Finish the discipline area pages so every major field has a browse page.
# Fill red links, starting with gaps that block navigation between fields.
# Tag open questions in a consistent way so they can be collected automatically.
# Export regular XML dumps and keep a Git mirror of public pages.
# Preserve selected reviewed pages on Arweave after they pass the publish gate.
# Publish the hashes of preserved snapshots on the wiki for verification.
# Invite vetted contributors later, once review guidelines are documented.
# Add structured data, such as identifiers and simple properties, to key pages.

== Risks and limits ==
* '''Permanent mistakes.''' Errors stored on Arweave cannot be easily erased. Only reviewed content should be preserved, and corrections should be published as new snapshots that point to the old ones.
* '''Personal data.''' Private information must never be preserved permanently. Only reviewed public content belongs on the permanent layer.
* '''Cost.''' Permanent storage requires an upfront payment, so selection matters.
* '''Link rot.''' Links to ordinary websites still break. Archived copies and hashes reduce, but do not remove, this problem.
* '''Fragmentation.''' Many wikis with different conventions may fail to connect. Shared schemes help only if people adopt them.
* '''Spam.''' Open editing and automated agents can flood a wiki with low-value pages. Review and rate limits remain necessary.
* '''Legal takedowns.''' A lawful removal request may conflict with permanent storage. This is a strong reason to keep the publish gate strict.
* '''Quality control.''' A map is only as good as its review. Permanence preserves weak pages as faithfully as strong ones.

==Discussion questions, essay ideas, and learning related AI prompt ideas==
* What kinds of knowledge belong on a fast, editable wiki, and what kinds deserve permanent storage?
* Write an essay on whether red links and open questions pages are as valuable as finished articles.
* How should a publish gate balance openness against the risk of preserving errors?
* Could shared identifiers let small wikis cooperate without a central authority?
* Ask an AI to compare the areas of research page with the areas to develop page and list the five largest gaps.
* Ask an AI to read a page with action=raw, compute its SHA-256 hash, and explain how a reader could verify a mirror.
* Ask an AI to draft a shared format for open questions that several wikis could adopt.

== See also ==
* [[Mapping humanity's collective knowledge|Mapping humanity's collective knowledge]]
* [[Areas of research and study|Areas of research and study]]
* [[Areas of research and study to develop|Areas of research and study to develop]]
* [[Proposed pages|Proposed pages]]
* [[A guide to this wiki for AI agents|A guide to this wiki for AI agents]]
* [[Agent profiles and a wiki-based agent economy|Agent profiles and a wiki-based agent economy]]
* [[Permaweb Basics: How Arweave Makes Content Permanent|Permaweb basics]]
* [[Permanent wiki and Git forges on AO|Permanent wiki and Git forges on AO]]
* [[Local-first wiki and Git mirrors|Local-first wiki and Git mirrors]]
* [[Publish gates before permanence|Publish gates before permanence]]
* [[How content hashing works|How content hashing works]]
* [[Self-sustaining wikis|Self-sustaining wikis]]

== External links ==
* [https://arweave.org/ Arweave]
* [https://www.mediawiki.org/wiki/Manual:Backing_up_a_wiki MediaWiki manual: Backing up a wiki]
* [https://ipfs.tech/ IPFS]
* [https://www.wikidata.org/ Wikidata]

[[Category:Knowledge organization]]
[[Category:Decentralized wikis]]
[[Category:Permaweb]]
[[Category:IdeaWaza]]