Maori speech recognition and community-owned data
How a Tribal Radio Station Built Maori Speech Recognition Without Selling Its Voices
This page serves futurism and abundance on this wiki. It looks at how a small Maori radio station in the far north of New Zealand turned a scarce resource, transcribed recordings of an endangered language, into a growing supply of speech tools that its own people control. The story also touches solving problems in living: for a community whose language was suppressed for generations, getting the language back into daily life is a practical problem, not an abstract one.
The central claim of this page is that for small languages, the scarce input for useful speech recognition is community trust rather than computing power, and a community that keeps control of its voice data can build tools that are both good and safe from extraction, even when that means saying no to "open" datasets.
The research question
Speech recognition works well for a handful of large languages and poorly or not at all for most others. A common assumption is that the gap closes when large companies scrape enough audio from the web. Te Hiku Media, a charitable media organization set up by the tribes of the far north more than thirty years ago to revitalize te reo Maori, tested a different path.[1] The question for readers is concrete: can a small community with limited money out-organize the data hunger of global platforms, and what does that teach anyone who wants local AI that serves people instead of harvesting them?
An archive nobody had time to transcribe
Te Hiku began as a radio station keeping te reo on the airwaves. In 2013 it met with community elders to plan for the digital era, and instead of uploading its audio and video to global platforms whose terms require signing over certain rights, it built its own distribution platform, Whare Korero, or "house of speech." That archive holds about 1,000 hours of native speakers, some born in the late 19th century, and around 20 Maori radio stations now upload to it.[2]
The archive was a treasure with a bottleneck. Manual transcription pulled time from a small staff, so in 2016 the station turned to automatic speech recognition.[2] The usual rule of thumb was discouraging. A cofounder of an open source speech company told Wired that building a recognizer from scratch typically needs on the order of 10,000 hours of annotated audio, a figure that is out of reach for most small languages.[3]
Three hundred hours in ten days
Te Hiku started by explaining its purpose to elders and recording them reading phrases at the station. That did not scale, so it ran a crowdsourcing campaign called Korero Maori. In ten days more than 2,500 people signed up and read over 200,000 phrases, producing more than 300 hours of labeled speech.[2][4] With that corpus and Mozilla's open source DeepSpeech, the team built the first speech recognizer for te reo Maori, with a reported word error rate of 14 percent.[5] By mid 2018 the team reported about 10 percent, although Wired noted that the figure had not been externally validated.[3]
The surprising part is not the software. It is that a community campaign, run by people the speakers already trusted, gathered in ten days about 3 percent of the 10,000 hour rule of thumb, and that was enough to build a working recognizer. Later models built with the open source NVIDIA NeMo toolkit, trained on eight GPUs in what the technology lead called a "derelict, rundown" building, are reported to transcribe te reo with 92 percent accuracy and bilingual Maori and English speech with 82 percent accuracy.[2] These models run behind Kaituhi, a transcription service, and support Rongo, an app for practicing pronunciation.[2][1]
Saying no to money and to open datasets
In 2018 the American localization company Lionbridge contacted Maori academics and radio groups offering 45 US dollars an hour to anyone who would record Maori speech. Te Hiku refused and published its reasons, arguing that the people who profit from the language should be Maori themselves.[3][6] More striking for freedom tech readers, Te Hiku also declined a request to add its data to Mozilla's open Common Voice project. Its leaders said that while they value open source, most of their people lack the resources to benefit from fully open data, so opening it would mainly help outsiders who already have those resources.[3]
When OpenAI released Whisper, its paper reported training on 1,381 hours of te reo Maori. Te Hiku's team wrote that the paper did not say where that audio came from and asked who gave anyone the right to build and release a derived model from it.[7][8]
The Kaitiakitanga license as a design tool
Instead of choosing between closed corporate ownership and fully open release, Te Hiku wrote a third option. "Kaitiaki" is close in meaning to guardian or custodian. Under the Kaitiakitanga license for its Papa Reo API, the tools may not be used for surveillance, tracking, discrimination, persecution, building Maori corpora, or mining Maori data. Data a user sends stays under that user's guardianship, is deleted after processing, and must not be passed on to third party services that could collect it.[9] Access requires a legally binding agreement, and the conditions draw on tribal protocols.[1]
This is the core idea worth learning from. Freedom tech often treats openness as the only alternative to platform control. Te Hiku shows that a community can be generous with use (free access for Maori developers, transcription for radio archives) while being strict about extraction. Abundance here means more speakers, more transcribed stories, and more tools in the hands of the people whose language it is, not the largest possible pile of training data for whoever has the most compute.
What other communities can borrow
- Build trust before building models. Elders first, then a public campaign with a clear purpose, produced data that payment alone did not.
- Own the archive. Te Hiku's choice in 2013 to host its own platform is what later made the speech work possible.
- Use open tools without opening the data. DeepSpeech and NeMo are open source; the corpus stayed under community guardianship.
- Write the rules down. A license that names forbidden uses gives partners something concrete to sign.
Honest limits
Most accuracy figures come from Te Hiku itself or from partner publications, including a vendor blog, and early figures were not independently validated.[3] Te Hiku received a grant of 13 million dollars in 2019 that funded additional data scientists and language experts, so its path is not free for a community without similar support.[3] A restrictive license can slow outside researchers who mean well, and this page found no public record of such a license being tested in court against scraping across borders. Some languages lack a living archive or enough speakers to run a campaign like Korero Maori. Finally, data sovereignty protects a community's tools, but it cannot by itself stop large models trained on scraped web audio, which is why Te Hiku has argued for transparency about training data.[7]
Discussion questions, essay ideas, and learning related AI prompt ideas
- Explain why community trust might matter more than computing power when building speech tools for a small language, using the Korero Maori campaign as evidence.
- Compare the Kaitiakitanga license with a standard open source license and describe who gains and who loses under each.
- Argue whether refusing to add data to an open project like Common Voice strengthens or weakens freedom tech as a movement.
- Describe how a local community in your region could collect and govern its own voice or text data for a local AI tool that serves its members.
- Evaluate what a large company would need to change before an Indigenous community could reasonably trust it with language data.
- Analyze whether abundance in AI should be measured by the size of training datasets or by how many people can use and control the resulting tools.
- Design a short research plan to test independently whether a community built speech model outperforms a general purpose model for that community's speakers.
See also
- Local AI as a public library
- Offline captions
- Local-first software and data ownership
- Reproducible builds
- Natural language processing
References
- ↑ 1.0 1.1 1.2 Tahu Kukutai, "How Indigenous communities in New Zealand are protecting their data," Science 384 (6691), April 2024. https://www.waikato.ac.nz/assets/Uploads/Research/Research-institutes-centres-and-groups/Institutes/TKRI/Publications/Science_Kukutai2024_How_Indigenous_Communities_Protecting_Data.pdf
- ↑ 2.0 2.1 2.2 2.3 2.4 Angie Lee, "Maori Speech AI Model Helps Preserve and Promote New Zealand Indigenous Language," NVIDIA Blog, January 16, 2024. https://blogs.nvidia.com/blog/te-hiku-media-maori-speech-ai/
- ↑ 3.0 3.1 3.2 3.3 3.4 3.5 Donavyn Coffey, "Maori are trying to save their language from Big Tech," Wired, April 28, 2021. https://www.wired.com/story/maori-language-tech/
- ↑ Te Hiku Media, "Indigenous ASR: recognising more than speech," NeurIPS 2020 demonstration abstract. https://neurips.cc/virtual/2020/20652
- ↑ "Korero Maori: indigenous language revitalisation powered by machine learning," talk listing, University of Cambridge. https://talks.cam.ac.uk/talk/index/113905/
- ↑ "Free te reo language software developer upset about multinational's forays," 1News, August 27, 2018. https://www.1news.co.nz/2018/08/27/free-te-reo-language-software-developer-upset-about-multinationals-forays/
- ↑ 7.0 7.1 Papa Reo blog, "OpenAI's Whisper is another case study in Colonisation." https://blog.papareo.nz/whisper-is-another-case-study-in-colonisation/
- ↑ Alec Radford and others, "Robust Speech Recognition via Large-Scale Weak Supervision," arXiv:2212.04356, 2022. https://arxiv.org/abs/2212.04356
- ↑ Papa Reo, "Kaitiakitanga License." https://papareo.io/kaitiakitanga