Success in research: Difference between revisions
wikademia>Eme m Created page with 'This page will explore what contributes to success in research. Can lots of amateurs, or those without lots of formal education collectively produce research results that are mor…' |
Expand the research comparison with a Galaxy Zoo example and a test design |
||
| Line 1: | Line 1: | ||
A large group of amateur researchers can make observations across many places, sort through more material than a small team can handle, and bring experience from outside a field. Professional researchers may bring specialized methods, equipment, or experience interpreting evidence. How could we test whether a large group of people with little formal training can produce more useful or original findings than a smaller team of trained researchers? | |||
How can | Amateur and professional describe how people participate in research. An amateur astronomer may have decades of observing experience, and a paid researcher may be new to astronomy. A comparison should record relevant experience, formal education, and training for the task. It should also say how the teams used their time and resources. | ||
== What counts as a successful result? == | |||
Choose a measure that fits the question. A team building a public dataset might be judged by how much it covers, how accurate its records are, and whether someone else can use them. A team looking for new patterns might be judged by how many of its leads hold up under further investigation. A surprising observation can be easy to generate and expensive to check, so count the time spent checking rejected leads as well. | |||
A finding that changes what a field teaches or how people solve a problem may take years to assess. A short experiment can track whether a finding is new to the published literature and whether an independent team can confirm it. It can also record whether another group uses the result later. | |||
== What Galaxy Zoo compared == | |||
In the original Galaxy Zoo project, volunteers classified images of galaxies from the Sloan Digital Sky Survey. Multiple participants looked at each image. The project combined their classifications and compared the results with classifications made by professional astronomers for subsets of the galaxies. The researchers found that the results were consistent for the simple galaxy shapes that the project tracked. The study gives an example of volunteers and professionals contributing to different parts of the same research project. [https://arxiv.org/abs/0804.4483 Lintott and colleagues, Galaxy Zoo] | |||
Agreement among volunteers is useful for finding images that need another look. It can also conceal a shared mistake. If most participants interpret a blurry spiral galaxy as an elliptical galaxy, a majority vote may preserve the mistake. Comparing classifications with an independent reference and examining disagreements can show which images or instructions are causing trouble. | |||
== How to test the idea == | |||
# '''Give teams the same research task.''' For example, ask volunteer, professional, and mixed teams to mark visible features in previously unseen images. Choose the images in advance, including clear and ambiguous cases. Give teams access to the same examples, tools, and task instructions. | |||
# '''Decide what the comparison is about.''' For a comparison of experience, give teams the same total number of work hours. To test whether a crowd can do more with a given budget, let teams vary in size and include training, coordination, and verification in their costs. Record both spending and participant time, including unpaid time. | |||
# '''Set the measures before seeing the results.''' Have reviewers assess the markings without knowing which team made them. Use a reference set prepared independently and record how many features each team finds, misses, or marks by mistake. Allow reviewers to flag ambiguous images and report their uncertainty. | |||
# '''Repeat the comparison.''' Try new images and recruit new teams. A result that holds up on several sets of images may still depend on the type of work. Try a second task, such as proposing an explanation for a pattern, and see whether the same arrangement works there. | |||
For a classroom version, divide into small groups and pick a set of public images. Have each group label the same images independently. Compare individual labels, the group vote, and a reference prepared ahead of time. Then look at the cases where voting helped and the cases where asking for another opinion was more valuable. | |||
==Discussion questions, essay ideas, and learning related AI prompt ideas== | |||
* What kinds of research tasks benefit most from having more observers? What kinds depend on a person learning a difficult method over time? | |||
* Write an essay on how you would compare a volunteer team with a professional team when one finishes faster and the other finds more errors. Explain which result matters most to the people who will use the work. | |||
* Ask an AI tool to propose a study comparing volunteer, professional, and mixed teams on a public image dataset. Have it specify the outcome and the costs for each team. Exchange designs with a classmate and see whether both of you could score the same sample result using the instructions. | |||
==Readings from Wikipedia== | |||
* [[Wikipedia:Citizen science|Citizen science]] | |||
* [[Wikipedia:Galaxy Zoo|Galaxy Zoo]] | |||
* [[Wikipedia:Reproducibility|Reproducibility]] | |||
== Research readings == | |||
* [https://www.nationalacademies.org/read/25303/chapter/3 National Academies, Reproducibility and Replicability in Science]. The summary explains what can be learned by checking the original analysis and by repeating a study with new data. | |||
== See also == | |||
* [[Research]] | * [[Research]] | ||
* [[ | * [[Collaborative research]] | ||
* [[Experimentation]] | |||
[[Category:Research]] | [[Category:Research]] | ||