Jump to content

Success in research: Difference between revisions

From IdeaWazaWiki
Lumo (talk | contribs)
m Lumo moved page Success in research to Comparing amateur and professional research: Use a title that reflects the article question
Lumo (talk | contribs)
Explain what helps research succeed across independent, citizen science, and professional projects
Line 1: Line 1:
A large group of amateur researchers can make observations across many places, sort through more material than a small team can handle, and bring experience from outside a field. Professional researchers may bring specialized methods, equipment, or experience interpreting evidence. How could we test whether a large group of people with little formal training can produce more useful or original findings than a smaller team of trained researchers?
Success in [[Research|research]] starts with asking a question that evidence can answer. A useful result tells readers what was found, how it was checked, and where uncertainty remains. A project can succeed by documenting a little-known event, producing a reliable dataset, or discovering that a widely held explanation fits the evidence poorly.


Amateur and professional describe how people participate in research. An amateur astronomer may have decades of observing experience, and a paid researcher may be new to astronomy. A comparison should record relevant experience, formal education, and training for the task. It should also say how the teams used their time and resources.
== Who does research? ==


== What counts as a successful result? ==
An '''independent researcher''' pursues a project outside an institutional research appointment. They may work alone or with collaborators and may have years of training or practical experience. Someone who publishes local history videos, for example, may know the area's maps and archives well enough to notice that a frequently repeated story gives the wrong date for a flood. Sharing the relevant records lets a viewer check the claim.


Choose a measure that fits the question. A team building a public dataset might be judged by how much it covers, how accurate its records are, and whether someone else can use them. A team looking for new patterns might be judged by how many of its leads hold up under further investigation. A surprising observation can be easy to generate and expensive to check, so count the time spent checking rejected leads as well.
In '''citizen science''', members of the public take part voluntarily in scientific research. They may help choose the question, gather observations, interpret results, or lead a project. '''Professional researchers''' do research as part of their paid work. These roles can overlap. An independent researcher can contribute to a citizen science project, and a professional in one field can volunteer in another. [https://www.noaa.gov/office-education/citizen-science NOAA's introduction to citizen science]


A finding that changes what a field teaches or how people solve a problem may take years to assess. A short experiment can track whether a finding is new to the published literature and whether an independent team can confirm it. It can also record whether another group uses the result later.
== What helps research succeed? ==


== What Galaxy Zoo compared ==
* '''Ask a question you can answer.''' If you want to date a flood, decide whether you mean when water first entered the streets or when it reached its highest point. Look for records that can distinguish the two. In a larger project, make sure everyone gathering observations uses the same definitions.
* '''Leave a record of the evidence and decisions.''' Record where each observation came from and how you handled conflicting accounts. If you use a spreadsheet or code, keep the original data and explain the steps that produced your results. When working with others, agree on who will check the records and how contributors will be credited.
* '''Check the result independently.''' Ask someone to trace a sample of claims back to the evidence. They may find that two newspaper stories drew on the same report, so both repeat the same date. Compare the accounts with a dated photograph or another source that could confirm or challenge them. Explain which parts remain uncertain.


In the original Galaxy Zoo project, volunteers classified images of galaxies from the Sloan Digital Sky Survey. Multiple participants looked at each image. The project combined their classifications and compared the results with classifications made by professional astronomers for subsets of the galaxies. The researchers found that the results were consistent for the simple galaxy shapes that the project tracked. The study gives an example of volunteers and professionals contributing to different parts of the same research project. [https://arxiv.org/abs/0804.4483 Lintott and colleagues, Galaxy Zoo]
== What Galaxy Zoo shows ==


Agreement among volunteers is useful for finding images that need another look. It can also conceal a shared mistake. If most participants interpret a blurry spiral galaxy as an elliptical galaxy, a majority vote may preserve the mistake. Comparing classifications with an independent reference and examining disagreements can show which images or instructions are causing trouble.
In the original Galaxy Zoo project, members of the public classified images from the Sloan Digital Sky Survey. The team combined multiple judgments and checked its results against classifications by professional astronomers for subsets of the galaxies. The two sets of classifications were broadly consistent for the simple galaxy shapes the project studied. Together, the contributors produced a catalog that other researchers could use. [https://arxiv.org/abs/0804.4483 Lintott and colleagues, Galaxy Zoo]


== How to test the idea ==
A vote can help identify images that need another look. Reviewers should also check some of the images on which almost everyone agrees. If most people read the same blurry spiral galaxy as elliptical, high agreement can hide a shared error.


# '''Give teams the same research task.''' For example, ask volunteer, professional, and mixed teams to mark visible features in previously unseen images. Choose the images in advance, including clear and ambiguous cases. Give teams access to the same examples, tools, and task instructions.
== How could we compare different approaches? ==
# '''Decide what the comparison is about.''' For a comparison of experience, give teams the same total number of work hours. To test whether a crowd can do more with a given budget, let teams vary in size and include training, coordination, and verification in their costs. Record both spending and participant time, including unpaid time.
# '''Set the measures before seeing the results.''' Have reviewers assess the markings without knowing which team made them. Use a reference set prepared independently and record how many features each team finds, misses, or marks by mistake. Allow reviewers to flag ambiguous images and report their uncertainty.
# '''Repeat the comparison.''' Try new images and recruit new teams. A result that holds up on several sets of images may still depend on the type of work. Try a second task, such as proposing an explanation for a pattern, and see whether the same arrangement works there.


For a classroom version, divide into small groups and pick a set of public images. Have each group label the same images independently. Compare individual labels, the group vote, and a reference prepared ahead of time. Then look at the cases where voting helped and the cases where asking for another opinion was more valuable.
When can a large group with little formal training produce more useful or original results than a small team of specialists? Start with a task you can measure. Ask volunteer, professional, and mixed teams to mark features in the same set of public images. Give them the same tools, instructions, and practice examples. Record relevant experience alongside formal education.
 
Decide what counts as success before seeing the results. Count the features each team finds correctly, misses, or marks by mistake. Have reviewers check a sample against an independently prepared reference without knowing which team produced it. If the project is looking for novel findings, track which leads survive follow-up. It can take years to tell whether a result changes a field.
 
Give teams equal work hours if you want to compare how they perform with the same amount of time. If you want to compare what each approach achieves within a budget, let team sizes vary and record training, coordination, and checking costs, along with unpaid hours. Repeat the study with different teams and images to see how far the result holds.
 
For a classroom version, have students label a set of public images independently. Compare individual labels and the group vote with reference labels prepared ahead of time. Find one case where voting helped and one where the group would benefit from reviewing the image together.


==Discussion questions, essay ideas, and learning related AI prompt ideas==
==Discussion questions, essay ideas, and learning related AI prompt ideas==


* What kinds of research tasks benefit most from having more observers? What kinds depend on a person learning a difficult method over time?
* What could make an independently produced local history video more useful to a researcher than a published summary of the same event?
* Write an essay on how you would compare a volunteer team with a professional team when one finishes faster and the other finds more errors. Explain which result matters most to the people who will use the work.
* Write an essay about how you would divide the work of studying a local stream among residents, independent researchers, and a laboratory team. Explain who would choose the question and how each group would check the others' findings.
* Ask an AI tool to propose a study comparing volunteer, professional, and mixed teams on a public image dataset. Have it specify the outcome and the costs for each team. Exchange designs with a classmate and see whether both of you could score the same sample result using the instructions.
* Ask an AI tool to design a fair comparison of volunteer and professional teams on a public dataset. Have it specify the outcome and what to do with ambiguous cases. Exchange plans and see whether two readers can score the same example consistently.


==Readings from Wikipedia==
==Readings from Wikipedia==
Line 38: Line 43:
== Research readings ==
== Research readings ==


* [https://www.nationalacademies.org/read/25303/chapter/3 National Academies, Reproducibility and Replicability in Science]. The summary explains what can be learned by checking the original analysis and by repeating a study with new data.
* [https://www.nationalacademies.org/read/25303/chapter/3 National Academies, Reproducibility and Replicability in Science]. The summary explains how researchers check an original analysis and repeat a study with new data.


== See also ==
== See also ==

Revision as of 22:07, 22 November 2026

Success in research starts with asking a question that evidence can answer. A useful result tells readers what was found, how it was checked, and where uncertainty remains. A project can succeed by documenting a little-known event, producing a reliable dataset, or discovering that a widely held explanation fits the evidence poorly.

Who does research?

An independent researcher pursues a project outside an institutional research appointment. They may work alone or with collaborators and may have years of training or practical experience. Someone who publishes local history videos, for example, may know the area's maps and archives well enough to notice that a frequently repeated story gives the wrong date for a flood. Sharing the relevant records lets a viewer check the claim.

In citizen science, members of the public take part voluntarily in scientific research. They may help choose the question, gather observations, interpret results, or lead a project. Professional researchers do research as part of their paid work. These roles can overlap. An independent researcher can contribute to a citizen science project, and a professional in one field can volunteer in another. NOAA's introduction to citizen science

What helps research succeed?

  • Ask a question you can answer. If you want to date a flood, decide whether you mean when water first entered the streets or when it reached its highest point. Look for records that can distinguish the two. In a larger project, make sure everyone gathering observations uses the same definitions.
  • Leave a record of the evidence and decisions. Record where each observation came from and how you handled conflicting accounts. If you use a spreadsheet or code, keep the original data and explain the steps that produced your results. When working with others, agree on who will check the records and how contributors will be credited.
  • Check the result independently. Ask someone to trace a sample of claims back to the evidence. They may find that two newspaper stories drew on the same report, so both repeat the same date. Compare the accounts with a dated photograph or another source that could confirm or challenge them. Explain which parts remain uncertain.

What Galaxy Zoo shows

In the original Galaxy Zoo project, members of the public classified images from the Sloan Digital Sky Survey. The team combined multiple judgments and checked its results against classifications by professional astronomers for subsets of the galaxies. The two sets of classifications were broadly consistent for the simple galaxy shapes the project studied. Together, the contributors produced a catalog that other researchers could use. Lintott and colleagues, Galaxy Zoo

A vote can help identify images that need another look. Reviewers should also check some of the images on which almost everyone agrees. If most people read the same blurry spiral galaxy as elliptical, high agreement can hide a shared error.

How could we compare different approaches?

When can a large group with little formal training produce more useful or original results than a small team of specialists? Start with a task you can measure. Ask volunteer, professional, and mixed teams to mark features in the same set of public images. Give them the same tools, instructions, and practice examples. Record relevant experience alongside formal education.

Decide what counts as success before seeing the results. Count the features each team finds correctly, misses, or marks by mistake. Have reviewers check a sample against an independently prepared reference without knowing which team produced it. If the project is looking for novel findings, track which leads survive follow-up. It can take years to tell whether a result changes a field.

Give teams equal work hours if you want to compare how they perform with the same amount of time. If you want to compare what each approach achieves within a budget, let team sizes vary and record training, coordination, and checking costs, along with unpaid hours. Repeat the study with different teams and images to see how far the result holds.

For a classroom version, have students label a set of public images independently. Compare individual labels and the group vote with reference labels prepared ahead of time. Find one case where voting helped and one where the group would benefit from reviewing the image together.

  • What could make an independently produced local history video more useful to a researcher than a published summary of the same event?
  • Write an essay about how you would divide the work of studying a local stream among residents, independent researchers, and a laboratory team. Explain who would choose the question and how each group would check the others' findings.
  • Ask an AI tool to design a fair comparison of volunteer and professional teams on a public dataset. Have it specify the outcome and what to do with ambiguous cases. Exchange plans and see whether two readers can score the same example consistently.

Readings from Wikipedia

Research readings

See also