Success in research: Difference between revisions
Make the steps for a successful research project easier to follow and add relevant wiki links |
Show how to plan and combine several studies using an AI and automation example |
||
| Line 22: | Line 22: | ||
Then state what the evidence supports. If you found three floods in the 1960s, give their dates and sources. If the archive is missing several years, say which ones and leave open how often the street flooded. Share the timeline and enough of your method for someone else to check or extend it. This is one way to put [[Open research|open research]] into practice. | Then state what the evidence supports. If you found three floods in the 1960s, give their dates and sources. If the archive is missing several years, say which ones and leave open how often the street flooded. Share the timeline and enough of your method for someone else to check or extend it. This is one way to put [[Open research|open research]] into practice. | ||
== When a big question needs several studies == | |||
How could [[Artificial intelligence|AI]] and automation make goods and services more affordable while limiting surveillance and the concentration of wealth and power? Keep that question at the top of a shared plan. Start with a service you can compare across local governments, such as building permits. | |||
# '''Decide what progress would look like.''' Track fees, wait times, and errors. Check separately for residents who need help applying or who apply offline, since an average can hide who is worse off. Look at which data the agency and vendor can collect or reuse, who can reverse a decision, and whether the agency can switch vendors and keep its records. | |||
# '''Divide the work and agree on what to record.''' One team can compare service records across offices. Another can read public contracts and data policies. A third can interview residents and staff about appeals and how the rules work in practice. Use the same offices, time periods, and definitions. If records are unavailable, ask for summary figures or mark the gap in the shared log. | |||
# '''Check what else might explain a change.''' Measure outcomes before and after adoption. If possible, compare the changes with those in a similar office that kept its previous process. Check whether the offices had similar trends before adoption and record changes in staffing or application volume. If those factors cannot be sorted out, report what changed and how uncertain its cause is. | |||
# '''Bring the results back to the big question.''' Make a table with one row for each arrangement studied. Record who benefited, who gained control over data or decisions, and how strong the evidence is. A shorter wait alongside broader vendor access to residents' data is a mixed result. Look for offices that gained speed under tighter data rules. Where the sources disagree, record what would help resolve the difference. | |||
A permit study can show which arrangements are worth testing elsewhere. To study abundance on a larger scale, repeat the approach where automation might change the cost of essentials or workers' pay. Explain which results appear across studies and which depend on the rules or conditions in one place. | |||
== When working with others helps == | == When working with others helps == | ||
| Line 48: | Line 59: | ||
* [https://www.nationalacademies.org/read/25303/chapter/3 National Academies, Reproducibility and Replicability in Science]. See the summary for ways to check an analysis and repeat a study with new data. | * [https://www.nationalacademies.org/read/25303/chapter/3 National Academies, Reproducibility and Replicability in Science]. See the summary for ways to check an analysis and repeat a study with new data. | ||
* [https://www.gov.uk/government/publications/the-magenta-book/guidance-on-the-impact-evaluation-of-ai-interventions-html HM Treasury, Guidance on the Impact Evaluation of AI Interventions]. See the sections on choosing a comparison and combining different sources of evidence. | |||
== See also == | == See also == | ||
Latest revision as of 22:24, 22 November 2026
Success in research means finding something useful and showing how you found it. You might establish when an event happened, collect observations that others can use, or narrow down which explanations fit the evidence. A small answer with clear evidence can give the next person a place to start.
Suppose you want to know why one street seems to flood more often. Start by finding out which floods you can document. A timeline gives you something to compare with rainfall records or changes to the storm drains.
Choose a question you can answer
A good research question gives you a way to start looking for evidence. For the flooding project, choose one street, a range of years, and a definition of flooding. Do you mean water over the curb, water entering homes, or either? The definition affects which accounts belong in your timeline.
Make three decisions before you start.
- Where to look. Pick sources you can get to, such as digitized newspapers, dated photographs, or city records.
- What to make. Choose something you can finish and share. A table with one row per flood, a date, and a source is enough to get started.
- When to review. Set aside an hour to check how the search is going. If the newspaper archive has too many gaps, shorten the period or try city records. Write down what changed and why.
Keep a record as you go
Use a research log to save search terms, dates, page numbers, links, and notes on what you found. For a flood photo, record where you found it, who supplied the date, and how you recognized the street. A folder of photos and clippings is easier to use when each item has a note that ties it to a claim. This also makes citing your sources easier when you write.
Check the most important claim
Start with the claim your answer depends on. If two newspapers give the same date, see whether one copied the other's account or whether both drew on the same report. Look for a separate piece of evidence, such as a dated photograph or a road closure notice. Ask someone else to trace one or two entries from your timeline back to the original records. A fresh reader may catch a date you copied incorrectly or a location you assumed was obvious.
Then state what the evidence supports. If you found three floods in the 1960s, give their dates and sources. If the archive is missing several years, say which ones and leave open how often the street flooded. Share the timeline and enough of your method for someone else to check or extend it. This is one way to put open research into practice.
When a big question needs several studies
How could AI and automation make goods and services more affordable while limiting surveillance and the concentration of wealth and power? Keep that question at the top of a shared plan. Start with a service you can compare across local governments, such as building permits.
- Decide what progress would look like. Track fees, wait times, and errors. Check separately for residents who need help applying or who apply offline, since an average can hide who is worse off. Look at which data the agency and vendor can collect or reuse, who can reverse a decision, and whether the agency can switch vendors and keep its records.
- Divide the work and agree on what to record. One team can compare service records across offices. Another can read public contracts and data policies. A third can interview residents and staff about appeals and how the rules work in practice. Use the same offices, time periods, and definitions. If records are unavailable, ask for summary figures or mark the gap in the shared log.
- Check what else might explain a change. Measure outcomes before and after adoption. If possible, compare the changes with those in a similar office that kept its previous process. Check whether the offices had similar trends before adoption and record changes in staffing or application volume. If those factors cannot be sorted out, report what changed and how uncertain its cause is.
- Bring the results back to the big question. Make a table with one row for each arrangement studied. Record who benefited, who gained control over data or decisions, and how strong the evidence is. A shorter wait alongside broader vendor access to residents' data is a mixed result. Look for offices that gained speed under tighter data rules. Where the sources disagree, record what would help resolve the difference.
A permit study can show which arrangements are worth testing elsewhere. To study abundance on a larger scale, repeat the approach where automation might change the cost of essentials or workers' pay. Explain which results appear across studies and which depend on the rules or conditions in one place.
When working with others helps
Independent researchers pursue a project outside an institutional research appointment. They may bring years of experience to it. In citizen science, members of the public volunteer to take part in scientific research. Professional researchers do research as part of their paid work. The same project can draw on all three. In a local flood study, one person might trace old street names, others might read newspapers, and someone with mapping experience might check where each flood happened. NOAA's introduction to citizen science
One example from astronomy is Galaxy Zoo. Volunteers looked at telescope images and identified galaxy shapes. Several volunteers examined each image. The team combined their judgments and checked a sample against classifications by professional astronomers. The original Galaxy Zoo study
Try a small comparison
Could a group with little formal training produce a more useful result than a smaller, experienced team? Give both groups the same set of newspaper pages and ask them to find flood reports. Record the time they spend, including training and checking. Ask a third group to score the entries without knowing which team made them. Count well-supported entries, missed reports, and duplicates. If you want to see whether experience or group size accounts for a difference, repeat the task while changing one of those factors at a time.
Discussion questions, essay ideas, and learning related AI prompt ideas
- What would convince you that flooding became more frequent if newspaper coverage also increased?
- Write a short essay about a project you could do independently and one that would benefit from a group. Explain how you would check the most important claim in each project.
- Give an AI tool a draft research question and ask it to suggest records to look for and a way to check the findings. Look up two of the suggested sources yourself and see whether they contain the information the tool expects.
Readings from Wikipedia
Research readings
- National Academies, Reproducibility and Replicability in Science. See the summary for ways to check an analysis and repeat a study with new data.
- HM Treasury, Guidance on the Impact Evaluation of AI Interventions. See the sections on choosing a comparison and combining different sources of evidence.