Bias Enters at Every Stage
Standard quoted exactly
Analyze how decisions made at different stages of working with data can lead to biased data, misleading conclusions, and compromised AI models.
Example from the standards. In a statistics or math course, students can take a collection of data and submit into an AI system then compare the differences.
Student-friendly learning targets
- I can name specific decisions at collection, cleaning, labeling, splitting, modeling, and deployment that can bias data or conclusions.
- I can show how two different decisions on the same table produce two different stories.
- I can compare a human analysis of a small dataset with an AI summary of the same table and explain the difference.
Essential questions
- At which stage did the slant get in — and how would a later user even see it?
- If two groups clean the same Idaho table differently, why might both outputs look precise?
- What does it mean for a model to be compromised when the math still runs?
Objectives
- Map at least six stages of working with data and attach a bias risk to each.
- Make two deliberate cleaning or inclusion decisions on a public Idaho-relevant table and document how conclusions move.
- Compare a student-made summary of the table with a teacher-run AI summary of the same table (printed), as the standards example suggests.
- Explain misleading conclusions versus compromised models: a wrong story now, and a system that will keep telling it.
- Recommend a documentation practice (who is missing, what was dropped, what the split was) that a later user would need.
Key vocabulary
- Collection bias
- Whose data never entered the table: people offline, counties not surveyed, night-shift rows, languages not offered.
- Cleaning
- Decisions to drop, fix, or impute values. Dropping “messy” rows can drop a population.
- Labeling
- Human or automated assignment of categories; a disputed category becomes a fake certainty downstream.
- Train/dev/test split
- How examples are divided for fitting, tuning, and checking; a leak or an unrepresentative split makes metrics look better than the world.
- Feature choice
- Which columns are treated as meaningful. Proxy variables (ZIP code for income, for example) can smuggle bias.
- Threshold
- The cutoff that turns a score into a yes. Moving it changes who is flagged, hired, or denied.
- Compromised model
- A system whose outputs are systematically untrustworthy because of data or design decisions, even if it produces numbers on demand.
- Data leakage
- Information from the test world sneaking into training, so the model appears accurate until it meets a true new case.
Teacher background
Fluency lessons introduced the idea that data choice can bias output. This standard asks for stages. Bias is not a goblin that inhabits a finished model; it is a series of human decisions: who is counted, what is repaired, how rows are labeled, how the split is cut, which features are blessed, where the threshold sits, and what happens when the system is deployed back into the world that produced the table. The state’s example is practical: in a statistics or math course, students take a collection of data, submit it to an AI system, and compare the differences. Do that with a tiny public table — county indicators, sports statistics, weather, FFA contest scores with names removed — never with a class roster or health file. Run the AI pass yourself on a school account before class and print the summary; live submission is optional and often blocked. Students should first do a human summary so the comparison is real. Teach misleading conclusions (a chart that is true of the cleaned table and false of Idaho) as distinct from a compromised model (a tool that will keep amplifying the same omission). Civic tone: no villain data-scientist stereotype, and no claim that more data automatically heals a bad stage. FERPA: no student PII in the table. Offline fallback: the printed table, two cleaning scripts described in English, and a printed AI summary.
Materials and prep
Materials
- One-page public table (10–20 rows) with a documented source and date. Names of people removed. Idaho-relevant if possible (counties, crops, enrollment aggregates, weather).
- Stage map: Collect, Clean, Label, Split, Choose features, Set threshold, Deploy, Feedback.
- Two cleaning instruction cards that disagree (for example: drop missing rows versus keep them as unknown; include small counties versus drop n < 10).
- Printed teacher-run AI summary of the raw table and, if space, of one cleaned version.
- Offline fallback: no student uploads. If the district forbids even teacher-run tools, a second human summary written by a colleague can stand in for the AI pass; label that substitution honestly.
Before class
- Build or copy a tiny public table. Write the source on the page.
- Run the AI summary yourself; print hallucinations if they happen — they are useful.
- Decide which two cleaning decisions will move a conclusion students can see in twelve minutes.
- Block plan: full lab write-up comparing human, AI, and two cleaning paths, plus a seminar on documentation.
Instructional sequence
Who is not in this table?
5 min- Hand out the table face down. On the back, students guess what an Idaho education or ag table usually misses (home school, tribal schools, small counties, migrant seasons, cash wages).
- Turn it over. Circle missingness: blank cells, tiny n, categories that were never offered.
- Tell them every blank is a decision waiting to happen.
- Read the standard, including the statistics-course example.
Stages, not a single original sin
10 min- Walk the stage map with one running example: predicting chronic absenteeism from an aggregate table. Collection misses some programs. Cleaning drops small rural rows. Labeling treats any absence alike. Split puts two large districts in train and a reservation school in test, or the reverse. Features use ZIP. Threshold flags a school. Deployment sends extra enforcement, which changes the next year’s data.
- Define misleading conclusion versus compromised model.
- Show the printed AI summary. Ask what stage the model cannot see (the dropped rural rows) and how it still sounds sure.
- FERPA line: we use public aggregates. We do not build a table of this class.
Two cleanings, two stories
10 min- As a class, apply Cleaning Card A (drop missing / drop small n). Compute one simple statistic by hand (mean, median, or a count).
- Apply Card B (keep missing as a category). Recompute.
- Write two headlines. Both will look precise. Circle the stage that caused the fork.
- Read the AI summary’s headline. Does it match A, B, or a third invention?
Human summary versus AI summary
12 min- Individually, write an eight-line human summary of the table, naming at least one limitation.
- Compare to the printed AI summary using a three-column chart: Human said / AI said / Table actually shows.
- Mark any AI claim that is not in the table (hallucination) and any omission of small groups (bias at collection or cleaning).
- Finish with one sentence: the conclusion is misleading because… or the model would be compromised if we trained on…
Real-world examples
- A hiring screen trained on past employees of a plant that historically hired from one town.
- An absenteeism model that drops small rural and alternative programs as noise.
- An agricultural yield model that never saw a smoke-heavy fire year as its own regime.
- A sentiment tool labeled by workers who treated dialect as error.
- A split that leaks the target (including the outcome column in features) so accuracy looks perfect until deployment.
Hands-on activity
Stage stickers
8 min- Groups get a failure story and must place it on the stage map with a sticky note.
- Two groups may argue: is this cleaning or labeling? The argument is the point if it stays specific.
- Add a documentation line each group wishes the original team had written.
- Collect human/AI comparison charts.
Discussion questions
- Why does dropping messy rows feel responsible and still produce bias?
- If the AI summary is smoother than yours, should you trust it more? What does smoothness hide?
- How can a model be compromised if the loss function improved?
- Which stage is easiest for a later user to audit, and which is almost invisible?
- What would you refuse to submit to an AI system even in a math class, and why?
Differentiation
Support
- Pre-compute the two statistics; students interpret rather than calculate if needed.
- Provide a fill-in comparison chart.
Challenge
- Design a split that would leak, then a split that would not, on a 16-row toy set.
- Write a model card section: known missing groups, cleaning rules, threshold.
Multilingual learners
- Discuss labeling of language variety as error. Invite examples without requiring students to perform dialect.
- Allow the human summary draft in the student’s strongest language, with key claims also in English.
IEP / 504
- Large-print table. Partner for arithmetic. Oral comparison chart acceptable.
- Reduce to four stages if working memory is the barrier; keep collect, clean, deploy, feedback.
Assessment
Formative
- Headline pair from two cleanings.
- Who is not in this table warmup notes.
Summative
- Human versus AI comparison chart plus a stage-specific explanation of a misleading conclusion or compromised model.
- Block extension: lab write-up with methods, two cleaning paths, and a documentation appendix.
Success criteria
- Identifies more than one stage, not only biased data as a slogan.
- Shows that a decision changed a conclusion, with a number or a count from the table.
- Compares human and AI treatments of the same collection, or honestly labels a substitution.
Responsible use, ethics, and privacy
Responsible use
Public, tiny, de-identified tables only. Students do not upload class surveys with names. Teacher runs any AI pass. If tools are blocked, a second human summary is an honest substitute and should be labeled. No scraping of student information systems.
Ethics
Every stage is a human choice with a distributional effect. Precision can launder those choices. The ethical habit is documentation: who is missing, what was dropped, what the threshold does to people. Students should not treat data workers as villains or as powerless.
Privacy
Do not use this class as a data source. FERPA forbids casual reuse of education records for a model experiment. Aggregate public data is enough. If students later collect a survey, they need a purpose, a consent line, and a plan to destroy identifiers — not an AI bucket.
Reflection
- At which stage would you have made the same decision as Card A, and what group would that have cost?
- What did the AI summary sound like it knew that the table did not show?
- What one line of documentation would you require before trusting a school dashboard?
Homework
On paper, pick a dashboard or infographic you see this week (school, news, sports). Name two stages you cannot see from the graphic and one question you would ask the person who cleaned the table. Do not upload the graphic to a model.
Closing
Hold up the two headlines. Same Idaho table, different human decisions, smoother AI prose on top. Collect charts. Next class returns to a debate you may have met in 9–10, now with evidence instead of vibes: consciousness, creativity, ethics, and human responsibility.
Extensions and cross-curricular links
Go further
- 90-minute block: fuller statistics lab and a seminar on documentation standards.
- Partner with a math teacher to use a course dataset that is already public and de-identified.
- Rebuild the table including the dropped counties and see whether the AI summary (teacher-run later) moves.
- Connect to AITA.DA standards if CS students are in the room: train/dev/test as a formal lab.
- Statistics / algebra
- Means versus medians, missingness, and why a model summary is not a substitute for looking at the table.
- Government
- Indicators used in policy (absenteeism, crime, unemployment) and who is left out of administrative data.
- Agriculture science
- Weather regimes, irrigation districts, and models that never saw a smoke year.
- Computer science
- Splits, leakage, and model cards.