← Back to course
Data and Society 9-12 / Week 06 / Tuesday
2/6
Week 06 · Cleaning Data

Tuesday

Cleaning rules
// Fixing messy data without changing what it means
⏱ about 20 min

Tuesday: Cleaning Rules

Comet has a quick fix. "Let's just change every weird answer to longer evening hours. It is the most popular anyway."

Wren holds up the blank row. "Even this one? M18 did not choose anything."

"It would make the numbers tidy," Comet says.

"Tidy, but not true," Wren says gently. "What do you notice about what that would do to the totals?"

Comet counts on her fingers. "Longer evening hours would get answers nobody gave it."

Nova hovers over the sheet. "Would you like a hint?" she asks. "Write rules before you change a single answer. Then everyone cleans the same way."

Comet picks up her pencil. "Rules first. Got it."

Clean without changing meaning

Cleaning data makes it uniform without changing its meaning.

For example, every spelling and capitalization of the same answer is replaced with one word.

Blank and invalid answers are challenges too. The crew decides what to do with them and writes it down.

RuleWhat the crew does
1. Capitals and marksWrite every answer in small letters and drop extra marks, like !
2. Clear misspellingsFix a spelling only when the meaning is clear, like repiar to repair
3. Same meaningReplace every answer that means the same thing with one category name
4. Unsure answersDecide as a crew, then write the choice and the reason in the cleaning log
5. Blank answersMark them blank. Never fill in a guess
6. Invalid answersMark them invalid. Keep the row and note why in the log

These are the crew's rules in the Commons Count story.

APPLY THE RULES
  • Read the question.
  • Tap your answer.
M31 wrote "repiar bench." Which rule applies?
M10 wrote "open later." The crew agrees it means longer evening hours. What else must they do?
M18 left Q3 blank. What does the crew write?
M19 wrote "a launch pad." What does the crew do?
StatementTrue or false?
Writing "Bike rack" as "bike rack" changes its meaning.?
Filling a blank with the most popular answer is good cleaning.?
Cleaning should make data uniform without changing meaning.?
A cleaning log lets other people check every choice.?
WHY THIS EXERCISEHonest cleaning keeps every member's answer meaning what they meant.
CLEAN ONE RAW ANSWER
  • ?Read the raw answer.
  • ?Match it to a category, or mark it blank or invalid.
  • ?Fix a clear misspelling.
  • ?If you had to decide, write the choice in the log.
  • ?Write it in small letters and drop extra marks.
WHY THIS EXERCISEThe same steps in the same order mean every cleaner gets the same result.
A blank answer is an example of ____ data.
An answer that cannot be a real answer to the question is ____.
Try it
Write three spellings of the same word you might see in an open box.
Then write the one clean word you would replace them with.

Nice work. Tomorrow in Count Lab you will clean all 40 answers and check the totals.

← Monday