← Back to course
4/6
Week 06 Β· Cleaning Data

Thursday

Filter and transform
// Fixing messy data without changing what it means
⏱ about 20 min

Thursday: Filter and Transform

With Q3 clean, Wren has a new question. "The paper forms started on day 3. Do the paper-form members visit as often as everyone else?"

Comet groans. "Do we have to read all 40 rows again?"

"What do you notice about the Q5 column?" Wren asks.

Comet looks. "It already says screen or paper. We could keep just the paper rows."

Keeper Ines leans over the desk. "And the council thinks in days, not weeks. Could you show visits per day?"

Nova hovers over the sheet. "Would you like a hint?" she asks. "Keeping some rows and changing every number are two different jobs. Do them one at a time."

Four ways to get information from data

Programs can process data to get information. Here are four common processes.

Transform: change every item the same way, such as doubling every number in a list.

Filter: keep only some items, such as only the positive numbers.

Combine or compare: for example, add up a list of numbers.

Visualize: show the data in a chart, graph or other picture.

Filter: only the paper forms

Wren filters the sheet. He keeps only the rows where Q5 says paper form. Nine rows are left.

IDQ2 WhenQ4 Visits last weekVisits per day (Q4 divided by 7)
M15morning71
M16morning142
M21morning71
M22morning00
M23afternoon71
M30morning213
M35afternoon71
M36weekend142
M37weekend71

This is made-up data from the Commons Count story.

The last column is a transform. Every Q4 number was divided by 7, the days in a week.

FILTER, TRANSFORM, COMBINE
  • Read the question.
  • Tap your answer.
How many rows are left after keeping only the paper forms?
Filter again: keep only the morning rows. How many are left?
Add the Q4 column: 7, 14, 7, 0, 7, 21, 7, 14 and 7. What is the total?
What is 84 divided by 7?
Which process is it?True or false?
Keeping only the paper-form rows is a filter.?
Dividing every Q4 number by 7 is a transform.?
Adding up the Q4 column is a way of combining data.?
Drawing a bar graph of the totals is a filter.?
WHY THIS EXERCISENaming the process helps you plan each step and explain it to others.
Keeping only the rows that match a rule is called a ____. Type one word.
WHY THIS EXERCISEFiltering is one of the most common ways to find patterns in data.

Tools and size

What you can do with data depends on your skills and your tools. The size of a data set affects how much information you can get from it.

Search tools find information fast. Filters help find patterns. Spreadsheets help organize data and find trends.

Nine rows are easy by hand. Thousands of rows would need a program.

If you have a spreadsheet (optional)
Type the nine rows into a blank sheet and use its filter to keep only the morning rows.
You can do the same task on paper, as you just did.

Sharp thinking. Tomorrow is Impact Friday: when cleaning could erase a voice.

← Wednesday