Measurement bias

Conquering Data Bias

Konstantinos Kattidis

Data Analytics Lead

Understanding measurement bias

It occurs when the measurement procedure introduces distortions or misleading outcomes

Common bias types include:

  • Instrument bias
  • Observer bias
  • Recall bias
  • Social desirability bias

 

Person measuring a bar chart

Conquering Data Bias

Instrument bias

It occurs when the tools used to measure variables, such as surveys or analytics software, introduce inaccuracies

  • For example:
    • An analytics software failing to accurately capture certain user behaviors or attributes
    • Poorly designed survey questions causing biased respondents answers

Peaople conducting a survey and measuring

Conquering Data Bias

Observer bias

It's the systematic difference between what is observed due to variation in observers, and the true value

  • It arises when data collectors bring their own biases or expectations into the analysis

  • For example:

    • Researchers observing classroom behavior but have preconceived notions about what constitutes "engagement"
    • Personal biases in performance evaluations

Person observing employee performance

Conquering Data Bias

Recall bias

People recalling

It occurs when participants inaccurately remember past events or experiences, affecting data reliability

  • It is common when customers provide feedback on past experiences or preferences
  • It can be due to various factors such as memory decay, or external influences
Conquering Data Bias

Social desirability bias

Two people looking for social desirability

It occurs when respondents provide answers they believe are socially acceptable or desirable, rather than truthful responses

  • In customer satisfaction surveys, respondents may exaggerate positive experiences to avoid appearing critical or negative
  • In employee feedback surveys, respondents may inflate their ratings to maintain a favorable image within the organization
Conquering Data Bias

Impact of measurement bias

Flow with incorrect inputs and outputs

  • It adheres to the principle of "garbage in, garbage out"
  • Inaccuracies in measurement methods leads to flawed data inputs
  • It inevitably results in unreliable outputs and flawed decision-making
Conquering Data Bias

Let's practice!

Conquering Data Bias

Preparing Video For Download...