Data science ethics
Announcements
HW 04 due TODAY at 11:59pm
Project: Preliminary analysis due April 7
Statistics experience due April 15
Application exercise
Cross validation for logistic regression
Which model do you select?
Data science ethics
When things go wrong
Gap in public trust
“…coverage of polls [data science] often does not adequately convey the many decisions that pollsters [data scientists] must make … as well as the potential consequences of those decisions”
Clinton, J. (2021, January 11). Polling problems and why we should still trust (some) polls. The Vanderbilt Project on Unity and American Democracy. https://www.vanderbilt.edu/unity/2021/01/11/polling-problems-and-why-we-should-still-trust-some-polls/
“Ideal objectivity?”
Desire for “ideal objectivity”
Work that is free from interference of human perception (Feinberg 2023)
Conclusions that are observer independent (Gelman and Hennig 2017)
Feels like the ethical approach (Feinberg 2023)
There are implications (Feinberg 2023)
Distorted reality of data analysis process
Lack of communication about decisions that don’t seem “objective”
The role of data science ethics
Illuminates the the decision-making in data science
Makes explicit the existence of choice throughout an analysis
Makes explicit the moral implications of our choices
Align data science practices with what we ought to do and moral duties to stakeholders
Source: Colando and Hardin (2024b)
Processes in data science
- Problem definition: Question we wish to answer with data
- What is the likelihood a customer purchases product S?
- Our model (built on training data) is successful if it achieves at least 75% accuracy on testing data
Source: Colando and Hardin (2024b)
Processes in data science
- Raw data: Information collected by interacting with the world
- Information from each time the customer clicks on an advertisement for product S
- Includes timestamps for each advertisement interaction and the customer’s demographic information
Source: Colando and Hardin (2024b)
Processes in data science
- Data: Processed form of raw data
- Data table where each row represents a unique customer and the columns are the variables that describe that customer
- Includes information from raw data and engineered variables (e.g., average time between clicks)
- We decide what to do with missing data
Source: Colando and Hardin (2024b)
Processes in data science
- Data model(s): Product from running input data through learning algorithm (generalize the relationship between variables in the data)
- We choose a logistic regression model of the form
\[ \log(\frac{\pi}{1-\pi}) = \beta_0 + \beta_1 ~ \text{age} + \beta_2 ~ \text{number clicks} + \beta_3 ~ \text{avg time between clicks} \]
Source: Colando and Hardin (2024b)
Processes in data science
- Tuned data model(s): Data model in which the parameters are adjusted
- Use 5-fold cross validation to determine which variables to include in order to improve model’s accuracy
Source: Colando and Hardin (2024b)
Processes in data science
- Deployment and usage: Generating predictions (or other output) from the tuned model
- Determine that the model should only be used for customers under a certain age
- Determine only data scientists at the company selling product S should be able to access model and sell products from it
Source: Colando and Hardin (2024b)
Your data science process
What are some decisions you have made (or will make) in your project or other data analysis work?
What is data science ethics?
Data science ethics studies and evaluates moral problems related to:
Data: generation, recording, process, dissemination, and sharing
Algorithms: artificial intelligence, machine learning, large language models, and statistical learning models
Corresponding practices: responsible innovation, programming, hacking, professional code
Data science ethics issues
Bias, fairness, and justice
Causation
Data privacy and informed consent
Explainability, interpretability, and transparency
Responsibility
Applications in government and policing
Professional ethics
Reproducibility
..and more
Data Science Oath
I will not be ashamed to say, “I know not,” nor will I fail to call in my colleagues when the skills of another are needed for solving a problem.
I will respect the privacy of my data subjects, for their data are not disclosed to me that the world may know, so I will tread with care in matters of privacy and security.
I will remember that my data are not just numbers without meaning or context, but represent real people and situations, and that my work may lead to unintended societal consequences, such as inequality, poverty, and disparities due to algorithmic bias.
From National Academies of Sciences, Engineering, and Medicine (2018) based on the Hippocratic Oath for physicians.
Responsibility
The data scientist is (morally) responsible for
- How the data are handled and stored
- Modeling decisions
- Implications of data handling and modeling
- Providing clear documentation and guidance on model use
. . .
The user is (morally) responsible for
- Understanding the model and its limitations
- How the model is deployed
Source: Colando and Hardin (2024b)
Example: Mistaken identity
Angela Lipps, from Tennessee, spent 5 months in jail after facial recognition software connected her to a string of bank fraud cases in Fargo, North Dakota
- Police were investigating a string of cases in Fargo, ND in which a woman used a fake ID to withdraw money from a bank account or home-equity line of credit
- Investigators used facial recognition technology to identify a potential suspect based on bank surveillance video
- The software “identified a potential suspect with similar features to Angela Lipps”
Source: New York Times
Example: Mistaken identity
- Police used Angela Lipps’ Facebook and Instagram accounts, along with her Tennessee ID to determine she matched the identified suspect
- She was released from jail after a judge dismissed the case
Source: New York Times
Example: Mistaken identity
Facial recognition company, Clearview AI uses “publicly available images” for its data base and to train its models used by law enforcement agencies
Their statement on how models should be used:
“Once a search is performed, the search may return a set of potential leads, which the investigator is required to independently verify by both peer review and other means, before continuing with their investigation.”
Website reports “99% accuracy for all demographics”
Source: https://www.clearview.ai/
Example: Mistaken identity
What are the ethical considerations for the data scientists building such facial recognition technology?
What are the ethical responsibilities of the data scientists building such facial recognition technology?
What are the ethical considerations for those using such facial recognition technology?
What are the ethical responsibilities of those using such facial recognition technology?
Example case study
A data analyst received permission to analyze a data set that was scraped from a social media site. The full data set included name, screen name, email address, geographic location, IP (internet protocol) address, demographic profiles, and preferences for relationships.
What are ethical considerations of putting a deidentified data set with name and email address removed in a LLM (e.g., Claude or ChatGPT) to help with analysis?
Adapted from Chapter 8 of Baumer, Kaplan, and Horton (2024)
Further reading
Further reading
When things go wrong examples:
Cambridge Analytica made ‘ethical mistakes’ because it was too focused on regulation, former COO says from Vox.com
How big data is helping states kick poor people off welfare from Vox.com
How AI researchers uncover ethical, legal risks to using popular data sets from the Washington Post
Boeing’s manufacturing, ethical lapses go back decades from the Seattle Times
Further reading
Gap in public trust examples
Can we trust the polls this year? from Vox.com
Polling problems and why we should still trust (some) polls from Vanderbilt Project on Unity and American Democracy
So, can we trust the polls? from New York Times
Can we still trust the polls? from University of Southern California
Next class
Special topic: Causal inference
Complete Lecture 21 prepare