SQL Data Quality with PostgreSQL

Overview

This project demonstrates a data quality workflow using PostgreSQL to inspect, clean, standardize, and prepare survey data for analysis. The dataset contains 124 records related to food preferences, eating habits, and demographic information.

The project focuses on identifying data quality issues, applying targeted cleaning decisions, and verifying changes after modification.

Data Quality Issues

Initial review identified several issues affecting the consistency and analytical usability of the dataset:

• Missing values represented as the text value nan

• Invalid and non-numeric values in the GPA field

• An inconsistent categorical value in fav_cuisine

• Useful information contained within unstructured text in comfort_food_reasons

Approach

Missing Value Standardization

The calories_day column contained nan as a text value. These values were converted to NULL to provide a consistent representation of missing data.

GPA Data Quality

Of the 124 GPA records, four (approximately 3%) contained invalid responses such as nan, Personal, or Unknown. Because these strings are not valid GPA measurements and represented a small portion of the dataset, the affected records were removed to preserve the integrity of GPA-based analysis while minimizing the impact on the overall dataset.

Categorical Standardization

The fav_cuisine column contained the response Any type of Colombian cuisine, which was inconsistent with the format of the other cuisine categories. The value was standardized to Colombian, allowing it to be grouped and analyzed consistently while preserving the respondent's intended answer.

Feature Engineering from Free Text

The comfort_food_reasons column contains unstructured text, making recurring themes difficult to aggregate directly. Because multiple responses referenced boredom, a new indicator variable, bored_in_the_house, was created to identify boredom-related responses.

This transformation converted a recurring pattern in free-text responses into a structured variable that can be aggregated and analyzed.

SQL Techniques Demonstrated

• Data profiling and validation

• SELECT, GROUP BY, and ORDER BY

• Missing-value handling

• UPDATE and DELETE

• Categorical data standardization

• Pattern matching with ILIKE

• Conditional logic with CASE

• Feature engineering

• Schema modification with ALTER TABLE

• Post-transformation validation

Data Integrity

A backup of the original dataset was created before modifications were made. Changes were subsequently validated using distinct-value and frequency checks to confirm that the intended transformations were successfully applied.

Tools

• PostgreSQL

• pgAdmin

• SQL