Check for NaN Values : Pandas DataFrame
In Pandas, a DataFrame is a two-dimensional tabular data structure that allows you to store and manipulate data efficiently. Checking for NaN (Not A Number) values is a crucial step in data analysis and data cleaning, as missing data can significantly impact the accuracy and validity of your results.
Pandas provides two main methods for checking NaN values in a DataFrame: isnull() and isna(). Both methods return a DataFrame of the same shape as the input DataFrame, but with boolean values indicating whether each element is NaN or not. A True value indicates a NaN value, while False indicates a non-NaN value.
Check for single column
Count the NaN under a single column
The isnull() method is used to detect missing values (NaN) in a DataFrame. It returns a DataFrame with the same shape, where each element is True if it's NaN and False otherwise.
In this example, result_isnull is a DataFrame with the same shape as the original DataFrame df. It indicates that the first row has no NaN values, the second row has a NaN value in the 'Salary' column, the third row has a NaN value in the 'Age' column, and the fourth row has a NaN value in the 'Name' column.
Check for NaN under entire DataFrame
Count the NaN under entire DataFrame
The isna() method is an alias for isnull(), meaning they are entirely interchangeable. Both methods serve the same purpose of detecting NaN values.
As you can see, the result_isna DataFrame is identical to the previous result_isnull DataFrame, confirming that both methods produce the same output.
Which rows have NaNs in a specific column
Which rows have NaN values
How many rows there are with "one or more NaNs"
Display the columns that has nulls
Check the percentage of nulls in every column
Pandas provides the isnull() and isna() methods to efficiently detect NaN values in a DataFrame, and you can use them interchangeably. By understanding and utilizing these methods, you can identify and handle missing data effectively in your data analysis workflows.