Pandas DataFrame Examples

Check for NaN Values

Pandas uses numpy.nan as NaN value . NaN stands for Not A Number and is one of the most common ways to represent the missing value in the Pandas DataFrame . At the core level, DataFrame provides two methods to test for missing data , isnull() and isna(). These two Pandas methods do exactly the same thing, even their docs are identical.

Check for single column

df[ColumnName].isnull().values.any()

Count the NaN under a single column

df[ColumnName].isnull().values.sum()

Check for NaN under entire DataFrame

df.isnull().values.any()

Count the NaN under entire DataFrame

df.isnull().sum().sum()

Which rows have NaNs in a specific column

df[df[ColumnName].isnull()]

Which rows have NaN values

df[df.isnull().any(1)]

How many rows there are with "one or more NaNs"

df.isnull().T.any().T.sum()

Display the columns that has nulls

df.loc[:, df.isnull().any()].columns

Check the percentage of nulls in every column

df.isna().sum()/(len(df))*100