Pandas DataFrame Examples

Check for NaN Values

Pandas uses numpy.nan as NaN value . NaN stands for Not A Number and is one of the most common ways to represent the missing value in the Pandas DataFrame . At the core level, DataFrame provides two methods to test for missing data , isnull() and isna(). These two Pandas methods do exactly the same thing, even their docs are identical.

Check for single column

df[ColumnName].isnull().values.any()

Count the NaN under a single column

df[ColumnName].isnull().values.sum()

Check for NaN under entire DataFrame

df.isnull().values.any()

Count the NaN under entire DataFrame

df.isnull().sum().sum()

Which rows have NaNs in a specific column

df[df[ColumnName].isnull()]

Which rows have NaN values

df[df.isnull().any(1)]

How many rows there are with "one or more NaNs"

df.isnull().T.any().T.sum()

Display the columns that has nulls

df.loc[:, df.isnull().any()].columns

Check the percentage of nulls in every column

df.isna().sum()/(len(df))*100


Related Topics
  1. Pandas DataFrame: GroupBy Examples
  2. Pandas DataFrame Aggregation and Grouping
  3. How to Sort Pandas DataFrame
  4. Pandas DataFrame: query() function
  5. Finding and removing duplicate rows in Pandas DataFrame
  6. How to Replace NaN Values With Zeros in Pandas DataFrame
  7. How to read CSV File using Pandas DataFrame.read_csv()
  8. How to Convert Pandas DataFrame to NumPy Array
  9. How to shuffle a DataFrame rows
  10. Import multiple csv files into one pandas DataFrame
  11. Create new column in DataFrame based on the existing columns
  12. New Pandas dataframe column based on if-else condition
  13. How to Convert a Dictionary to Pandas DataFrame
  14. Rename Pandas columns/index names (labels)