Pandas DataFrame Examples
Check for NaN Values
Pandas uses numpy.nan as NaN value . NaN stands for Not A Number and is one of the most common ways to represent the missing value in the Pandas DataFrame . At the core level, DataFrame provides two methods to test for missing data , isnull() and isna(). These two Pandas methods do exactly the same thing, even their docs are identical.Check for single column
df[ColumnName].isnull().values.any()
Count the NaN under a single column
df[ColumnName].isnull().values.sum()
Check for NaN under entire DataFrame
df.isnull().values.any()
Count the NaN under entire DataFrame
df.isnull().sum().sum()
Which rows have NaNs in a specific column
df[df[ColumnName].isnull()]
Which rows have NaN values
df[df.isnull().any(1)]
How many rows there are with "one or more NaNs"
df.isnull().T.any().T.sum()
Display the columns that has nulls
df.loc[:, df.isnull().any()].columns
Check the percentage of nulls in every column
df.isna().sum()/(len(df))*100
Related Topics
- Pandas DataFrame: GroupBy Examples
- Pandas DataFrame Aggregation and Grouping
- How to Sort Pandas DataFrame
- Pandas DataFrame: query() function
- Finding and removing duplicate rows in Pandas DataFrame
- How to Replace NaN Values With Zeros in Pandas DataFrame
- How to read CSV File using Pandas DataFrame.read_csv()
- How to Convert Pandas DataFrame to NumPy Array
- How to shuffle a DataFrame rows
- Import multiple csv files into one pandas DataFrame
- Create new column in DataFrame based on the existing columns
- New Pandas dataframe column based on if-else condition
- How to Convert a Dictionary to Pandas DataFrame
- Rename Pandas columns/index names (labels)