Introduction to Pandas

Pandas is a powerful open-source Python library for data analysis and manipulation. It provides easy-to-use data structures and data analysis tools for working with structured (tabular, multidimensional, potentially heterogeneous) and time series data.

Importing Pandas

import pandas as pd

This line imports the Pandas library and assigns it the conventional abbreviation 'pd'.

Series

A Pandas Series is a one-dimensional labeled array capable of holding any data type.

data = pd.Series([1, 2, 3, 4, 5])
print(data)

Output:

0    1
1    2
2    3
3    4
4    5
dtype: int64

DataFrames

A Pandas DataFrame is a 2-dimensional labeled data structure, like a 2D array, with columns of potentially different data types.

data = {'Name': ['John', 'Jane', 'Jim', 'Joan'],
        'Age': [25, 32, 19, 27]}
df = pd.DataFrame(data)
print(df)

Output:

   Name  Age
0  John   25
1  Jane   32
2   Jim   19
3  Joan   27

Reading Data

Pandas can read data from various file formats like CSV, Excel, SQL databases, and more.

df = pd.read_csv('data.csv')

Data Selection

Selecting data from a DataFrame is easy with Pandas indexing.

print(df['Name'])    # Select a column
print(df.loc[0])     # Select a row by label
print(df.iloc[0, 1]) # Select a value by row/column number

Data Manipulation

Pandas provides powerful tools for reshaping, merging, and cleaning data.

df['Age_months'] = df['Age'] * 12  # Add a new column
df.dropna(inplace=True)             # Drop rows with missing values
df.rename(columns={'Age': 'Years'}, inplace=True) # Rename a column

Grouping and Aggregating

Grouping and aggregating data is a common operation in data analysis.

grouped = df.groupby('Name')['Age'].sum()
print(grouped)

Output:

Name
Jane    32
Jim     19
Joan    27
John    25
Name: Age, dtype: int64

Plotting

Pandas integrates well with Matplotlib and other data visualization libraries.

import matplotlib.pyplot as plt
df.plot(kind='scatter', x='Age', y='Height')
plt.show()

Data Cleaning

Pandas provides utilities for cleaning and preprocessing data.

import numpy as np

# Replace values
df['Age'].replace([19, 27], np.nan, inplace=True)

# Drop duplicates
df.drop_duplicates(inplace=True)

# Handle missing values
df['Age'] = df['Age'].fillna(df['Age'].mean())

Merging and Joining

Pandas makes it easy to combine datasets using merges and joins.

# Merge two DataFrames
pd.merge(df1, df2, on='key', how='inner')

# Join on indexes
df1.join(df2, lsuffix='_left', rsuffix='_right')

Time Series Data

Pandas has excellent support for working with time series data.

# Convert to datetime
df['Date'] = pd.to_datetime(df['Date'])

# Set index
df = df.set_index('Date')

# Resample
df.resample('M').mean()

Handling Large Datasets

Pandas provides tools for efficient handling of large datasets.

# Chunking data
for chunk in pd.read_csv('large_file.csv', chunksize=10000):
    process_data(chunk)

# Data types and memory usage
df.info(memory_usage='deep')

Integration with Other Libraries

Pandas integrates well with other data science libraries in Python.

# NumPy for numerical operations
df['New_Col'] = np.sqrt(df['Col1'] ** 2 + df['Col2'] ** 2)

# Scikit-learn for machine learning
from sklearn.linear_model import LinearRegression
X = df[['Col1', 'Col2']]
y = df['Target']
model = LinearRegression().fit(X, y)

These additional slides cover more advanced topics in Pandas, such as data cleaning, merging and joining datasets, working with time series data, handling large datasets, and integrating Pandas with other Python libraries like NumPy and Scikit-learn.

Introduction to Pandas