Lesson 2: Exploratory Data Analysis (EDA)

2.1 Introduction to Exploratory Data Analysis (EDA)

Exploratory Data Analysis (EDA) is the process of examining, summarizing, and visualizing a dataset to understand its structure, identify patterns, detect anomalies, check assumptions, and discover relationships between variables before applying statistical tests or machine learning models.

EDA is considered one of the most important steps in any data analysis project because it helps ensure data quality and provides valuable insights that guide further analysis.

2.2 Why is EDA Important?

Exploratory Data Analysis helps analysts and researchers to:

• Understand the dataset structure.
• Identify missing values.
• Detect duplicate records.
• Find outliers and unusual observations.
• Explore the distribution of variables.
• Understand relationships between variables.
• Verify data quality.
• Select appropriate statistical methods.
• Generate hypotheses for further analysis.
• Improve model performance by identifying data issues.

Without EDA, incorrect conclusions and poor predictive models may result from hidden problems in the data.

2.3 Objectives of EDA

The main objectives of Exploratory Data Analysis are:

• Understand the dimensions of the dataset.
• Identify variable types.
• Summarize key statistics.
• Detect missing data.
• Identify outliers.
• Examine data distributions.
• Explore relationships between variables.
• Assess data quality.
• Prepare the dataset for further analysis.

2.4 Types of Exploratory Data Analysis

EDA can be classified into two main types:

1. Univariate Analysis

Analyzes one variable at a time.

Examples:

• Mean
• Median
• Mode
• Standard deviation
• Histogram
• Density plot
• Boxplot
• Frequency table

2. Multivariate Analysis

Examines relationships among two or more variables.

Examples:

• Scatter plots
• Correlation analysis
• Pair plots
• Heatmaps
• PCA
• Clustering
• Regression analysis

2.5 Importing Data into R

Before exploring data, it must first be imported into R.

CSV File

data <- read.csv("students.csv")

Excel File

library(readxl)

data <- read_excel("students.xlsx")

2.6 Viewing the Dataset

Display the first six observations:

head(data)

Display the last six observations:

tail(data)

Display the dataset structure:

str(data)

View the dataset dimensions:

dim(data)

Display variable names:

names(data)

Preview the data in a spreadsheet-style viewer:

View(data)

2.7 Understanding Variable Types

Check the data type of each variable:

str(data)

Common R data types include:

• Numeric
• Integer
• Character
• Factor
• Logical
• Date

Example:

class(data$Age)

2.8 Summary Statistics

Generate summary statistics for all variables:

summary(data)

For numerical variables, the summary includes:

• Minimum
• 1st Quartile
• Median
• Mean
• 3rd Quartile
• Maximum

Example:

summary(data$Height)

2.9 Measuring Central Tendency

Calculate the mean:

mean(data$Height)

Calculate the median:

median(data$Height)

Calculate the mode (custom function):

Mode <- function(x){
  ux <- unique(x)
  ux[which.max(tabulate(match(x, ux)))]
}

Mode(data$Gender)

2.10 Measuring Dispersion

Range

range(data$Height)

Variance

var(data$Height)

Standard Deviation

sd(data$Height)

Interquartile Range

IQR(data$Height)

2.11 Frequency Tables

For categorical variables:

table(data$Gender)

Relative frequencies:

prop.table(table(data$Gender))

Percentage frequencies:

round(prop.table(table(data$Gender))*100,2)

2.12 Checking Missing Values

Count missing values:

sum(is.na(data))

Missing values by variable:

colSums(is.na(data))

Rows containing missing values:

data[!complete.cases(data),]

2.13 Detecting Duplicate Records

Check duplicate rows:

duplicated(data)

Count duplicates:

sum(duplicated(data))

Remove duplicates:

data <- unique(data)

2.14 Detecting Outliers

Outliers are observations that differ significantly from the rest of the data.

Using Boxplot

boxplot(data$Height)

Extract outlier values:

boxplot.stats(data$Height)$out

2.15 Data Distribution

Histogram

hist(data$Height)

Density Plot

plot(density(data$Height))

Normal Q-Q Plot

qqnorm(data$Height)
qqline(data$Height)

These plots help determine whether the data follow a normal distribution.

2.16 Correlation Analysis

Calculate the Pearson correlation:

cor(data$Height,
    data$Weight,
    use="complete.obs")

Correlation Matrix

cor(data[sapply(data,is.numeric)])

2.17 Data Visualization for EDA

Histogram

hist(data$Height,
     col="lightblue",
     main="Height Distribution",
     xlab="Height")

Boxplot

boxplot(data$Height,
        col="orange",
        main="Height")

Bar Plot

barplot(table(data$Gender),
        col="steelblue")

Scatter Plot

plot(data$Height,
     data$Weight,
     pch=19,
     col="blue")

Density Plot

plot(density(data$Height),
     lwd=2)

2.18 Using the dplyr Package

Load the package:

library(dplyr)

Display dataset summary:

glimpse(data)

Select variables:

select(data,
       Height,
       Weight)

Filter rows:

filter(data,
       Gender=="Male")

Arrange data:

arrange(data,
        Height)

Create a new variable:

mutate(data,
       BMI = Weight/(Height/100)^2)

Summarize data:

summarise(data,
          MeanHeight = mean(Height))

2.19 Common EDA Workflow

A typical Exploratory Data Analysis workflow includes:

1. Import the dataset.
2. Inspect the data structure.
3. Check dimensions and variable names.
4. Identify data types.
5. Generate summary statistics.
6. Detect missing values.
7. Remove duplicates.
8. Identify outliers.
9. Explore data distributions.
10. Analyze relationships between variables.
11. Create informative visualizations.
12. Prepare the cleaned dataset for further analysis.

2.20 Best Practices for EDA

• Always inspect your data before analysis.
• Check for missing values and duplicates.
• Identify and investigate outliers before removing them.
• Use visualizations alongside summary statistics.
• Validate data types and variable formats.
• Keep a copy of the raw dataset before cleaning.
• Document every data cleaning step for reproducibility.

Lesson 6: Analysis of Variance (ANOVA)

Lesson Preview

Analysis of Variance (ANOVA) is one of the most widely used statistical techniques for comparing the means of two or more groups to determine whether significant differences exist among them. Instead of performing multiple t-tests, which increase the risk of Type I errors, ANOVA provides a reliable framework for evaluating treatment effects, experimental factors, and interactions in research studies.

In this lesson, you will learn the theoretical foundations of ANOVA and its practical implementation in R. You will explore one-way, two-way, and repeated measures ANOVA, understand the underlying assumptions, perform assumption testing, calculate effect sizes, conduct post-hoc multiple comparison tests, and interpret ANOVA outputs. Additionally, you will learn how to create publication-quality visualizations and report ANOVA results according to scientific standards. By the end of this lesson, you will be able to confidently apply ANOVA to real-world datasets in agriculture, biology, medicine, psychology, business, and other research fields.

Lesson 7: Non-Parametric Tests

Lesson Preview

Non-parametric tests are statistical methods used when data do not satisfy the assumptions required for parametric tests, such as normality or homogeneity of variance, or when data are ordinal or ranked. These tests are robust, flexible, and widely applied in research fields including biology, medicine, agriculture, psychology, business, and social sciences.

In this lesson, you will learn when and why to use non-parametric tests, understand their assumptions, and perform the most commonly used tests in R. You will explore rank-based methods such as the Mann–Whitney U test, Wilcoxon Signed-Rank test, Kruskal–Wallis test, Friedman test, Chi-Square test, and Spearman's rank correlation. You will also learn how to conduct post-hoc analyses, calculate effect sizes, create publication-quality visualizations, and interpret results according to scientific reporting standards.

Lesson 8: Multivariate Analysis

Lesson Preview

Multivariate analysis is a collection of statistical techniques used to analyze datasets containing multiple variables simultaneously. Unlike univariate and bivariate methods, multivariate techniques examine complex relationships among variables, identify hidden patterns, reduce data dimensionality, classify observations, and uncover structures within large datasets. These methods are extensively used in biological sciences, agriculture, ecology, medicine, bioinformatics, business analytics, social sciences, and machine learning.

In this lesson, you will learn the theoretical concepts and practical applications of the most widely used multivariate statistical techniques in R. You will explore methods such as Principal Component Analysis (PCA), Principal Coordinate Analysis (PCoA), Cluster Analysis, Multidimensional Scaling (MDS), Canonical Correspondence Analysis (CCA), Redundancy Analysis (RDA), Discriminant Analysis, PERMANOVA, and other ordination techniques. You will also learn how to visualize multivariate results using high-quality publication-ready figures and interpret the outputs for research and real-world applications.

Lesson 9: Time Series Analysis

Lesson Preview

Time Series Analysis is a statistical technique used to analyze data collected sequentially over time. It enables analysts to identify trends, seasonal patterns, cyclical movements, and irregular fluctuations, while also providing powerful forecasting methods for predicting future values. Time series methods are widely applied in economics, finance, agriculture, environmental science, healthcare, engineering, and business analytics to support planning and decision-making.

In this lesson, you will learn the fundamental concepts of time series data and explore advanced forecasting techniques using R. You will understand how to prepare time series data, assess stationarity, decompose time series components, apply smoothing methods, build forecasting models such as ARIMA and Exponential Smoothing, and evaluate model performance. Additionally, you will create professional visualizations and interpret forecasting results using real-world datasets.

Lesson 10: Introduction to Machine Learning in R

Lesson Preview

Machine Learning (ML) is a branch of artificial intelligence that enables computers to learn patterns from data and make predictions or decisions without being explicitly programmed. It has become an essential tool in data science and is widely applied in healthcare, finance, agriculture, business analytics, bioinformatics, environmental science, and many other fields. R provides a rich ecosystem of machine learning packages that allow users to build, evaluate, and deploy predictive models efficiently.

In this lesson, you will learn the fundamental concepts of machine learning and how to implement basic machine learning workflows in R. You will explore different types of machine learning, prepare datasets for modeling, split data into training and testing sets, build predictive models, evaluate model performance, and understand model validation techniques. By the end of this lesson, you will have a solid foundation for applying machine learning algorithms to real-world problems.

Lesson 11: Specialized Visualization Techniques

Lesson Preview

Specialized visualization techniques enable analysts to explore and communicate complex relationships that cannot be effectively represented using traditional charts. These methods are particularly valuable for analyzing networks, interconnected systems, multidimensional relationships, and survey data. They are widely used in social network analysis, bioinformatics, genomics, business intelligence, epidemiology, marketing research, and social sciences.

In this lesson, you will learn how to create advanced visualizations in R using powerful packages such as igraph, ggraph, tidygraph, circlize, networkD3, Likert, and sjPlot. You will build network graphs, chord diagrams, and professional survey visualizations while learning best practices for customization, interpretation, and publication-quality figure preparation.