Introduction of Statistics in R
install.packages("rlang")
Run to view results
install.packages("ggplot2")
install.packages("dplyr")
Run to view results
library(ggplot2)
library(dplyr)
Run to view results
Summary Statistics
What is statistics?
The field of statistics: the practice and study of collecting and analyzing data
A summary statistic: a fact about or summary of some data
What can statistics do?
How likely is someone to purchase a product? Are people more likely to purchase it if they can use a different payment system?
How many occupants will your hotel have? How can you optimize occupancy?
How many sizes of jeans need to be manufactured so they can fit 95% of the population? Should the same number of each size be produced?
A/B tests: Which ad is more effective in getting people to purchase a product?
What can't statistics do?
Why is Game of Thrones so popular?
Instead...
Are series with more violent scenes viewed by more people?
But...
Even so, this can´t tell us if more violent scenes lead to more views
Type of statistics
Descriptive statistics: Describe and summarize data
Inferential statistics: Use a sample of data to make inference about a larger population
Types of data
Numeric (Quantitative)
Continuous (Measured): Airplane speed, Time spent waiting in line
Discrete (Counted): Number of pets, number of packages shipped
Categorical (Qualitative)
Nominal (Unordered): Married/unmarried, Country of residence
Ordinal (Ordered): Strongly disagree, Somewhat disagree, Neither agree nor disagree, Somewhat agree, Strongly agree
Categorical data can be represented as numbers
Nominal (Unordered): Married/unmarried (1/0), Country of residence (1,2,...)
Ordinal (Ordered): Strongly disagree (1), Somewhat disagree (2), Neither agree nor disagree (3), Somewhat agree (4), Strongly agree (5)
Measures of centers
msleep
Run to view results
Mean
mean(msleep$sleep_total)
Run to view results
Median
sort(msleep$sleep_total)
sort(msleep$sleep_total)[42]
median(msleep$sleep_total)
Run to view results
Mode
msleep %>% count(sleep_total, sort = TRUE)
Run to view results
msleep %>% count(vore, sort = TRUE)
Run to view results
Summary of Sleep Statistics for Insectivores
msleep %>%
filter(vore == "insecti") %>%
summarize(mean_sleep = mean(sleep_total),
median_sleep = median(sleep_total))
Run to view results
Measures of spread
Spread: How spread apart or close together the data points are
Variance: Average distance from each data point to the data's mean
# Distance between each point and the mean
dists <- msleep$sleep_total - mean(msleep$sleep_total)
dists
Run to view results
# Square each distance
squared_dists <- (dists)^2
squared_dists
# Sum all the squared distances
sum_sq_dists <- sum(squared_dists)
sum_sq_dists
# Divide the sum by the number of data points minus 1
variance <- sum_sq_dists / (length(msleep$sleep_total) - 1)
variance
# Standard deviation is the square root of the variance
sd <-sqrt(var(msleep$sleep_total))
sd
Run to view results
Variance function
var(msleep$sleep_total)
Run to view results
Standard desviation
sd(msleep$sleep_total)
Run to view results
Mean absolute deviation
dists <- msleep$sleep_total - mean(msleep$sleep_total)
mean(abs(dists))
Run to view results
Standard deviation vs. mean absolute deviation
SD squares distances, penalizing longer distances more than shorter ones.
MAD penalizes each distance equally
One isn't better than the other, but SD is more common than MAD.
Quartiles
Split up the data into four equal parts
quantile(msleep$sleep_total)
Run to view results
Second quartile/50th percentile = median
25% of the date is between 1.9 and 7.85
Boxplots use quartiles
Bottom of the box: first quartile
MIddle line: second quartile, or the median.
Top of the box: third quartile
ggplot(msleep, aes(y = sleep_total)) +
geom_boxplot()
Run to view results
Quantiles
Split data into five pieces or ten pieces
quantile(msleep$sleep_total, probs = c(0, 0.2, 0.4, 0.8, 1))
Run to view results
# seq(from, to, by)
quantile(msleep$sleep_total, probs = seq(0, 1, 0.2))
Run to view results
Interquartile range (IQR)
Distance between the 25th and 75th percentile
Also is the height of the box in a boxplot
iqr = quantile(msleep$sleep_total, 0.75) - quantile(msleep$sleep_total, 0.25)
iqr
Run to view results
Outliers
Data point that is substantially different from the others
data < Q1 - 1.5 X IQR or
data > Q3 - 1.5 X IQR
Finding outliers
iqr <- quantile(msleep$bodywt, 0.75) - quantile(msleep$bodywt, 0.25)
lower_threshold <- quantile(msleep$bodywt, 0.25) - 1.5 * iqr
upper_threshold <- quantile(msleep$bodywt, 0.75) + 1.5 * iqr
msleep %>%
filter(bodywt < lower_threshold | bodywt > upper_threshold) %>%
select(name, vore, sleep_total, bodywt)
Run to view results
Random Numbers and Probability
What are the chances?
name <- c("Amir", "Brian", "Claire", "Damian")
n_sales <- c(178, 126, 75, 69)
sales_counts <- data.frame(name, n_sales)
sales_counts
Run to view results
Sampling without replacement
sales_counts %>%
sample_n(1)
Run to view results
Setting a random seed
The seed is a number that R's random number generator uses as a starting point, it will generate the same random value each time
set.seed(5)
sales_counts %>%
sample_n(1)
Run to view results
Sampling with replacement in R
sales_counts %>%
sample_n(1, replace = TRUE)
Run to view results
Discrete distribution
n <- c(1,2,3,4,5,6)
die <- data.frame(n)
mean(die$n)
rolls_10 <- die %>%
sample_n(10, replace = TRUE)
mean(rolls_10$n)
ggplot(rolls_10, aes(n)) +
geom_histogram(bins = 6)
Run to view results
rolls_100 <- die %>%
sample_n(100, replace = TRUE)
mean(rolls_100$n)
ggplot(rolls_100, aes(n)) +
geom_histogram(bins = 6)
Run to view results
rolls_1000 <- die %>%
sample_n(1000, replace = TRUE)
mean(rolls_1000$n)
ggplot(rolls_1000, aes(n)) +
geom_histogram(bins = 6)
Run to view results