#!/usr/bin/env python
# coding: utf-8

# # **TikTok Project**
# **Topic: Descriptive Statistics & Hypothesis Testing**

# # **Data Exploration and Hypothesis Testing**
# 
# In this project, we will explore the TikTok dataset and conduct a hypothesis testing.
# <br/>
# 
# **The purpose** of this project is to demostrate knowledge of preparing, creating, and analyzing hypothesis tests.
# 
# **The goal** is to apply descriptive and inferential statistics, probability distributions, and hypothesis testing in Python.
# <br/>
# 
# *The project consists of three parts:*
# 
# **Part 1:** Imports and data loading
# 
# **Part 2:** Conduct hypothesis testing
# 
# **Part 3:** Communicate insights with stakeholders

# # **Data Exploration and Hypothesis Testing**

# # **PACE stages: Plan, Analyze, Construct, and Execute**
# 

# # **PACE: Plan**
# 
# 1) Do videos from verified accounts and videos unverified accounts have different average view counts?
# 
# 2) Is there a relationship between the account being verified and the associated videos' view counts?
# 

# ### **Part 1 - Imports and Data Loading**

# Import packages and libraries needed to compute descriptive statistics and conduct a hypothesis test.

# In[2]:


# Import packages for data manipulation
import pandas as pd
import numpy as np

# Import packages for data visualization
import matplotlib.pyplot as plt
import seaborn as sns

# Import packages for statistical analysis/hypothesis testing
from scipy import stats


# In[3]:


# Load dataset into dataframe
data = pd.read_csv("tiktok_dataset.csv")


# ## **PACE: Analyze and Construct**
# Descriptive statistics are useful because they allow us to quickly explore and understand large amounts of data. In this case, descriptive statistics helps quickly compute the mean values of video_view_count for each group of verified_status in the sample data.

# ### **Task 2. Data exploration**
# 
# Use descriptive statistics to conduct Exploratory Data Analysis (EDA).
# 

# Inspect the first five rows of the dataframe.

# In[4]:


# Display first few rows
data.head()


# In[5]:


# Generate a table of descriptive statistics about the data
data.describe()


# Check for and handle missing values.

# In[6]:


# Check for missing values
data.isna().sum()


# In[7]:


# Drop rows with missing values
data = data.dropna(axis=0)


# In[8]:


# Display first few rows after handling missing values
data.head()


# Let's look into the relationship between `verified_status` and `video_view_count`. One approach is to examine the mean values of `video_view_count` for each group of `verified_status` in the sample data.

# In[9]:


# Compute the mean `video_view_count` for each group in `verified_status`
### YOUR CODE HERE ###
data.groupby("verified_status")["video_view_count"].mean()


# ### **Task 3. Hypothesis testing**

# *   **Null hypothesis**: There is no difference in number of views between TikTok videos posted by verified accounts and TikTok videos posted by unverified accounts (any observed difference in the sample data is due to chance or sampling variability).
# *    **Alternative hypothesis**: There is a difference in number of views between TikTok videos posted by verified accounts and TikTok videos posted by unverified accounts (any observed difference in the sample data is due to an actual difference in the corresponding population means).
# 
# 

# 
# 
# Our goal is to conduct a two-sample t-test. Here are the steps:
# 
# 
# 1.   State the null hypothesis and the alternative hypothesis
# 2.   Choose a signficance level (5% in this project)
# 3.   Find the p-value
# 4.   Reject or fail to reject the null hypothesis
# 
# 

# In[10]:


# Conduct a two-sample t-test to compare means
### YOUR CODE HERE ###

# Save each sample in a variable
not_verified = data[data["verified_status"] == "not verified"]["video_view_count"]
verified = data[data["verified_status"] == "verified"]["video_view_count"]

# Implement a t-test using the two samples
stats.ttest_ind(a=not_verified, b=verified, equal_var=False)


# **Result:**
# 
# Since the p-value is extremely small (much smaller than the significance level of 5%), we can reject the null hypothesis. You conclude that there **is** a statistically significant difference in the mean video view count between verified and unverified accounts on TikTok.
# 

# # **PACE: Execute**
# 
# Consider the questions in your PACE Strategy Document to reflect on the Execute stage.

# ### **Task 4. Communicate insights with stakeholders**

# *Question:*
# 
# *  What business insight(s) can you draw from the result of your hypothesis test?
# 

# The analysis shows that there is a statistically significant difference in the average view counts between videos from verified accounts and videos from unverified accounts. This suggests there might be fundamental behavioral differences between these two groups of accounts.
# 
# It would be interesting to investigate the root cause of this behavioral difference. For example, do unverified accounts tend to post more clickbait-y videos? Or are unverified accounts associated with spam bots that help inflate view counts?
# 
# The next step will be to build a regression model on verified_status. A regression model is the natural next step because the end goal is to make predictions on claim status. A regression model for verified_status can help analyze user behavior in this group of verified users. Technical note to prepare regression model: because the data is skewed, and there is a significant difference in account types, it will be key to build a logistic regression model.
# 
#