I love watching TV series and movies, most recently Ted Lasso seems to be the talk of the town. I loved the first season and can't wait to watch the second season. I also have an interest in the Emmy's, I watch it every year. The reason for this analysis is to determine if Twitter has any influence on the Emmy winner of Best Comedy Series. I first pulled Tweets that mention Ted Lasso or #tedlasso along with blackish or #blackish. Once I got a reasonable amount of Tweets i used the VADER toolkit(?) to determine the positive and negative sentiments of the Tweets that i pulled. Using VADER i can measure the sentiment of the Tweets to then compare the two shows. Once i got the sentiments of each show i created a histogram of the negative, positive, neutral, and compound of the tweets. This allowed me to get a visual of each sentiment and see which had more weight to it. Looking at these graphs comparatively, I can estimate that Ted Lasso has more positive Tweets than blackish.
import requests
import pandas as pd
import urllib
import json
bearer_token = pd.read_csv('bearer_token.txt', header = 0)
bearer_token
bearer_token['bearer_token'].iloc[0]
header = {'Authorization' : 'Bearer {}'.format(bearer_token['bearer_token'].iloc[0])}
bearer_token.keys()
Index(['bearer_token'], dtype='object')
endpoint_url = 'https://api.twitter.com/2/tweets/search/recent'
query = urllib.parse.quote('(#TedLasso OR Ted Lasso lang:en)')
query
'%28%23TedLasso%20OR%20Ted%20Lasso%20lang%3Aen%29'
my_api_url = endpoint_url + '?query={}'.format(query)
my_api_url
'https://api.twitter.com/2/tweets/search/recent?query=%28%23TedLasso%20OR%20Ted%20Lasso%20lang%3Aen%29'
tweet_fields = 'public_metrics,created_at,author_id,lang'
my_api_url = endpoint_url + '?query={}&tweet.fields={}'.format(query, tweet_fields)
my_api_url
'https://api.twitter.com/2/tweets/search/recent?query=%28%23TedLasso%20OR%20Ted%20Lasso%20lang%3Aen%29&tweet.fields=public_metrics,created_at,author_id,lang'
expansions = 'author_id'
url = endpoint_url + '?query={}&max_results=10&tweet.fields{}'.format(query, tweet_fields)
response = requests.request("GET", url, headers = header)
url_expansions = endpoint_url + '?query={}&max_results=100&tweet.fields={}&expansions={}&user.fields={}'.format(query, tweet_fields, expansions, 'username')
url_expansions
'https://api.twitter.com/2/tweets/search/recent?query=%28%23TedLasso%20OR%20Ted%20Lasso%20lang%3Aen%29&max_results=100&tweet.fields=public_metrics,created_at,author_id,lang&expansions=author_id&user.fields=username'
first_response = requests.request("GET", url_expansions, headers = header)
first_response.text
first_dict = json.loads(first_response.text)
first_dict.keys()
dict_keys(['data', 'includes', 'meta'])
my_df = pd.DataFrame(first_dict['data'])
my_df
author_id | text | created_at | id | public_metrics | lang | |
---|---|---|---|---|---|---|
0 | 1855042940 | RT @goldenglobes: The #GoldenGlobes nominees f... | 2021-12-13T23:40:21.000Z | 1470539115343011846 | {'retweet_count': 48, 'reply_count': 0, 'like_... | en |
1 | 1322320427507765248 | RT @JarettSays: My heart feels so full when I ... | 2021-12-13T23:40:06.000Z | 1470539054353629189 | {'retweet_count': 43, 'reply_count': 0, 'like_... | en |
2 | 152531825 | RT @QUEERLASSO: I dont think we talk about thi... | 2021-12-13T23:39:52.000Z | 1470538993666301952 | {'retweet_count': 154, 'reply_count': 0, 'like... | en |
3 | 25866804 | late to ted lasso but woowee that nate charact... | 2021-12-13T23:39:37.000Z | 1470538932492324864 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | en |
4 | 1060508317204791296 | RT @QUEERLASSO: I dont think we talk about thi... | 2021-12-13T23:39:06.000Z | 1470538802523430912 | {'retweet_count': 154, 'reply_count': 0, 'like... | en |
... | ... | ... | ... | ... | ... | ... |
95 | 294151373 | RT @goldenglobes: The #GoldenGlobes nominees f... | 2021-12-13T23:12:34.000Z | 1470532126265204736 | {'retweet_count': 48, 'reply_count': 0, 'like_... | en |
96 | 1909483010 | RT @WestLondonFS: The @goldenglobes nomination... | 2021-12-13T23:11:56.000Z | 1470531964050370561 | {'retweet_count': 34, 'reply_count': 0, 'like_... | en |
97 | 1392853644840685570 | 📸 | jason sudeikis, the cast of ted lasso and ... | 2021-12-13T23:11:13.000Z | 1470531785352228867 | {'retweet_count': 9, 'reply_count': 0, 'like_c... | en |
98 | 828768590392389632 | RT @DiscussingFilm: The #GoldenGlobes nominees... | 2021-12-13T23:11:05.000Z | 1470531751131086848 | {'retweet_count': 216, 'reply_count': 0, 'like... | en |
99 | 1076748693490016256 | RT @DiscussingFilm: The #GoldenGlobes nominees... | 2021-12-13T23:10:09.000Z | 1470531518502158336 | {'retweet_count': 332, 'reply_count': 0, 'like... | en |
100 rows × 6 columns
first_dict['meta']
{'newest_id': '1470539115343011846', 'oldest_id': '1470531518502158336', 'result_count': 100, 'next_token': 'b26v89c19zqg8o3fpe15ccn9w787zzmkvi2jx3avr9tvh'}
my_api_url2 = my_api_url + '&next_token={}&max_results=100'.format(first_dict['meta']['next_token'])
second_response = requests.request("GET", my_api_url2, headers = header)
second_response.text
my_df = my_df.append(pd.DataFrame(json.loads(second_response.text)['data']), ignore_index= True)
my_df
author_id | text | created_at | id | public_metrics | lang | |
---|---|---|---|---|---|---|
0 | 1855042940 | RT @goldenglobes: The #GoldenGlobes nominees f... | 2021-12-13T23:40:21.000Z | 1470539115343011846 | {'retweet_count': 48, 'reply_count': 0, 'like_... | en |
1 | 1322320427507765248 | RT @JarettSays: My heart feels so full when I ... | 2021-12-13T23:40:06.000Z | 1470539054353629189 | {'retweet_count': 43, 'reply_count': 0, 'like_... | en |
2 | 152531825 | RT @QUEERLASSO: I dont think we talk about thi... | 2021-12-13T23:39:52.000Z | 1470538993666301952 | {'retweet_count': 154, 'reply_count': 0, 'like... | en |
3 | 25866804 | late to ted lasso but woowee that nate charact... | 2021-12-13T23:39:37.000Z | 1470538932492324864 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | en |
4 | 1060508317204791296 | RT @QUEERLASSO: I dont think we talk about thi... | 2021-12-13T23:39:06.000Z | 1470538802523430912 | {'retweet_count': 154, 'reply_count': 0, 'like... | en |
... | ... | ... | ... | ... | ... | ... |
195 | 1575603678 | RT @BatatinhaGeek: As indicadas a 'Melhor Atri... | 2021-12-13T22:35:03.000Z | 1470522682773061635 | {'retweet_count': 1, 'reply_count': 0, 'like_c... | en |
196 | 1575603678 | RT @BatatinhaGeek: Os indicados a 'Melhor Ator... | 2021-12-13T22:35:01.000Z | 1470522675860754434 | {'retweet_count': 2, 'reply_count': 0, 'like_c... | en |
197 | 363940693 | In honor of the holiday season and the upcomin... | 2021-12-13T22:34:33.000Z | 1470522556549668869 | {'retweet_count': 0, 'reply_count': 1, 'like_c... | en |
198 | 471528838 | I miss Ted Lasso Twitter https://t.co/N5dQWP14hd | 2021-12-13T22:34:04.000Z | 1470522434923089924 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | en |
199 | 1409283084520890370 | RT @manvillebea: if there’s one thing the ted ... | 2021-12-13T22:33:52.000Z | 1470522384541052928 | {'retweet_count': 3, 'reply_count': 0, 'like_c... | en |
200 rows × 6 columns
my_df.to_csv('tedLassoAnalysis.csv')
lasso = pd.read_csv('tedLassoAnalysis.csv', encoding = 'utf-8')
lasso.keys()
lasso['lang'].astype('category')
lasso_posts = lasso[(lasso['lang']) == 'en']
lasso_posts['lang'].astype('category')
lasso_posts['text']
0 RT @goldenglobes: The #GoldenGlobes nominees f... 1 RT @JarettSays: My heart feels so full when I ... 2 RT @QUEERLASSO: I dont think we talk about thi... 3 late to ted lasso but woowee that nate charact... 4 RT @QUEERLASSO: I dont think we talk about thi... ... 195 RT @BatatinhaGeek: As indicadas a 'Melhor Atri... 196 RT @BatatinhaGeek: Os indicados a 'Melhor Ator... 197 In honor of the holiday season and the upcomin... 198 I miss Ted Lasso Twitter https://t.co/N5dQWP14hd 199 RT @manvillebea: if there’s one thing the ted ... Name: text, Length: 193, dtype: object
lasso_posts['text'].iloc[0]
lasso_posts['text'].iloc[0].lower()
lowercase = [x.lower() for x in lasso_posts['text']]
lowercase[0]
from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
analyze= SentimentIntensityAnalyzer()
print(analyze.polarity_scores(lasso_posts['text'].iloc[3]))
{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}
senti = analyze.polarity_scores(lasso_posts['text'].iloc[3])
senti['neu']
1.0
all_sents = [analyze.polarity_scores(x) for x in lasso_posts['text']]
all_sents
In this section I created all of the individual tweets sentiments into a dataframe so that it is easier to analyze. Although the dataframe does not give me much information so i visualized it by making a histogram for the positive and negative sentiment.
sents_df = pd.DataFrame(all_sents)
sents_df
neg | neu | pos | compound | |
---|---|---|---|---|
0 | 0.000 | 0.729 | 0.271 | 0.7717 |
1 | 0.000 | 0.769 | 0.231 | 0.6369 |
2 | 0.000 | 1.000 | 0.000 | 0.0000 |
3 | 0.000 | 1.000 | 0.000 | 0.0000 |
4 | 0.000 | 1.000 | 0.000 | 0.0000 |
... | ... | ... | ... | ... |
188 | 0.091 | 0.785 | 0.124 | 0.2023 |
189 | 0.087 | 0.794 | 0.119 | 0.2023 |
190 | 0.000 | 0.730 | 0.270 | 0.9551 |
191 | 0.242 | 0.758 | 0.000 | -0.1531 |
192 | 0.095 | 0.905 | 0.000 | -0.3182 |
193 rows × 4 columns
sents_df['pos'].describe()
count 193.000000 mean 0.130731 std 0.142610 min 0.000000 25% 0.000000 50% 0.087000 75% 0.258000 max 0.463000 Name: pos, dtype: float64
from matplotlib import pyplot as plt
fig0, ax0 = plt.subplots()
ax0.hist(sents_df['pos'])
plt.show()
fig1, ax1 = plt.subplots()
ax1.hist(sents_df['neg'])
plt.show()
Here I visualized each sentiment into its own histogram. Here you can see there are few negative tweets but looks like there are more positive tweets than negative.
fig4, ax4 = plt.subplots(1, 4, figsize = (10, 5))
ax4[0].hist(sents_df['neg'], bins = 8, color = 'red')
ax4[1].hist(sents_df['pos'], bins = 8, color = 'blue')
ax4[2].hist(sents_df['neu'], bins = 8, color = 'green')
ax4[3].hist(sents_df['compound'], bins = 8, color = 'yellow')
ax4[0].set_xlabel("Negative")
ax4[1].set_xlabel("Positive")
ax4[2].set_xlabel("Neutrual")
ax4[3].set_xlabel("Compound")
fig4.suptitle("Ted Lasso Sentiments")
plt.show()
endpoint_url2 = 'https://api.twitter.com/2/tweets/search/recent'
query2 = urllib.parse.quote('(#blackish OR blackish lang:en)')
query2
'%28%23blackish%20OR%20blackish%20lang%3Aen%29'
my_api_url3 = endpoint_url2 + '?query={}'.format(query2)
my_api_url3
'https://api.twitter.com/2/tweets/search/recent?query=%28%23blackish%20OR%20blackish%20lang%3Aen%29'
tweet_fields2 = 'public_metrics,created_at,author_id,lang'
my_api_url2 = endpoint_url2 + '?query={}&tweet.fields={}'.format(query2, tweet_fields2)
my_api_url2
'https://api.twitter.com/2/tweets/search/recent?query=%28%23blackish%20OR%20blackish%20lang%3Aen%29&tweet.fields=public_metrics,created_at,author_id,lang'
expansions2 = 'author_id'
url2 = endpoint_url2 + '?query={}&max_results=10&tweet.fields{}'.format(query2, tweet_fields2)
response2 = requests.request("GET", url, headers = header)
url_expansions2 = endpoint_url2 + '?query={}&max_results=100&tweet.fields={}&expansions={}&user.fields={}'.format(query2, tweet_fields2, expansions2, 'username')
url_expansions2
'https://api.twitter.com/2/tweets/search/recent?query=%28%23blackish%20OR%20blackish%20lang%3Aen%29&max_results=100&tweet.fields=public_metrics,created_at,author_id,lang&expansions=author_id&user.fields=username'
blackish_response = requests.request("GET", url_expansions2, headers = header)
blackish_response.text
blackish_dict = json.loads(blackish_response.text)
blackish_dict.keys()
dict_keys(['data', 'includes', 'meta'])
blackish_df = pd.DataFrame(blackish_dict['data'])
blackish_df
id | public_metrics | text | created_at | author_id | lang | |
---|---|---|---|---|---|---|
0 | 1470539115921870855 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | blackish-grey ossel hitch | 2021-12-13T23:40:21.000Z | 796444880675438592 | en |
1 | 1470539014117765122 | {'retweet_count': 2, 'reply_count': 0, 'like_c... | RT @IamW4TC: Congratulations @IssaRae and @Tra... | 2021-12-13T23:39:56.000Z | 957821109444972545 | en |
2 | 1470527694148452358 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | And I really feel like they nominate Blackish ... | 2021-12-13T22:54:58.000Z | 26905493 | en |
3 | 1470527263489802240 | {'retweet_count': 0, 'reply_count': 1, 'like_c... | @Chemicalboyy Banshee\nVikings\nElites\nBlacki... | 2021-12-13T22:53:15.000Z | 1080035996027273217 | en |
4 | 1470520951964065796 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | 3) Mejor actor principal en comedia #GlobosDeO... | 2021-12-13T22:28:10.000Z | 1863836348 | es |
... | ... | ... | ... | ... | ... | ... |
95 | 1470204422399610884 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | Blackish be so inspiring like I wanna be succe... | 2021-12-13T01:30:24.000Z | 2398245080 | en |
96 | 1470203673200537601 | {'retweet_count': 2, 'reply_count': 0, 'like_c... | RT @johnrfisher1: Been kinda snowbound of late... | 2021-12-13T01:27:25.000Z | 1463583253714059269 | en |
97 | 1470202539828289537 | {'retweet_count': 3, 'reply_count': 0, 'like_c... | RT @ChastyRaunchSub: A third item from the AC ... | 2021-12-13T01:22:55.000Z | 1328506635397046273 | en |
98 | 1470193554962145282 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | The tv show Blackish is racist, unless there i... | 2021-12-13T00:47:13.000Z | 1319384189968461826 | en |
99 | 1470192856186990595 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | Blackish is ending? A show I swore I'd never w... | 2021-12-13T00:44:26.000Z | 242948954 | en |
100 rows × 6 columns
blackish_dict['meta']
{'newest_id': '1470539115921870855', 'oldest_id': '1470192856186990595', 'result_count': 100, 'next_token': 'b26v89c19zqg8o3fpe15c4662ywe1xnmve7tuh31tur25'}
my_api_url4 = my_api_url2 + '&next_token={}&max_results=100'.format(blackish_dict['meta']['next_token'])
my_api_url4
'https://api.twitter.com/2/tweets/search/recent?query=%28%23blackish%20OR%20blackish%20lang%3Aen%29&tweet.fields=public_metrics,created_at,author_id,lang&next_token=b26v89c19zqg8o3fpe15c4662ywe1xnmve7tuh31tur25&max_results=100'
blackish2_response = requests.request("GET", my_api_url4, headers = header)
blackish2_response.text
blackish_df = blackish_df.append(pd.DataFrame(json.loads(blackish2_response.text)['data']), ignore_index= True)
blackish_df
id | public_metrics | text | created_at | author_id | lang | |
---|---|---|---|---|---|---|
0 | 1470539115921870855 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | blackish-grey ossel hitch | 2021-12-13T23:40:21.000Z | 796444880675438592 | en |
1 | 1470539014117765122 | {'retweet_count': 2, 'reply_count': 0, 'like_c... | RT @IamW4TC: Congratulations @IssaRae and @Tra... | 2021-12-13T23:39:56.000Z | 957821109444972545 | en |
2 | 1470527694148452358 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | And I really feel like they nominate Blackish ... | 2021-12-13T22:54:58.000Z | 26905493 | en |
3 | 1470527263489802240 | {'retweet_count': 0, 'reply_count': 1, 'like_c... | @Chemicalboyy Banshee\nVikings\nElites\nBlacki... | 2021-12-13T22:53:15.000Z | 1080035996027273217 | en |
4 | 1470520951964065796 | {'retweet_count': 0, 'reply_count': 0, 'like_c... | 3) Mejor actor principal en comedia #GlobosDeO... | 2021-12-13T22:28:10.000Z | 1863836348 | es |
... | ... | ... | ... | ... | ... | ... |
195 | 1469655354086760451 | {'retweet_count': 1, 'reply_count': 0, 'like_c... | That girl from Blackish... https://t.co/JNoPuq... | 2021-12-11T13:08:35.000Z | 984971006245769216 | en |
196 | 1469651840207884288 | {'retweet_count': 15, 'reply_count': 0, 'like_... | RT @_myvpl_: @NCares17 @jessicawhitIly Yh it w... | 2021-12-11T12:54:38.000Z | 387132443 | en |
197 | 1469648138906763267 | {'retweet_count': 15, 'reply_count': 0, 'like_... | RT @_myvpl_: @NCares17 @jessicawhitIly Yh it w... | 2021-12-11T12:39:55.000Z | 452280828 | en |
198 | 1469624514417991681 | {'retweet_count': 195, 'reply_count': 0, 'like... | RT @VasuBharat09: It's a ray.\nLight and warmt... | 2021-12-11T11:06:03.000Z | 813312379509506049 | en |
199 | 1469622812642091010 | {'retweet_count': 6, 'reply_count': 0, 'like_c... | RT @TvBoxSA: اعلنت @ABCNetwork تجديد المسلسل ا... | 2021-12-11T10:59:17.000Z | 1067634457 | ar |
200 rows × 6 columns
blackish_df.to_csv('blackishAnalysis.csv')
blackish = pd.read_csv('blackishAnalysis.csv', encoding = 'utf-8')
blackish.keys()
blackish['lang'].astype('category')
blackish_posts = blackish[(blackish['lang']) == 'en']
blackish_posts['lang'].astype('category')
blackish_posts['text']
0 blackish-grey ossel hitch 1 RT @IamW4TC: Congratulations @IssaRae and @Tra... 2 And I really feel like they nominate Blackish ... 3 @Chemicalboyy Banshee\nVikings\nElites\nBlacki... 5 2) Mejor actriz principal en comedia #GlobosDe... ... 194 RT @hetros2022: That girl from Blackish... 195 That girl from Blackish... https://t.co/JNoPuq... 196 RT @_myvpl_: @NCares17 @jessicawhitIly Yh it w... 197 RT @_myvpl_: @NCares17 @jessicawhitIly Yh it w... 198 RT @VasuBharat09: It's a ray.\nLight and warmt... Name: text, Length: 193, dtype: object
blackish_posts['text'].iloc[0]
blackish_lowercase = [x.lower() for x in blackish_posts['text']]
blackish_lowercase[0]
blackish_analyze= SentimentIntensityAnalyzer()
print(blackish_analyze.polarity_scores(blackish_posts['text'].iloc[0]))
{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}
blackish_senti = blackish_analyze.polarity_scores(blackish_posts['text'].iloc[0])
blackish_senti['neg']
0.0
all_bsenti = [blackish_analyze.polarity_scores(x) for x in blackish_posts['text']]
all_bsenti
[{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.69, 'pos': 0.31, 'compound': 0.8442}, {'neg': 0.207, 'neu': 0.692, 'pos': 0.101, 'compound': -0.4441}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.102, 'neu': 0.8, 'pos': 0.098, 'compound': -0.0258}, {'neg': 0.0, 'neu': 0.633, 'pos': 0.367, 'compound': 0.4404}, {'neg': 0.0, 'neu': 0.873, 'pos': 0.127, 'compound': 0.7506}, {'neg': 0.0, 'neu': 0.344, 'pos': 0.656, 'compound': 0.9855}, {'neg': 0.0, 'neu': 0.676, 'pos': 0.324, 'compound': 0.8016}, {'neg': 0.099, 'neu': 0.734, 'pos': 0.166, 'compound': 0.3155}, {'neg': 0.281, 'neu': 0.514, 'pos': 0.206, 'compound': -0.7136}, {'neg': 0.153, 'neu': 0.625, 'pos': 0.222, 'compound': 0.25}, {'neg': 0.06, 'neu': 0.94, 'pos': 0.0, 'compound': -0.3736}, {'neg': 0.096, 'neu': 0.814, 'pos': 0.09, 'compound': -0.0931}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.216, 'neu': 0.607, 'pos': 0.177, 'compound': -0.0711}, {'neg': 0.024, 'neu': 0.726, 'pos': 0.25, 'compound': 0.9459}, {'neg': 0.041, 'neu': 0.865, 'pos': 0.094, 'compound': 0.2846}, {'neg': 0.0, 'neu': 0.902, 'pos': 0.098, 'compound': 0.0772}, {'neg': 0.282, 'neu': 0.718, 'pos': 0.0, 'compound': -0.6249}, {'neg': 0.0, 'neu': 0.69, 'pos': 0.31, 'compound': 0.8442}, {'neg': 0.0, 'neu': 0.353, 'pos': 0.647, 'compound': 0.6705}, {'neg': 0.034, 'neu': 0.966, 'pos': 0.0, 'compound': -0.0516}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.131, 'neu': 0.802, 'pos': 0.067, 'compound': -0.5994}, {'neg': 0.0, 'neu': 0.913, 'pos': 0.087, 'compound': 0.4501}, {'neg': 0.09, 'neu': 0.581, 'pos': 0.328, 'compound': 0.8974}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.0, 'neu': 0.508, 'pos': 0.492, 'compound': 0.7003}, {'neg': 0.0, 'neu': 0.947, 'pos': 0.053, 'compound': 0.34}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.822, 'pos': 0.178, 'compound': 0.3818}, {'neg': 0.0, 'neu': 0.375, 'pos': 0.625, 'compound': 0.875}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.245, 'neu': 0.682, 'pos': 0.073, 'compound': -0.8902}, {'neg': 0.081, 'neu': 0.605, 'pos': 0.314, 'compound': 0.8439}, {'neg': 0.0, 'neu': 0.525, 'pos': 0.475, 'compound': 0.9517}, {'neg': 0.0, 'neu': 0.624, 'pos': 0.376, 'compound': 0.9524}, {'neg': 0.0, 'neu': 0.691, 'pos': 0.309, 'compound': 0.7399}, {'neg': 0.0, 'neu': 0.857, 'pos': 0.143, 'compound': 0.3612}, {'neg': 0.29, 'neu': 0.588, 'pos': 0.122, 'compound': -0.6124}, {'neg': 0.0, 'neu': 0.82, 'pos': 0.18, 'compound': 0.5106}, {'neg': 0.0, 'neu': 0.712, 'pos': 0.288, 'compound': 0.7708}, {'neg': 0.138, 'neu': 0.623, 'pos': 0.239, 'compound': 0.5046}, {'neg': 0.056, 'neu': 0.944, 'pos': 0.0, 'compound': -0.1655}, {'neg': 0.231, 'neu': 0.769, 'pos': 0.0, 'compound': -0.4019}, {'neg': 0.088, 'neu': 0.912, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.0, 'neu': 0.753, 'pos': 0.247, 'compound': 0.5574}, {'neg': 0.1, 'neu': 0.588, 'pos': 0.311, 'compound': 0.7269}, {'neg': 0.208, 'neu': 0.708, 'pos': 0.085, 'compound': -0.3541}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.211, 'neu': 0.789, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.255, 'neu': 0.745, 'pos': 0.0, 'compound': -0.34}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.543, 'pos': 0.457, 'compound': 0.6369}, {'neg': 0.0, 'neu': 0.679, 'pos': 0.321, 'compound': 0.6369}, {'neg': 0.203, 'neu': 0.482, 'pos': 0.315, 'compound': 0.7998}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.904, 'pos': 0.096, 'compound': 0.3716}, {'neg': 0.048, 'neu': 0.783, 'pos': 0.169, 'compound': 0.7156}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.385, 'neu': 0.615, 'pos': 0.0, 'compound': -0.3612}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.891, 'pos': 0.109, 'compound': 0.4019}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.127, 'neu': 0.574, 'pos': 0.299, 'compound': 0.4939}, {'neg': 0.196, 'neu': 0.804, 'pos': 0.0, 'compound': -0.296}, {'neg': 0.0, 'neu': 0.779, 'pos': 0.221, 'compound': 0.6597}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.832, 'pos': 0.168, 'compound': 0.872}, {'neg': 0.0, 'neu': 0.519, 'pos': 0.481, 'compound': 0.8805}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.909, 'pos': 0.091, 'compound': 0.2732}, {'neg': 0.248, 'neu': 0.752, 'pos': 0.0, 'compound': -0.6476}, {'neg': 0.0, 'neu': 0.813, 'pos': 0.187, 'compound': 0.5704}, {'neg': 0.0, 'neu': 0.429, 'pos': 0.571, 'compound': 0.8402}, {'neg': 0.0, 'neu': 0.909, 'pos': 0.091, 'compound': 0.2732}, {'neg': 0.04, 'neu': 0.907, 'pos': 0.053, 'compound': 0.1779}, {'neg': 0.068, 'neu': 0.738, 'pos': 0.194, 'compound': 0.7184}, {'neg': 0.0, 'neu': 0.905, 'pos': 0.095, 'compound': 0.4767}, {'neg': 0.0, 'neu': 0.909, 'pos': 0.091, 'compound': 0.2732}, {'neg': 0.231, 'neu': 0.769, 'pos': 0.0, 'compound': -0.7518}, {'neg': 0.0, 'neu': 0.933, 'pos': 0.067, 'compound': 0.3182}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.085, 'neu': 0.813, 'pos': 0.102, 'compound': -0.2846}, {'neg': 0.0, 'neu': 0.854, 'pos': 0.146, 'compound': 0.627}, {'neg': 0.0, 'neu': 0.852, 'pos': 0.148, 'compound': 0.8388}, {'neg': 0.127, 'neu': 0.728, 'pos': 0.144, 'compound': 0.34}, {'neg': 0.0, 'neu': 0.853, 'pos': 0.147, 'compound': 0.6369}, {'neg': 0.0, 'neu': 0.911, 'pos': 0.089, 'compound': 0.5106}, {'neg': 0.0, 'neu': 0.798, 'pos': 0.202, 'compound': 0.6486}, {'neg': 0.0, 'neu': 0.843, 'pos': 0.157, 'compound': 0.7515}, {'neg': 0.0, 'neu': 0.924, 'pos': 0.076, 'compound': 0.34}, {'neg': 0.03, 'neu': 0.913, 'pos': 0.057, 'compound': 0.0772}, {'neg': 0.0, 'neu': 0.798, 'pos': 0.202, 'compound': 0.5106}, {'neg': 0.0, 'neu': 0.769, 'pos': 0.231, 'compound': 0.5106}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.816, 'pos': 0.184, 'compound': 0.8399}, {'neg': 0.215, 'neu': 0.545, 'pos': 0.24, 'compound': 0.1436}, {'neg': 0.0, 'neu': 0.779, 'pos': 0.221, 'compound': 0.6588}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.444, 'pos': 0.556, 'compound': 0.3612}, {'neg': 0.0, 'neu': 0.621, 'pos': 0.379, 'compound': 0.6705}, {'neg': 0.0, 'neu': 0.899, 'pos': 0.101, 'compound': 0.4003}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.444, 'pos': 0.556, 'compound': 0.3612}, {'neg': 0.0, 'neu': 0.445, 'pos': 0.555, 'compound': 0.5777}, {'neg': 0.0, 'neu': 0.535, 'pos': 0.465, 'compound': 0.787}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.169, 'neu': 0.535, 'pos': 0.296, 'compound': 0.908}, {'neg': 0.086, 'neu': 0.624, 'pos': 0.289, 'compound': 0.7063}, {'neg': 0.092, 'neu': 0.596, 'pos': 0.311, 'compound': 0.7063}, {'neg': 0.028, 'neu': 0.866, 'pos': 0.107, 'compound': 0.7622}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.338, 'neu': 0.662, 'pos': 0.0, 'compound': -0.5574}, {'neg': 0.351, 'neu': 0.649, 'pos': 0.0, 'compound': -0.4019}, {'neg': 0.0, 'neu': 0.633, 'pos': 0.367, 'compound': 0.4404}, {'neg': 0.0, 'neu': 0.643, 'pos': 0.357, 'compound': 0.7273}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.112, 'neu': 0.77, 'pos': 0.119, 'compound': -0.2865}, {'neg': 0.0, 'neu': 0.815, 'pos': 0.185, 'compound': 0.3612}, {'neg': 0.0, 'neu': 0.828, 'pos': 0.172, 'compound': 0.3612}, {'neg': 0.209, 'neu': 0.791, 'pos': 0.0, 'compound': -0.5719}, {'neg': 0.155, 'neu': 0.845, 'pos': 0.0, 'compound': -0.6486}, {'neg': 0.0, 'neu': 0.867, 'pos': 0.133, 'compound': 0.3818}, {'neg': 0.086, 'neu': 0.667, 'pos': 0.247, 'compound': 0.7579}, {'neg': 0.201, 'neu': 0.625, 'pos': 0.174, 'compound': -0.1531}, {'neg': 0.0, 'neu': 0.767, 'pos': 0.233, 'compound': 0.743}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.204, 'neu': 0.389, 'pos': 0.408, 'compound': 0.969}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.947, 'pos': 0.053, 'compound': 0.4173}, {'neg': 0.08, 'neu': 0.92, 'pos': 0.0, 'compound': -0.0772}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.445, 'neu': 0.555, 'pos': 0.0, 'compound': -0.9436}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.053, 'neu': 0.825, 'pos': 0.121, 'compound': 0.4215}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.741, 'pos': 0.259, 'compound': 0.4199}, {'neg': 0.0, 'neu': 0.905, 'pos': 0.095, 'compound': 0.2975}, {'neg': 0.136, 'neu': 0.864, 'pos': 0.0, 'compound': -0.296}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.262, 'neu': 0.738, 'pos': 0.0, 'compound': -0.4939}, {'neg': 0.0, 'neu': 0.588, 'pos': 0.412, 'compound': 0.8885}, {'neg': 0.0, 'neu': 0.798, 'pos': 0.202, 'compound': 0.6486}, {'neg': 0.0, 'neu': 0.798, 'pos': 0.202, 'compound': 0.6486}, {'neg': 0.187, 'neu': 0.813, 'pos': 0.0, 'compound': -0.6486}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.187, 'neu': 0.813, 'pos': 0.0, 'compound': -0.6486}, {'neg': 0.052, 'neu': 0.708, 'pos': 0.24, 'compound': 0.7845}, {'neg': 0.0, 'neu': 0.496, 'pos': 0.504, 'compound': 0.7269}, {'neg': 0.157, 'neu': 0.843, 'pos': 0.0, 'compound': -0.68}, {'neg': 0.0, 'neu': 0.8, 'pos': 0.2, 'compound': 0.3612}, {'neg': 0.029, 'neu': 0.912, 'pos': 0.059, 'compound': 0.3519}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.385, 'neu': 0.615, 'pos': 0.0, 'compound': -0.3612}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}, {'neg': 0.0, 'neu': 0.798, 'pos': 0.202, 'compound': 0.6486}, {'neg': 0.0, 'neu': 0.798, 'pos': 0.202, 'compound': 0.6486}, {'neg': 0.087, 'neu': 0.758, 'pos': 0.155, 'compound': 0.3182}]
blacksents_df = pd.DataFrame(all_bsenti)
Here I created a dataframe for the blackish sentiment scores. This allows me to better analyze the data and to visualize it. I then made a histogram with the positive sentiments and it visualizes it for me/
blacksents_df
neg | neu | pos | compound | |
---|---|---|---|---|
0 | 0.000 | 1.000 | 0.000 | 0.0000 |
1 | 0.000 | 0.690 | 0.310 | 0.8442 |
2 | 0.207 | 0.692 | 0.101 | -0.4441 |
3 | 0.000 | 1.000 | 0.000 | 0.0000 |
4 | 0.102 | 0.800 | 0.098 | -0.0258 |
... | ... | ... | ... | ... |
188 | 0.000 | 1.000 | 0.000 | 0.0000 |
189 | 0.000 | 1.000 | 0.000 | 0.0000 |
190 | 0.000 | 0.798 | 0.202 | 0.6486 |
191 | 0.000 | 0.798 | 0.202 | 0.6486 |
192 | 0.087 | 0.758 | 0.155 | 0.3182 |
193 rows × 4 columns
blacksents_df['pos'].describe()
count 193.000000 mean 0.127321 std 0.160015 min 0.000000 25% 0.000000 50% 0.076000 75% 0.202000 max 0.656000 Name: pos, dtype: float64
blacksents_df['neg'].describe()
count 193.00000 mean 0.06943 std 0.10193 min 0.00000 25% 0.00000 50% 0.00000 75% 0.13600 max 0.44500 Name: neg, dtype: float64
fig2, ax2 = plt.subplots()
ax2.hist(blacksents_df['pos'])
plt.show()
Here I have the blackish histograms with each sentiment in a histogram. Interpreting this data I can look at it and say that overall the tweets seem to be neutral. Next I will compare the two shows by each sentiment.
fig3, ax3 = plt.subplots(1, 4, figsize = (10,5))
ax3[0].hist(blacksents_df['neg'], bins = 8, color = 'red')
ax3[1].hist(blacksents_df['pos'], bins = 8, color = 'blue')
ax3[2].hist(blacksents_df['neu'], bins = 8, color = 'green')
ax3[3].hist(blacksents_df['compound'], bins = 8, color = 'yellow')
ax3[0].set_xlabel('Negative')
ax3[1].set_xlabel('Positive')
ax3[2].set_xlabel('Neutral')
ax3[3].set_xlabel('Compound')
fig3.suptitle("black-ish sentiments")
plt.show()
Here I combined the two shows sentiment scores which allow me to look at each shows sentiment scores next to each other. The first histogram I am just putting the sentiment scores next to each other.
fig5, ax5 = plt.subplots(2, 4, figsize = (10, 8))
ax5[0,0].hist(sents_df['neg'], bins = 10, color = 'red')
ax5[0,1].hist(sents_df['pos'], bins = 10, color = 'blue')
ax5[0,2].hist(sents_df['neu'], bins = 10, color = 'green')
ax5[0,3].hist(sents_df['compound'], bins = 10, color = 'yellow')
ax5[0,0].set_xlabel("Negative")
ax5[0,1].set_xlabel("Positive")
ax5[0,2].set_xlabel("Neutral")
ax5[0,3].set_xlabel("Compound")
ax5[0,0].set_ylabel("Ted Lasso")
ax5[0,0].set_ylim(0,140)
ax5[0,1].set_ylim(0,140)
ax5[0,2].set_ylim(0,140)
ax5[0,3].set_ylim(0,140)
ax5[1,0].hist(blacksents_df['neg'], bins = 10, color = 'red')
ax5[1,1].hist(blacksents_df['pos'], bins = 10, color = 'blue')
ax5[1,2].hist(blacksents_df['neu'], bins = 10, color = 'green')
ax5[1,3].hist(blacksents_df['compound'], bins = 10, color = 'yellow')
ax5[1,0].set_xlabel('Negative')
ax5[1,1].set_xlabel('Positive')
ax5[1,2].set_xlabel('Neutral')
ax5[1,3].set_xlabel('Compound')
ax5[1,0].set_ylabel("black-ish")
ax5[1,0].set_ylim(0,140)
ax5[1,1].set_ylim(0,140)
ax5[1,2].set_ylim(0,140)
ax5[1,3].set_ylim(0,140)
fig5.suptitle("Ted Lasso Sentiments vs. black-ish sentiments")
plt.show()
As for these histograms I overlapped the two shows data in order to get a more clear comparison of them. This is a better visualization because it can support my hypothesis. Ted Lasso is red and blackish is blue.
fig6, ax6 = plt.subplots(1, 4, figsize = (10, 8))
ax6[0].hist(sents_df['neg'], bins = 10, color = 'red', alpha = .5)
ax6[0].hist(blacksents_df['neg'], bins = 10, color = 'blue', alpha = .5)
ax6[1].hist(sents_df['pos'], bins = 10, color = 'red', alpha = .5)
ax6[1].hist(blacksents_df['pos'], bins = 10, color = 'blue', alpha = .5)
ax6[2].hist(sents_df['neu'], bins = 10, color = 'red', alpha = .5)
ax6[2].hist(blacksents_df['neu'], bins = 10, color = 'blue', alpha = .5)
ax6[3].hist(sents_df['compound'], bins = 10, color = 'red', alpha = .5)
ax6[3].hist(blacksents_df['compound'], bins = 10, color = 'blue', alpha = .5)
ax6[0].set_ylim(0,180)
ax6[1].set_ylim(0,180)
ax6[2].set_ylim(0,180)
ax6[3].set_ylim(0,180)
ax6[0].set_xlim(0,1)
ax6[1].set_xlim(0,1)
ax6[2].set_xlim(0,1)
ax6[3].set_xlim(0,1)
ax6[0].set_xlabel("Negative")
ax6[1].set_xlabel("Positive")
ax6[2].set_xlabel("Neutral")
ax6[3].set_xlabel("Compound")
fig6.suptitle("Ted Lasso Sentiments vs. black-ish Sentiments")
plt.show()
Overall, I believe that my hypothesis was right to some extent. Looking at the compound graph Ted Lasso seems to have a higher positive score than blackish. Even if you look at the graph above that one, you can see that Ted Lasso is more positive than negative. This has some limitations to it because I am just guessing by looking at the graphs, that Ted Lasso is more positive. If someone else were to look at the graphs they might say that blackish is more positive. Right now it is pretty bias to say that Ted Lasso is more positive. Another limitation I ran into was that I only have 200 Tweets for each show, I think collecting more Tweets would've help for my hypothesis. In the future I hope to get approved for the academic account for the Twitter API so that I can get Tweets from before the Emmy winners were announced.