In this project we will be looking at two different types of posts: Ask HN
and Show HN
.
Ask HN
is where users submit posts to ask the Hacker NEws community questions.
Show HN
is where users submits posts to showcase Hacker NEws a project, product or something interesting that was found.
In this project we will be specifically looking at:
It is important to note that the dataset being worked with has been significantly reduced from 300,000 rows to 20,000 due to removal of submittions without comments and randomly sampling the remaining submissions.
import csv
opened_file = open('hacker_news.csv')
hn = list(csv.reader(opened_file))
hn[:5]
[['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at'], ['12224879', 'Interactive Dynamic Video', 'http://www.interactivedynamicvideo.com/', '386', '52', 'ne0phyte', '8/4/2016 11:52'], ['10975351', 'How to Use Open Source and Shut the Fuck Up at the Same Time', 'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/', '39', '10', 'josep2', '1/26/2016 19:30'], ['11964716', "Florida DJs May Face Felony for April Fools' Water Joke", 'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/', '2', '1', 'vezycash', '6/23/2016 22:20'], ['11919867', 'Technology ventures: From Idea to Enterprise', 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429', '3', '1', 'hswarna', '6/17/2016 0:01']]
headers = hn[0]
hn = hn[1:]
print(headers)
print(hn[:5])
['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at'] [['12224879', 'Interactive Dynamic Video', 'http://www.interactivedynamicvideo.com/', '386', '52', 'ne0phyte', '8/4/2016 11:52'], ['10975351', 'How to Use Open Source and Shut the Fuck Up at the Same Time', 'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/', '39', '10', 'josep2', '1/26/2016 19:30'], ['11964716', "Florida DJs May Face Felony for April Fools' Water Joke", 'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/', '2', '1', 'vezycash', '6/23/2016 22:20'], ['11919867', 'Technology ventures: From Idea to Enterprise', 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429', '3', '1', 'hswarna', '6/17/2016 0:01'], ['10301696', 'Note by Note: The Making of Steinway L1037 (2007)', 'http://www.nytimes.com/2007/11/07/movies/07stein.html?_r=0', '8', '2', 'walterbell', '9/30/2015 4:12']]
ask_posts = []
show_posts = []
other_posts = []
for row in hn:
title = row[1]
if title.lower().startswith("ask hn"):
ask_posts.append(row)
elif title.lower().startswith("show hn"):
show_posts.append(row)
else:
other_posts.append(row)
print(len(ask_posts))
print(len(show_posts))
print(len(other_posts))
1744 1162 17194
total_ask_comments = 0
for row in ask_posts:
total_ask_comments += int(row[4])
avg_ask_comments = total_ask_comments / len(ask_posts)
print(avg_ask_comments)
total_show_comments = 0
for row in show_posts:
total_show_comments += int(row[4])
avg_show_comments = total_show_comments / len(show_posts)
print(avg_show_comments)
14.038417431192661 10.31669535283993
On average, ask posts receive approximately 14 comments whereas show posts receive 10 comments. As ask posts receive more comments, we will focus on this going forward.
import datetime as dt
result_list = []
for row in ask_posts:
result_list.append([row[6], int(row[4])])
comments_by_hour = {}
counts_by_hour = {}
date_format = "%m/%d/%Y %H:%M"
for row in result_list:
date = row[0]
comment = row[1]
time = dt.datetime.strptime(date, date_format).strftime("%H")
if time not in counts_by_hour:
comments_by_hour[time] = comment
counts_by_hour[time] = 1
else:
comments_by_hour[time] += comment
counts_by_hour[time] += 1
comments_by_hour
{'00': 447, '01': 683, '02': 1381, '03': 421, '04': 337, '05': 464, '06': 397, '07': 267, '08': 492, '09': 251, '10': 793, '11': 641, '12': 687, '13': 1253, '14': 1416, '15': 4477, '16': 1814, '17': 1146, '18': 1439, '19': 1188, '20': 1722, '21': 1745, '22': 479, '23': 543}
avg_by_hour = []
for hour in comments_by_hour:
avg_by_hour.append([hour, comments_by_hour[hour] / counts_by_hour[hour]])
avg_by_hour
[['21', 16.009174311926607], ['12', 9.41095890410959], ['03', 7.796296296296297], ['23', 7.985294117647059], ['20', 21.525], ['16', 16.796296296296298], ['05', 10.08695652173913], ['07', 7.852941176470588], ['10', 13.440677966101696], ['01', 11.383333333333333], ['15', 38.5948275862069], ['17', 11.46], ['22', 6.746478873239437], ['00', 8.127272727272727], ['02', 23.810344827586206], ['11', 11.051724137931034], ['08', 10.25], ['04', 7.170212765957447], ['19', 10.8], ['09', 5.5777777777777775], ['13', 14.741176470588234], ['14', 13.233644859813085], ['06', 9.022727272727273], ['18', 13.20183486238532]]
swap_avg_by_hour = []
for row in avg_by_hour:
swap_avg_by_hour.append([row[1], row[0]])
print(swap_avg_by_hour)
sorted_swap = sorted(swap_avg_by_hour, reverse = True)
sorted_swap
[[16.009174311926607, '21'], [9.41095890410959, '12'], [7.796296296296297, '03'], [7.985294117647059, '23'], [21.525, '20'], [16.796296296296298, '16'], [10.08695652173913, '05'], [7.852941176470588, '07'], [13.440677966101696, '10'], [11.383333333333333, '01'], [38.5948275862069, '15'], [11.46, '17'], [6.746478873239437, '22'], [8.127272727272727, '00'], [23.810344827586206, '02'], [11.051724137931034, '11'], [10.25, '08'], [7.170212765957447, '04'], [10.8, '19'], [5.5777777777777775, '09'], [14.741176470588234, '13'], [13.233644859813085, '14'], [9.022727272727273, '06'], [13.20183486238532, '18']]
[[38.5948275862069, '15'], [23.810344827586206, '02'], [21.525, '20'], [16.796296296296298, '16'], [16.009174311926607, '21'], [14.741176470588234, '13'], [13.440677966101696, '10'], [13.233644859813085, '14'], [13.20183486238532, '18'], [11.46, '17'], [11.383333333333333, '01'], [11.051724137931034, '11'], [10.8, '19'], [10.25, '08'], [10.08695652173913, '05'], [9.41095890410959, '12'], [9.022727272727273, '06'], [8.127272727272727, '00'], [7.985294117647059, '23'], [7.852941176470588, '07'], [7.796296296296297, '03'], [7.170212765957447, '04'], [6.746478873239437, '22'], [5.5777777777777775, '09']]
print("Top 5 Hours for Ask Posts Comments")
for avg, hour in sorted_swap[:5]:
print("{}: {:.2f} average comments per post".format(dt.datetime.strptime(hour, "%H").strftime("%H:%M"), avg))
Top 5 Hours for Ask Posts Comments 15:00: 38.59 average comments per post 02:00: 23.81 average comments per post 20:00: 21.52 average comments per post 16:00: 16.80 average comments per post 21:00: 16.01 average comments per post
In this project, we looked at both ask and show posts in order to find which type of post received the most comments on average. From this, we wanted to find, what time would receive the most comments on average.
From our analusis, we see that an ask post should be created around 15:00 - 16:00 to maximize the amount of comments received.
It must be noted this data excluded the analysis of posts with 0 comments. It is accurate to state that the posts that had received comments, ask posts received more comments on average and those posts created between 15:00 - 16:00 received the most comments on average