Hacker News is a site started by the startup incubator Y Combinator, where user-submitted stories (known as "posts") are voted and commented upon, similar to reddit. Hacker News is extremely popular in technology and startup circles, and posts that make it to the top of Hacker News' listings can get hundreds of thousands of visitors as a result.
In this project, we'll work with a data set of submissions to popular technology site Hacker News.
The Data set comprises of 30,000 Rows and 7 Columns. Each column description is given below for reference.
id: The unique identifier from Hacker News for the post\ title: The title of the post\ url: The URL that the posts links to, if it the post has a URL\ num_points: The number of points the post acquired, calculated as the total number of upvotes minus the total number of downvotes\ num_comments: The number of comments that were made on the post\ author: The username of the person who submitted the post\ created_at: The date and time at which the post was submitted
from csv import reader
hn_opened_file = open('HN_posts_year_to_Sep_26_2016.csv', encoding='utf8')
hn_csv_file = reader(hn_opened_file)
hn = list(hn_csv_file)
hn[:5]
headers = hn[:1] # column descriptions
hn = hn[1:] # remove the row that contains column descriptions
print(headers)
hn[:5]
Create 3 different lists named ask_posts, show_posts, and other_posts
ask_posts = []
show_posts = []
other_posts = []
for row in hn:
title = row[1]
title = title.lower()
if title.startswith('ask hn'):
ask_posts.append(row)
elif title.startswith('show hn'):
show_posts.append(row)
else:
other_posts.append(row)
print('Ask NH posts count : ', len(ask_posts))
print('Show NH posts count : ', len(show_posts))
print('Other posts count : ', len(other_posts))
total_ask_comments = 0
total_num_ask_comments = 0
for row in ask_posts:
total_ask_comments += int(row[4])
total_num_ask_comments += 1
avg_ask_comments = total_ask_comments/total_num_ask_comments
print("Average 'Ask NH' posts comments :", avg_ask_comments)
total_show_comments = 0
total_num_show_comments = 0
for row in show_posts:
total_show_comments += int(row[4])
total_num_show_comments += 1
avg_show_comments = total_show_comments/total_num_show_comments
print("Average 'SHow NH' posts comments :", avg_show_comments)
Above, we calculated the average comments received by the posts starting with title 'Ask NH' and 'Show NH'. We found Ask NH posts receive higher number of comments on average.
Since we found that 'Ask NH' received more comments in average, we'll analyze these posts further. To determine the time of posts that receive more comments, we'll perform below steps.
import datetime as dt
result_list = []
for row in ask_posts:
post_create_time = row[6] # post creation time
post_comments = int(row[4]) # number of comments received by the post
result_list.append([post_create_time, post_comments])
counts_by_hour = {}
comments_by_hour = {}
for row in result_list:
time = row[0]
num_comments = row[1]
dt_obj = dt.datetime.strptime(time, "%m/%d/%Y %H:%M")
hour = dt.datetime.strftime(dt_obj, "%H")
if hour not in counts_by_hour:
counts_by_hour[hour] = 1
comments_by_hour[hour] = num_comments
else:
counts_by_hour[hour] += 1
comments_by_hour[hour] += num_comments
comments_by_hour
avg_by_hour = []
for hour in counts_by_hour:
avg_by_hour.append([hour, (comments_by_hour[hour]/counts_by_hour[hour])])
avg_by_hour
swap_avg_by_hour = []
for row in avg_by_hour:
hour = row[0]
avg_comments = row[1]
swap_avg_by_hour.append([avg_comments, hour])
swap_avg_by_hour
sorted_swap = sorted(swap_avg_by_hour, reverse=True)
print("Top 5 Hours for Ask Posts Comments")
for avg_cmts, hour in sorted_swap[:5]:
display_str = "{}: {:.2f} average comments per post"
date_object = dt.datetime.strptime(hour, "%H")
time_str = dt.datetime.strftime(date_object, "%H:%M")
display_str = display_str.format(time_str, avg_cmts)
print(display_str)
From the output, posts created at time 15:00 receive highest comments in average. Next, posts created at 13:00 are likely to get more comments. From the data set, time is represented in Eastern Time (ET). I live in INDIA. Let's convert these top 5 entries to IST timezone which is 9 hrs 30 mins ahead of ET.
print("Top 5 Hours for Ask Posts Comments in IST timezone")
for avg_cmts, hour in sorted_swap[:5]:
display_str = "{}: {:.2f} average comments per post"
datetime_object = dt.datetime.strptime(hour, "%H")
time_object = dt.timedelta(hours=9, minutes=30)
ist_time_object = datetime_object + time_object
time_str = dt.datetime.strftime(ist_time_object, "%H:%M")
display_str = display_str.format(time_str, avg_cmts)
print(display_str)
Posts created from INDIA at 12:30 AM are likely to receive highest comments in average.