Hacker News Posts

In this project we will be looking at two different types of posts: Ask HN and Show HN.

Ask HN is where users submit posts to ask the Hacker NEws community questions.

Show HN is where users submits posts to showcase Hacker NEws a project, product or something interesting that was found.

In this project we will be specifically looking at:

  • which type of post receives more comments on average
  • are posts created at a certain time receving more comments on average.

It is important to note that the dataset being worked with has been significantly reduced from 300,000 rows to 20,000 due to removal of submittions without comments and randomly sampling the remaining submissions.

In [1]:
import csv
opened_file = open('hacker_news.csv')
hn = list(csv.reader(opened_file))
hn[:5]
Out[1]:
[['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at'],
 ['12224879',
  'Interactive Dynamic Video',
  'http://www.interactivedynamicvideo.com/',
  '386',
  '52',
  'ne0phyte',
  '8/4/2016 11:52'],
 ['10975351',
  'How to Use Open Source and Shut the Fuck Up at the Same Time',
  'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/',
  '39',
  '10',
  'josep2',
  '1/26/2016 19:30'],
 ['11964716',
  "Florida DJs May Face Felony for April Fools' Water Joke",
  'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/',
  '2',
  '1',
  'vezycash',
  '6/23/2016 22:20'],
 ['11919867',
  'Technology ventures: From Idea to Enterprise',
  'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429',
  '3',
  '1',
  'hswarna',
  '6/17/2016 0:01']]
In [2]:
headers = hn[0]
hn = hn[1:]
print(headers)
print(hn[:5])
['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at']
[['12224879', 'Interactive Dynamic Video', 'http://www.interactivedynamicvideo.com/', '386', '52', 'ne0phyte', '8/4/2016 11:52'], ['10975351', 'How to Use Open Source and Shut the Fuck Up at the Same Time', 'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/', '39', '10', 'josep2', '1/26/2016 19:30'], ['11964716', "Florida DJs May Face Felony for April Fools' Water Joke", 'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/', '2', '1', 'vezycash', '6/23/2016 22:20'], ['11919867', 'Technology ventures: From Idea to Enterprise', 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429', '3', '1', 'hswarna', '6/17/2016 0:01'], ['10301696', 'Note by Note: The Making of Steinway L1037 (2007)', 'http://www.nytimes.com/2007/11/07/movies/07stein.html?_r=0', '8', '2', 'walterbell', '9/30/2015 4:12']]
In [3]:
ask_posts = []
show_posts = []
other_posts = []
for row in hn:
    title = row[1]
    if title.lower().startswith("ask hn"):
        ask_posts.append(row)
    elif title.lower().startswith("show hn"):
        show_posts.append(row)
    else:
        other_posts.append(row)
        
print(len(ask_posts))
print(len(show_posts))
print(len(other_posts))
1744
1162
17194
In [4]:
total_ask_comments = 0
for row in ask_posts:
    total_ask_comments += int(row[4])

avg_ask_comments = total_ask_comments / len(ask_posts)
print(avg_ask_comments)

total_show_comments = 0
for row in show_posts:
    total_show_comments += int(row[4])

avg_show_comments = total_show_comments / len(show_posts)
print(avg_show_comments)
14.038417431192661
10.31669535283993

On average, ask posts receive approximately 14 comments whereas show posts receive 10 comments. As ask posts receive more comments, we will focus on this going forward.

In [5]:
import datetime as dt

result_list = []

for row in ask_posts:
    result_list.append([row[6], int(row[4])])

comments_by_hour = {}
counts_by_hour = {}
date_format = "%m/%d/%Y %H:%M"

for row in result_list:
    date = row[0]
    comment = row[1]
    time = dt.datetime.strptime(date, date_format).strftime("%H")
    if time not in counts_by_hour:
        comments_by_hour[time] = comment
        counts_by_hour[time] = 1
    else:
        comments_by_hour[time] += comment
        counts_by_hour[time] += 1

comments_by_hour
Out[5]:
{'00': 447,
 '01': 683,
 '02': 1381,
 '03': 421,
 '04': 337,
 '05': 464,
 '06': 397,
 '07': 267,
 '08': 492,
 '09': 251,
 '10': 793,
 '11': 641,
 '12': 687,
 '13': 1253,
 '14': 1416,
 '15': 4477,
 '16': 1814,
 '17': 1146,
 '18': 1439,
 '19': 1188,
 '20': 1722,
 '21': 1745,
 '22': 479,
 '23': 543}
In [6]:
avg_by_hour = []
for hour in comments_by_hour:
    
    avg_by_hour.append([hour, comments_by_hour[hour] / counts_by_hour[hour]])

avg_by_hour
Out[6]:
[['21', 16.009174311926607],
 ['12', 9.41095890410959],
 ['03', 7.796296296296297],
 ['23', 7.985294117647059],
 ['20', 21.525],
 ['16', 16.796296296296298],
 ['05', 10.08695652173913],
 ['07', 7.852941176470588],
 ['10', 13.440677966101696],
 ['01', 11.383333333333333],
 ['15', 38.5948275862069],
 ['17', 11.46],
 ['22', 6.746478873239437],
 ['00', 8.127272727272727],
 ['02', 23.810344827586206],
 ['11', 11.051724137931034],
 ['08', 10.25],
 ['04', 7.170212765957447],
 ['19', 10.8],
 ['09', 5.5777777777777775],
 ['13', 14.741176470588234],
 ['14', 13.233644859813085],
 ['06', 9.022727272727273],
 ['18', 13.20183486238532]]
In [7]:
swap_avg_by_hour = []
for row in avg_by_hour:
    swap_avg_by_hour.append([row[1], row[0]])

print(swap_avg_by_hour)

sorted_swap = sorted(swap_avg_by_hour, reverse = True)

sorted_swap
[[16.009174311926607, '21'], [9.41095890410959, '12'], [7.796296296296297, '03'], [7.985294117647059, '23'], [21.525, '20'], [16.796296296296298, '16'], [10.08695652173913, '05'], [7.852941176470588, '07'], [13.440677966101696, '10'], [11.383333333333333, '01'], [38.5948275862069, '15'], [11.46, '17'], [6.746478873239437, '22'], [8.127272727272727, '00'], [23.810344827586206, '02'], [11.051724137931034, '11'], [10.25, '08'], [7.170212765957447, '04'], [10.8, '19'], [5.5777777777777775, '09'], [14.741176470588234, '13'], [13.233644859813085, '14'], [9.022727272727273, '06'], [13.20183486238532, '18']]
Out[7]:
[[38.5948275862069, '15'],
 [23.810344827586206, '02'],
 [21.525, '20'],
 [16.796296296296298, '16'],
 [16.009174311926607, '21'],
 [14.741176470588234, '13'],
 [13.440677966101696, '10'],
 [13.233644859813085, '14'],
 [13.20183486238532, '18'],
 [11.46, '17'],
 [11.383333333333333, '01'],
 [11.051724137931034, '11'],
 [10.8, '19'],
 [10.25, '08'],
 [10.08695652173913, '05'],
 [9.41095890410959, '12'],
 [9.022727272727273, '06'],
 [8.127272727272727, '00'],
 [7.985294117647059, '23'],
 [7.852941176470588, '07'],
 [7.796296296296297, '03'],
 [7.170212765957447, '04'],
 [6.746478873239437, '22'],
 [5.5777777777777775, '09']]
In [8]:
print("Top 5 Hours for Ask Posts Comments")
for avg, hour in sorted_swap[:5]:
    print("{}: {:.2f} average comments per post".format(dt.datetime.strptime(hour, "%H").strftime("%H:%M"), avg))
Top 5 Hours for Ask Posts Comments
15:00: 38.59 average comments per post
02:00: 23.81 average comments per post
20:00: 21.52 average comments per post
16:00: 16.80 average comments per post
21:00: 16.01 average comments per post

Conclusion

In this project, we looked at both ask and show posts in order to find which type of post received the most comments on average. From this, we wanted to find, what time would receive the most comments on average.

From our analusis, we see that an ask post should be created around 15:00 - 16:00 to maximize the amount of comments received.

It must be noted this data excluded the analysis of posts with 0 comments. It is accurate to state that the posts that had received comments, ask posts received more comments on average and those posts created between 15:00 - 16:00 received the most comments on average

In [ ]: