Guided project no.2 (Hacker News)

We'll compare these two types of posts to determine the following:

  1. Do Ask HN or Show HN receive more comments on average?
  2. Do posts created at a certain time receive more comments on average?
In [1]:
from csv import reader
hn = list(reader(open("hacker_news.csv")))
print(hn[:5])
[['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at'], ['12224879', 'Interactive Dynamic Video', 'http://www.interactivedynamicvideo.com/', '386', '52', 'ne0phyte', '8/4/2016 11:52'], ['10975351', 'How to Use Open Source and Shut the Fuck Up at the Same Time', 'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/', '39', '10', 'josep2', '1/26/2016 19:30'], ['11964716', "Florida DJs May Face Felony for April Fools' Water Joke", 'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/', '2', '1', 'vezycash', '6/23/2016 22:20'], ['11919867', 'Technology ventures: From Idea to Enterprise', 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429', '3', '1', 'hswarna', '6/17/2016 0:01']]
In [2]:
headers = hn[0]
hn = hn[1:]
print(headers)
print(hn[:5])
['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at']
[['12224879', 'Interactive Dynamic Video', 'http://www.interactivedynamicvideo.com/', '386', '52', 'ne0phyte', '8/4/2016 11:52'], ['10975351', 'How to Use Open Source and Shut the Fuck Up at the Same Time', 'http://hueniverse.com/2016/01/26/how-to-use-open-source-and-shut-the-fuck-up-at-the-same-time/', '39', '10', 'josep2', '1/26/2016 19:30'], ['11964716', "Florida DJs May Face Felony for April Fools' Water Joke", 'http://www.thewire.com/entertainment/2013/04/florida-djs-april-fools-water-joke/63798/', '2', '1', 'vezycash', '6/23/2016 22:20'], ['11919867', 'Technology ventures: From Idea to Enterprise', 'https://www.amazon.com/Technology-Ventures-Enterprise-Thomas-Byers/dp/0073523429', '3', '1', 'hswarna', '6/17/2016 0:01'], ['10301696', 'Note by Note: The Making of Steinway L1037 (2007)', 'http://www.nytimes.com/2007/11/07/movies/07stein.html?_r=0', '8', '2', 'walterbell', '9/30/2015 4:12']]
In [3]:
ask_posts = []
show_posts = []
other_posts = []
for row in hn:
    title = row[1].lower()
    if title.startswith("ask hn"):
        ask_posts.append(row)
    elif title.startswith("show hn"):
        show_posts.append(row)
    else:
        other_posts.append(row)
print("Number of posts in ask_posts:", len(ask_posts))
print("Number of posts in show_posts:", len(show_posts))
print("Number of posts in other_posts:", len(other_posts))        
Number of posts in ask_posts: 1744
Number of posts in show_posts: 1162
Number of posts in other_posts: 17194
In [4]:
ask_posts[:5]
Out[4]:
[['12296411',
  'Ask HN: How to improve my personal website?',
  '',
  '2',
  '6',
  'ahmedbaracat',
  '8/16/2016 9:55'],
 ['10610020',
  'Ask HN: Am I the only one outraged by Twitter shutting down share counts?',
  '',
  '28',
  '29',
  'tkfx',
  '11/22/2015 13:43'],
 ['11610310',
  'Ask HN: Aby recent changes to CSS that broke mobile?',
  '',
  '1',
  '1',
  'polskibus',
  '5/2/2016 10:14'],
 ['12210105',
  'Ask HN: Looking for Employee #3 How do I do it?',
  '',
  '1',
  '3',
  'sph130',
  '8/2/2016 14:20'],
 ['10394168',
  'Ask HN: Someone offered to buy my browser extension from me. What now?',
  '',
  '28',
  '17',
  'roykolak',
  '10/15/2015 16:38']]
In [5]:
show_posts[:5]
Out[5]:
[['10627194',
  'Show HN: Wio Link  ESP8266 Based Web of Things Hardware Development Platform',
  'https://iot.seeed.cc',
  '26',
  '22',
  'kfihihc',
  '11/25/2015 14:03'],
 ['10646440',
  'Show HN: Something pointless I made',
  'http://dn.ht/picklecat/',
  '747',
  '102',
  'dhotson',
  '11/29/2015 22:46'],
 ['11590768',
  'Show HN: Shanhu.io, a programming playground powered by e8vm',
  'https://shanhu.io',
  '1',
  '1',
  'h8liu',
  '4/28/2016 18:05'],
 ['12178806',
  'Show HN: Webscope  Easy way for web developers to communicate with Clients',
  'http://webscopeapp.com',
  '3',
  '3',
  'fastbrick',
  '7/28/2016 7:11'],
 ['10872799',
  'Show HN: GeoScreenshot  Easily test Geo-IP based web pages',
  'https://www.geoscreenshot.com/',
  '1',
  '9',
  'kpsychwave',
  '1/9/2016 20:45']]
  • Next, let's determine if ask posts or show posts receive more comments on average.
In [6]:
#Finding the average number of comments per post in case of ask_posts
total_ask_comments = 0
for row in ask_posts:
    num_comments = int(row[4])
    total_ask_comments += num_comments
avg_ask_comments = total_ask_comments / len(ask_posts)
print("The average number of comments on ask_posts", avg_ask_comments)

#Finding the average number of comments per post in case of show_posts
total_show_comments = 0
for row in show_posts:
    num_comments = int(row[4])
    total_show_comments += num_comments
avg_show_comments = total_show_comments / len(show_posts)
print("The average number of comments on show_posts", avg_show_comments)
The average number of comments on ask_posts 14.038417431192661
The average number of comments on show_posts 10.31669535283993

Now We can see that Ask posts get more comments than show posts on Hacker News.Since ask posts are more likely to receive comments, we'll focus our remaining analysis just on these posts.

Next, we'll determine if ask posts created at a certain time are more likely to attract comments. We'll use the following steps to perform this analysis:

  1. Calculate the amount of ask posts created in each hour of the day, along with the number of comments received.
  2. Calculate the average number of comments ask posts receive by hour created.
In [7]:
import datetime as dt
result_list = []
counts_by_hour = {}
comments_by_hour = {}
for row in ask_posts:
    created_at = row[6]
    num_comments = int(row[4])
    result_list.append([created_at, num_comments])
#print(result_list[:3])    
    
for row in result_list:
    hour_with_date = dt.datetime.strptime(row[0], "%m/%d/%Y %H:%M")
    #print(hour)
    hour = hour_with_date.strftime("%H")
    if hour not in counts_by_hour:
        counts_by_hour[hour] = 1
        comments_by_hour[hour] = row[1]
    else:
        counts_by_hour[hour] += 1
        comments_by_hour[hour] += row[1]
        
    

In the last screen, we created two dictionaries:

  • counts_by_hour: * contains the number of ask posts created during each hour of the day.
  • comments_by_hour: * contains the corresponding number of comments ask posts created at each hour received.
In [8]:
avg_by_hour = []
for hour in comments_by_hour:
    avg_comment = comments_by_hour[hour]/counts_by_hour[hour]
    avg_by_hour.append([hour, avg_comment])
    
In [9]:
print(avg_by_hour)
[['23', 7.985294117647059], ['08', 10.25], ['03', 7.796296296296297], ['21', 16.009174311926607], ['12', 9.41095890410959], ['13', 14.741176470588234], ['10', 13.440677966101696], ['02', 23.810344827586206], ['20', 21.525], ['09', 5.5777777777777775], ['11', 11.051724137931034], ['17', 11.46], ['16', 16.796296296296298], ['15', 38.5948275862069], ['01', 11.383333333333333], ['22', 6.746478873239437], ['05', 10.08695652173913], ['06', 9.022727272727273], ['19', 10.8], ['04', 7.170212765957447], ['00', 8.127272727272727], ['07', 7.852941176470588], ['18', 13.20183486238532], ['14', 13.233644859813085]]
In [10]:
swap_avg_by_hour = []
for row in avg_by_hour:
    swap_avg_by_hour.append([row[1], row[0]])
print(swap_avg_by_hour)    
[[7.985294117647059, '23'], [10.25, '08'], [7.796296296296297, '03'], [16.009174311926607, '21'], [9.41095890410959, '12'], [14.741176470588234, '13'], [13.440677966101696, '10'], [23.810344827586206, '02'], [21.525, '20'], [5.5777777777777775, '09'], [11.051724137931034, '11'], [11.46, '17'], [16.796296296296298, '16'], [38.5948275862069, '15'], [11.383333333333333, '01'], [6.746478873239437, '22'], [10.08695652173913, '05'], [9.022727272727273, '06'], [10.8, '19'], [7.170212765957447, '04'], [8.127272727272727, '00'], [7.852941176470588, '07'], [13.20183486238532, '18'], [13.233644859813085, '14']]
In [11]:
sorted_swap = sorted(swap_avg_by_hour, reverse = True)
print(sorted_swap[:5])
[[38.5948275862069, '15'], [23.810344827586206, '02'], [21.525, '20'], [16.796296296296298, '16'], [16.009174311926607, '21']]
In [12]:
for row in sorted_swap:
    av = row[0]
    h = dt.datetime.strptime(row[1],"%H")
    h_str = h.strftime("%H:%M")
    output = "{}: {:.2f} average comments per post.".format(h_str,av )
    print(output)
15:00: 38.59 average comments per post.
02:00: 23.81 average comments per post.
20:00: 21.52 average comments per post.
16:00: 16.80 average comments per post.
21:00: 16.01 average comments per post.
13:00: 14.74 average comments per post.
10:00: 13.44 average comments per post.
14:00: 13.23 average comments per post.
18:00: 13.20 average comments per post.
17:00: 11.46 average comments per post.
01:00: 11.38 average comments per post.
11:00: 11.05 average comments per post.
19:00: 10.80 average comments per post.
08:00: 10.25 average comments per post.
05:00: 10.09 average comments per post.
12:00: 9.41 average comments per post.
06:00: 9.02 average comments per post.
00:00: 8.13 average comments per post.
23:00: 7.99 average comments per post.
07:00: 7.85 average comments per post.
03:00: 7.80 average comments per post.
04:00: 7.17 average comments per post.
22:00: 6.75 average comments per post.
09:00: 5.58 average comments per post.

I think The hour 15:00(3.00pm) is the best time to submit a post to get most number of comments according to this analysis.

It's the estern time zone in USA according to dataset, My time zone is BDT that is 10 hours ahead of it.

In [20]:
for row in sorted_swap:
    my_time = dt.datetime.strptime(row[1],"%H") + dt.timedelta(hours = 10)
    mytime = my_time.strftime("%H:%M")
    #print(mytime)
    output = "{}: {:.2f} average comments per post.".format(mytime,row[0] )
    print(output)
01:00: 38.59 average comments per post.
12:00: 23.81 average comments per post.
06:00: 21.52 average comments per post.
02:00: 16.80 average comments per post.
07:00: 16.01 average comments per post.
23:00: 14.74 average comments per post.
20:00: 13.44 average comments per post.
00:00: 13.23 average comments per post.
04:00: 13.20 average comments per post.
03:00: 11.46 average comments per post.
11:00: 11.38 average comments per post.
21:00: 11.05 average comments per post.
05:00: 10.80 average comments per post.
18:00: 10.25 average comments per post.
15:00: 10.09 average comments per post.
22:00: 9.41 average comments per post.
16:00: 9.02 average comments per post.
10:00: 8.13 average comments per post.
09:00: 7.99 average comments per post.
17:00: 7.85 average comments per post.
13:00: 7.80 average comments per post.
14:00: 7.17 average comments per post.
08:00: 6.75 average comments per post.
19:00: 5.58 average comments per post.

So, For me , the time will be 01:00 in 24 hours formate and it's 1:00 AM

In [ ]: