MIS 665 Final Project¶

Jace Bothner

Watch the video presentation here:
https://youtu.be/QsSIUkY10kY

If you're not able to view the interactive visuals try viewing the project on my github page:
https://github.com/jbothner21/MIS_665_Final_Project

Table of Contents

1. Project Introduction and Problem Definition¶

The goal of this project is to analyze an IMDb movie dataset and use the x variables to predict the y variable, IMDb score. We will accomplish this using data analytics methods data visualization, correlation analysis, regression, classification, and clustering.

2. Import and Clean Data¶

Some columns will clearly not be useful in our analysis. The columns 'IMDb link', 'plot keywords', and 'aspect ratio' won't be meaningful in analysis, so we'll drop these extraneous columns.

There are 45 duplicate rows in the dataset. We'll drop these duplicated rows to avoid repeated information that could influence our results.

Below are all the values for each column that are null or missing. We'll have to deal with these null values in various ways because some methods of analysis will not allow null values.

The three variables missing the most data are gross, budget, and content rating. Because of the nature of these variables, it's difficult to find a solution to fill the null values. There is too much variation in the dataset to replace with mean, median, or mode. Therefore we'll have to drop all rows that have null values for these three variables.

Even after dropping the null values in gross, budget, and content rating, there are still 3,806 observations in the dataset. This is still a sufficient sample size to analyze and draw conclusions from. Now there are still 11 columns that have null values.

One method of filling missing data, especially categorical data, is to replace the null value with the most commonly observed value.

In the case of the color and language columns, each variable has one value that occurs more frequently than others. So we'll replace the missing color values with color, and the missing language values with English. Now only 8 columns have missing values.

Another technique for filling missing data is by calculating the central tendency, like mean or median, for each column and replacing the null values with that specific central tendency. This is the methodology we'll use for missing integer variables, number of faces in the movie poster, the number of reviews given by critics, and actors Facebook likes. Since there is some variation in the data, and since all the numbers are whole numbers, we'll fill the missing values with the median.

After filling these values with the median three columns have null values: the names of actors one, two, and three. We could fill these missing values, but since there are numerous actors in the dataset, and since these few missing values won't affect our analysis, we'll leave these null values.

After filling null values, we'll want to handle any outliers in the data that might influence analysis. Since the median budget is 25 million dollars, some outrageously high budgets in the dataset are present and will be considered outliers. We need to remove these values that are either typos or inaccurate information since there are no movies that have had a budget even close to 12 billion dollars.

We'll treat any budget above 300 million dollars as an outlier, and drop the row. After doing so, 12 outliers were dropped from the dataset.

Now we'll sort out the content rating category. There are 12 different options in the content rating column. We can consolidate these ratings and organize them into more manageable categories.

We'll redistribute the content rating categories the following way:

• GP -> PG
• Approved -> PG
• Passed -> PG
• M -> R
• Not Rated -> Unrated
• X -> Unrated
• NC-17 -> Unrated

Now we have 5 different content ratings that follow a natural scale, from movies that are more family friendly to movies that have mature themes.

In order to analyze categorical columns with certain methodologies, we'll have to convert those columns from objects to integers. We'll do this for the following columns; color, language, country, and content rating. For color, we'll convert the column into dummy variables since there are only two options for the color column. The movie is either in black and white, which is assigned a 0, or in color, which is assigned a 1.

Since we already redistributed the content rating column into 5 groups that follow a natural scale, we can assign each of the ratings a number. G will be 1, PG will be 2, etc. In other words, a 1 will be a more family friendly movie, and a 5 will have more adult themes.

We've already established that the categorical columns language and country have a logical most frequent value: English is the most occurring language and the United States is the most common country. Therefore, we can create a new column with dummy variables for each of these categories. For language, 1 means the movie is in English, 0 means the movie is in a different language. For country, 1 means the movie is from the United States, and 0 means the movie is outside the USA.

Now we need to focus on the genres column. Since it's difficult to sum a single movie up into just one genre, many movies list multiple genres. We'll have to split each movie's listed genres so we can analyze them individually.

Since we're splitting up the genres, each movie could have multiple rows depending on the number of genres listed. For instance, if a certain movie has 5 genres listed, that movie will have 5 separate rows. For this reason, there are 11,325 rows after splitting the genres.

It may provide some insight to create a new column based on gross and budget. Subtracting budget from gross will yield the total profit of the movie.

For analysis later in the project, we'll set custom bins based on the IMDb score of the movie. The movies in each bin represent the following:

• Bin 1: 'bad' movies
• Bin 2: 'ok' movies
• Bin 3: 'good' movies
• Bin 4: 'excellent' movies

Now we'll reorder the columns so that similar variables are next to each other for easier analysis. We'll also create a copy of the dataframe with only integer columns for use later in the project.

3. Data Visualization¶

While the ultimate goal of this analysis is to predict the IMDb score of certain movies, it may help to first get a better understanding of our data through visualization.

a. Movie Characterisitics¶

This first figure shows the distribution of the IMDb scores in the form of a box plot. The median IMDb score is 6.6, the first quartile is a score of 5.9, and the third quartile is 7.2.

Now we'll visualize the data by year. We can see from the histogram that we have more data from movies made in the late 1990s and onward. This might be because we had to drop the rows that were missing gross and budget values, and that information is less readily available for older movies. Alternatively, it's also possible that the barrier to entry for making films has been reduced due to an increase in technology leading to more films being made.

Drama and comedy are the most popular movie genres shown by the bar chart below.

We've already seen the numbers for content rating while cleaning the data, but by visualizing the content rating in a pie chart, we can see just how prevalent PG-13 and R rated movies are. In fact, almost 80% of the movies in the dataset are represented in these two ratings.

A histogram of duration shows us that most movies are clustered around 100 minutes (1 hour and 40 minutes). However, the duration histogram is skewed right, meaning there are more values above the center.

b. Financials¶

The histogram of movie gross shown below is severely right-skewed. This means that it's fairly rare for a movie to gross more than 50 million dollars and a decent portion of the movies in the dataset gross below 10 million dollars.

The histogram displaying movie budget tells a similar story to that of movie gross. The histogram is skewed right, although less drastic, meaning it's more common for a movie to have a lower budget as opposed to a higher budget. It's actually fairly common for a movie to have a budget of fewer than 5 million dollars.

The relationship between movie gross and its budget is shown below. There is a moderate positive correlation between these two, which is fairly intuitive due to the similarities between the two histograms above. Essentially, as the movie budget increases, the movie's gross should increase too, the extent of which will be discussed in the correlation analysis section.

The profit histogram below shows that movie profit is fairly normally distributed, with the center around 0 dollars. This means that a large portion of movies will just breakeven in terms of profit. However, profit isn't perfectly normally distributed as the graph is slightly right-skewed, meaning that some movies make a huge profit.

The scatter plot below shows that movie gross and profit are moderately positively correlated. This is not surprising and is fairly intuitive, as a movie grosses more, it will likely make a larger profit.

c. People¶

The bar chart below shows the top 10 most prolific directors, or in other words, those who have directed the most movies.

The bar chart below shows the directors who have the highest average IMDb scores across all of their movies. For context, as we found from earlier analysis, the median IMDb score is 6.6.

Similar to the prolific directors graph, the graph below shows the highest number of movies for an actor in a leading role.

Also similar to the best directors graph, the graph below shows the top 10 leading actors, based on average IMDb score across all their movies.

Conversely, the graph below shows the 10 worst leading actors based on their average IMDb score.

The top 10 most prolific supporting actors are shown below.

The top 10 supporting actors based on average IMDb score is shown in the bar chart below.

d. Facebook¶

The scatter plot below shows the relationship between movie Facebook likes and IMDb score. There's a substantial amount of Facebook likes in the dataset that are 0. This means that a lot of these movies just don't have Facebook pages for one reason or another. Because of this, the variable may not be a strong determinant of IMDb score, but we'll discuss this more in the correlation analysis section.

The director likes by IMDb score scatter plot tells a similar story to that of movie Facebook likes. At the time of data collection, many of the directors do not have a Facebook page.

Similar to the other Facebook variables, there is seemingly little correlation between cast Facebook likes and IMDb score.

e. Reviews¶

This figure below shows the distribution of the IMDb reviews from critics in the form of a box plot.

The histogram below shows the distribution of the number of users who gave a review on IMDb. As with other histograms shown before, this one is also skewed right. Almost half (1852/3794) of the movies in the dataset have under 200 user reviews.

The histogram for the number of IMDb user votes is shown below, and much like the user reviews variable, the histogram is skewed right. However, the number of user votes has a much more drastic skew. A substantial portion of movies in the dataset have less than 20,000 votes.

4. Correlation Analysis¶

Correlation is a measure of relationship strength between two variables. If two variables are highly positively correlated, then an increase in one will generally lead to an increase in the other. If two variables are highly negatively correlated, then an increase in one will generally lead to a decrease in the other. The correlation coefficients for each of the x variables with the y variable, IMDb score, are shown below.

The variables that are the highest determinants of overall IMDb score are likely the ones that are highly correlated with IMDb score. Interestingly, there are no variables that have a strong correlation, so the variables that are 'highly' correlated will have to be based on the context of the other variables. In this case, the variables that we will deem to be 'highly' correlated (above 0.2) are:

• num_voted_users
• duration
• num_critic_for_reviews
• num_user_for_reviews
• movie_facebook_likes
• profit
• gross

These findings are generalized and the extent of the variables' likelihood to affect IMDb score is based on the correlation coefficient as mentioned above. So now that we have this information, what can we do with it? The next step is to set up a regression model based on the determinants to predict IMDb score.

5. Regression¶

The first step in regression is setting x and y variables. The y variable, the variable we're trying to predict, is IMDb score. The x variables, or determinants of the y variable, will be all the numerical columns aside from IMDb score.

a. Scikit-learn¶

The first model we'll build and test is Scikit-learn's linear regression algorithm.

The strength or accuracy of a regression model is evaluated through mean square error (MSE) and R-squared. The goal of a regression model is to minimize MSE and maximize R-squared. A lower MSE means less error in the model.

b. Statsmodel¶

Next we'll test Statsmodel regression algorithm based on ordinary least squares regression.

The R-squared terms of Statsmodel and Scikit-learn models are virtually identical (0.416), but Statsmodel has a slightly higher MSE, meaning it's a slightly less accurate model than Scikit-learn.

c. Regularization¶

Now we'll build a regression model using Lasso regularization algorithm.

This model has a higher MSE and a lower R-squared than previous models, meaning it's less accurate.

d. Feature selection¶

Next we'll build a model using feature selection. SelectKBest will choose the 12 most important variables that we'll be able to build a regression model from.

The 12 most important variables according to SelectKBest are:

• title year
• content rating
• duration
• from USA
• in english
• gross
• profit
• movie facebook likes
• director facebook likes
• num critic for reviews
• num user for reviews
• num voted users

The regression model built with feature selection has a higher MSE and a lower R-squared than previously tested models meaning it's less accurate.

e. Testing best model¶

By analyzing the MSE of each of the regression models, we can tell that Scikit-learn is the most accurate model because it has the lowest MSE of the 4 models tested. This means that the Scikit-learn model has reduced the total amount of error in its regression model.

The plot below shows the actual IMDb scores plotted against Scikit-learn's predicted scores. Below the plot is the root mean square (RMS) of the Scikit-learn model. This is helpful in analysis because RMS puts the error term into the same scale as the y variable. In other words, on average, this model wrongly predicts the IMDb score by 0.8. This seems very high, especially since 0.8 can be the difference between a good movie and a great movie. However, the reason for the high RMS is that the model doesn't predict IMDb scores below 5 or 6. This means that there is a greater amount of error in lower IMDb rated movies, which increases the error overall. This is most likely due to the nature of the dataset and not an issue with the model. In other words, because of the dataset, the model will be more accurate with IMDb scores above 5 or 6.

6. Classification¶

Earlier on in the project, we set custom bins based on the IMDb score of the movie.

• Movies in bin 1 represent 'bad' movies (0-4 IMDb score)
• Movies in bin 2 represent 'ok' movies (4-6 IMDb score)
• Movies in bin 3 represent 'good' movies (6-8 IMDb score)
• Movies in bin 4 movies represent 'excellent' movies (8-10 IMDb score)

Now we can use the dataset to build classification models to predict which bin movies will classify as. Effectively we are predicting whether a movie is 'bad,' 'ok,' 'good,' or 'excellent' instead of predicting the IMDb score overall.

a. Decision tree¶

First, we'll use a decision tree classification model. This model splits the dataset into training and testing groups. 70% of the data will be used to train the algorithm, while the remaining 30% will be used to test the model and its accuracy.

A classification model is evaluated by its accuracy, the amount of correctly classified data points in the test set divided by the total number of data points in the test set. In this case, the model correctly classifies the data 71.5% of the time. A confusion matrix is a visualization of the accuracy of a classification model. The goal of a confusion matrix is to maximize the true rates, the numbers diagonal from top left to bottom right. Naturally, when maximizing the true rates we'll want to minimize the rest, which are called false rates. In this case, the true rate is high enough to suggest a decent amount of accuracy in the model. However, it's worth noting that the model does not correctly predict any 'bad' movies and instead falsely classifies them as 'ok' or 'good.' Similarly, there are quite a few instances of 'ok' movies that are misclassified as 'good' movies. This is a similar phenomenon found in the regression model in which the model does not predict IMDb scores under 5. This further proves the theory that this error is due to the nature of the dataset and not the algorithm itself.

Shown below is the output of the decision tree used to classify each movie. The decision tree starts with num_voted_users, splits them into two groups, above and below 89,788, then moves to the next node. This process repeats until all observations have been classified. The gini score represents how homogeneous the data in each node is. Zero represents complete homogeneousness, which is the goal of classification. The issue with this decision tree is that some of the nodes on the final level are mostly heterogeneous when they should be completely homogeneous in an ideal model. Again, this is most likely due to the nature of the dataset where the algorithms tend to misclassify movies that have a lower IMDb score.

b. KNN¶

Next, we'll use KNN to build a classification model. Again, like the decision tree, the data is split into 70% training and 30% testing. KNN stands for k-nearest neighbor. The algorithm uses the training data to search for the nearest neighbor of a test data point and classifies it accordingly.

The accuracy of the KNN model is shown above. The model correctly classifies the data 59.4% of the time. This is substantially lower than the decision tree model. According to the confusion matrix below, it appears that most of the error is coming from 'ok' movies misclassified as 'good' movies.

c. Logistic regression¶

Logistic regression is similar to linear regression used earlier in the regression section. However, logistic regression is used for classification and is a non-linear model. Again we'll split the data into 70% training and 30% testing.

The accuracy of the logistic regression model is shown above. The model correctly classifies the data 65.9% of the time. This is more accurate than the KNN model but still less accurate than the decision tree model. The confusion matrix shown below reveals that the algorithm mostly predicts movies as 'good' because that's where the bulk of the data is. This is not conducive to a good classification model.

d. Random forest classifier¶

Now we'll build a random forest classification model. Random forest operates similarly to the decision tree model, but instead of building one decision tree, it builds 20 decision trees. Again we'll split the data into 70% training and 30% testing.

Random forest is the most accurate of any model tested so far. The model correctly classifies the data 76% of the time. The confusion matrix above shows that the random forest model is not only more accurate but also doesn't rely on classifying the most commonly occurred class for its high accuracy.

f. Recursive feature selection¶

Recursive feature selection ranks the importance of each variable. Those variables are then used to build a logistic regression classification model.

We'll use the top 10 most important variables according to RFE shown above to build a logistic regression model.

The RFE model correctly classifies the data 66.3% of the time. Because RFE uses logistic regression, most movies are classified as 'good' movies because that's where most of the training data is classified. This is not conducive to building a good classification model.

g. Best model¶

The random forest model is the most accurate of all models tested, both in its overall accuracy and in the visualization of the confusion matrix. The random forest model correctly classifies the data 76% of the time.

7. Clustering¶

The goal of clustering is to segment the data into distinct clusters and develop cluster profiles based on each cluster's information. The first step is to determine the most important variables to use in the clustering analysis. We'll select the 8 most important variables according to feature importance.

a. Normalize data¶

We'll need to normalize the data before performing cluster analysis. The reason for this is that the scale or variance of each variable is unique and will distort the importance of the clusters. For example, the variance of gross at 4.8 quadrillions, is much higher than the title year variance at 99.2, and the algorithm will give more importance to the higher variance. The point of normalization is to make sure all variables are on the same scale, and therefore on a level playing field when determining the importance of each variable.

Now that the data has been normalized, all variables share the same scale, and the size of the initial variance won't influence the clustering results.

b. Elbow method for determining number of clusters¶

The elbow plot below helps determine the optimal number of clusters for a dataset. Too few clusters and there will be a loss of individual information, too many clusters and analysis will be more difficult. The elbow plot's namesake comes from the optimal cluster point resembling an elbow joint. It appears that the 'elbow' is at 3 clusters. This analysis is subjective though and others may argue 2 or 4 clusters.

c. Assign clusters¶

Now we'll assign clusters using k-means clustering based on the number of clusters selected in the elbow plot.

The number of movies in each cluster is shown below. The majority of observations fit into cluster 2 (with the index 1), while only a select few have made it into cluster 1 (with the index 0).
Note: python indexing starts at 0 not 1

Now we can view each cluster's average values for each variable selected. Before we analyze this information and develop profiles from it, we need to make sure certain cluster values are significantly different from other clusters with more certainty than just an arbitrary guess.

d. t-Testing¶

The purpose of t-testing is to find statistically significant differences between two numbers. In this case, for each variable, we'll test whether each of the 3 cluster's values deviates significantly from other clusters.

Two numbers are statistically significantly different if the p-value is less than 0.05. The cells highlighted in red have a p-value above this cutoff value, therefore they are not statistically significant. So in terms of title year, cluster 1 is statistically different than cluster 2 and the same is true between clusters 1 and 3. However, title year is not significantly different between clusters 2 and 3. Profit is the only variable in which none of the clusters are statistically significant because the p-values are greater than 0.05. Therefore we will leave these highlighted cells out of our cluster profiles. The rest of the variables are statistically significant between each of the clusters. Now that we have this information, we can develop cluster profiles.

e. Cluster profiles¶

Cluster 1 - "Blockbusters"
These are the movies that are a hit at the box office, everyone's talking about them, and are in contention during awards season.
Compared to other clusters these movies on average are:

• More recently released
• Longer in duration
• Very high grossing with very high budget
• Very high number of critic reviews, user reviews, and voted users

Cluster 2 - "Indie Movies/Cult Classics"
These are the movies that are made with limited resources and don't make large amounts at the box office but still have their place in the cinema world.
Compared to other clusters these movies on average are:

• Shorter in duration
• Lower grossing movies with lower budget
• Lower number of critic reviews, user reviews, and voted users

Cluster 3 - "Rainy Day Films"
These movies bridge the gap between the extremes of blockbusters and indie movies/cult classics. They have a decent sized budget and usually recoup their budget at the box office, but don't garner the same amount of attention or award nominations.
Compared to other clusters these movies on average are:

• Right in the middle in terms of duration
• Average budget and gross
• Between the other clusters in terms of critic reviews, user reviews, and voted users

8. Storytelling and Conclusion¶

a. Limitations¶

Before concluding the analysis, it's worth noting that there are a few drawbacks to this dataset and project:

• The dataset features outdated and inaccurate information
  • The most recent movies in the dataset were released in 2016
  • IMDb scores, number of reviews and votes, change daily and are therefore outdated
  • Facebook likes were scraped around this time also and therefore are outdated
  • It's unclear whether or not financial information like budget and gross were adjusted for inflation and if gross is purely box office sales or incorporates streaming
  • Some outliers were removed, like a movie with a 12 billion dollar budget, but other, less obvious outliers may still exist putting the data integrity into question
• The dataset uses some variables that are useless for predicting the success of a movie before release
  • Some of the most important variables in the dataset involve the number of IMDb reviews a movie has which is impossible to tell before a movie is released
• The dataset fails to predict movies below a certain IMDb rating threshold
  • There is a strange phenomenon within this dataset where each algorithm, whether regression, classification, or clustering, fails to predict movies below an IMDb rating of about 5-6
  • This is very apparent in section 5e where the actual versus predicted values are plotted and no values are predicted below an IMDb score of 4.5
  • This means that each model will be more accurate with movies above an IMDb score of 5-6 and much less accurate below that threshold

b. Data Visualization & Correlation¶