Bad Movie Twins Bad Movie Data Analysis #1

Welcome to a special Bad Movie Twins bad movie data analysis post. I haven’t done one of these in a while, but it was high time I started refining my machine learning / data viz creds again. Recently I have been going through the process of collecting a large data set of movie information for use in analysis. This is the first fruit of that labor.

First, the briefly describe the data set. Collected via the yearly box office listings (e.g. For 2010) I collected the top 200 movies by theater count (ultimately this is something like every movie Box Office Mojo has on record released to 300 or so theaters back to 1980). These are Box Office Mojo links. I then meticulously collected the Wikipedia, IMDb, and Rotten Tomatoes links for all of these films (around 7500 films in total). The set ultimately included Rotten Tomatoes review / fresh counts, consensus, etc.; IMDb title, year, genres, etc.; Box Office Mojo gross and theater count, among other things. Eventually I would like to develop a model out of this data, but I wanted to first test out some hypothesis testing on it just to see how it performs.

One of the main things I’ve always been curious about Rotten Tomatoes data in particular was how the review number / freshness has changed over time. My initial priors were: (1) There are far more reviews from 2000 onward than prior to 2000, this is just my anecdotal experience; (2) More recently there have been more reviews and in general, and reviewers have been kinder. Again, just anecdotal. So why not test these hypotheses? So first a quick visualization:


So the first plot here shows two things. First, the probability of a fresh review from a rolling 12 month window for movies that have at least one review, and were released to at least 600 theaters (black line, left axis). This is calculated by summing all fresh reviews and dividing by the total number of reviews on Rotten Tomatoes for that 12 month span (so not an average TomatoMeter, although that generally has the same shape, just shifted downwards). Secondly, it shows the total number of fresh (green filled area) / rotten (red filled area) reviews in the 12 month window (right axis).

The main thing that pops out to me is that when you move backwards and forwards in time from around the year 2000 the “freshness” of the reviews seems to increase. But interestingly the first reason I proposed for why this might happen, the number of reviews increasing, isn’t actually true when you move either way. The review count decreases as you move into the past, and moving forwards in time there is an initial increase, but for the most part the total number of reviews has been pretty stable over the last two decades. The plot does seem to suggest that, maybe, reviewers are getting kinder over time though, so what is happening?

My next though was that the trend is a manifestation of “review inequality”. As you move backwards in time the number of films getting after-the-fact reviews placed on Rotten Tomatoes dwindles, and thus an increasing number of reviews end up being submitted for the best films of the year for the most part, thus the reviews will become increasingly fresh. Alternatively, as you move forwards in time more and more of the “small” reviewers on Rotten Tomatoes are added and they will tend to only review a subset of the, mostly good, major releases. So again, as you move forwards the bigger and generally better regarded films will get a larger and larger share of the reviews.

And an easy test of this hypothesis is to look at something like a Gini Coefficient. There are a multitude of complaints about the Gini Coefficient (as outlined in the wikipedia page), but considering it is a general first step in income inequality analysis, I decided to toss a flyer out there and see if the Gini Coefficient could tell us anything about whether my hypothesis holds water:


And what do you know, the plots are eerily similar. I had figured that this was all a single phenomenon, but the two domains (before somewhere around 2000 and after) do seem to act different. Here is the same plot transformed a bit:


This actually looks like there could be a good partition around the year 2000. So where does the data partition? One (likely simplistic) way would be to develop a piece wise linear regression on the Gini plot and point out the spot where the two regressions meet:


Now that’s pretty cool. July, 1998, was around the point in time (almost to the month) when Rotten Tomatoes was launched. As a matter of fact, for the analysis below I will treat the partition as specifically the launch month of August, 1998. I think the obvious hypothesis as to why the Gini coefficient correlates positively (and shockingly well) with the review probability is that the higher the number of reviews the higher your TomatoMeter tends to be. So when a high percentage of the reviews are distributed to a small number of movies that will drive the overall freshness of Rotten Tomatoes up. And just to hammer home the point as to why this partitioning is particularly interesting, here is a scatter plot of the Rotten Tomatoes reviews and the TomatoMeter for all of the films since 1980 which were released to 600 or more theaters and have at least one review on Rotten Tomatoes:


Super weird. The red squares are formed by grouping the data into 25 groups and taking the mean value. Note that this plot suggests that a movie is more likely to be good if it has around 50 reviews instead of, say, 100. Which without further testing you might think is due to independent movies or some other small-movies-are-better effect. But now let’s split the data by the Rotten Tomatoes launch date and treat movies which were released before (and backfilled) and movies which were released after differently:


And now you could come to a different conclusion. Both sides uniformly increase in quality with the number of reviews, it is just that backfilled films tend to have fewer reviews in total. Almost all movies released prior to Rotten Tomatoes launching have less than 100 reviews on Rotten Tomatoes, a number which today would represent a fairly small release.

I’ll leave the analysis there (this is already incredibly long), but I found this all very interesting and I think it shows the power of analyses like this. If I were to make a model to try and predict a film’s TomatoMeter using the total reviews as an input, for example, and wanted it to apply backwards in time, this analysis strongly suggests that you either want to (1) treat a movie released before and after Rotten Tomatoes was launched differently, or (2) Build a time dependent factor that can build this partition in for you. Again, I like the analysis because this isn’t a totally obvious result in my opinion, and it gives a simple and easy to follow guideline for eventual use in modelling.

And as for my hypotheses? There are indeed fewer reviews prior to 2000 than after, drastically so. But interestingly if you look at the piece wise regression on the Gini vs. fresh review probability plot above it actually kind of suggests that given the amount of inequality with how the reviews are being allocated, that the films are actually getting worse reviews in the last 2 years than between 2000-2016. Which is something I think I’ll want to explore more in a second installment of this series.

Hopefully I’ll get that analysis up soon, but I’ll probably start looking at building a simple predictive model as well. I will say the data set I’ve collected, while it takes ages to update, has already been incredibly useful in finding and analyzing potential BMT films.


The Sklogs


Righteous Kill Quiz

A quiz on the film Righteous Kill. To ace it won’t require much skill. Just patience and time. Obsess on fictional crime. And it’ll help if you’re mentally ill. – Poetry Sklog

Pop Quiz Hot Shot!

1. Detectives Cowan (De Niro) and Fisk (Pacino) intentionally do not use their names during the film (in order to not give away the big twist ending, what a twist!). What are their nicknames?

2. During the course of the film Poetry Boy (spoiler alert, it is Al Pacino) kills multiple people, but what event started it all?

3. Name or describe the five people we “see” Pacino kill during the course of the film.

4. When Pacino and De Niro send a lawyer into Spider’s club (a converted Bank, it is so Hollywood Badass I love it) to get some cocaine while wearing a wire, what does she say she needs the cocaine for?

5. We’ll end with a tough one. In the NYPD softball game we see De Niro score a run. How would you log the run in the scorecard?


Conan the Barbarian (2011) Quiz

You have battled and fought and made your body lean and sinewy, ready to cleave your enemies in two with your razor sharp sword. There is only one thing else to do, Crom demands it … it’s quiz time.

Pop Quiz Hot Shot!

  1. In the beginning of the film Conan’s father sends the youths of the camp off to compete for a spot in the hunt. What is the competition?
  2. What’s more important in forging a blade, fire? Or ice?
  3. The day of his father’s death left indelible scars on Conan’s soul. But it also left slightly less metaphorical scars on his body (something he then shows to the eeeeeevil Khalar Zym to prove he was the boy destined to kill him, how convenient …). What injury was inflicted onto Conan on that fateful day?
  4. Conan frees the thief Ela-Shan from slavery, befriending him for life (convenient, perhaps his lockpicking skills will be vitally important for the climax of this film …). But why did Conan allow himself to be enslaved in the first place?
  5. Straight up … what is the plot of this film? Who is Khalar Zym, what is he looking for, and why is he looking for it?


Alex & Emma Quiz

While dictating the next great American novel to the harried freelance stenographer you hired, your publisher called you up and said four words that would change the course of your romantic life! …

Pop Quiz Hot Shot!

  1. Throughout the movie Emma (Kate Hudson) bothers Alex (Luke Wilson) by doing what? Hint: It concerns books … I mean Luke Wilson seems perturbed throughout, this seems to offend him.
  2. Rob Reiner just loves Alex’s mojo. How many novels has Alex written in his illustrious career?
  3. In order to lure Emma to her murder … er, to be a freelance stenographer in a highly unlikely literary gambit, Alex advertised the job under what guise?
  4. What nationalities does Emma’s fictional alter-egos in the story-within-a-movie take on throughout the film?
  5. The target of Adam Shipley’s lust is the single mother Penelope Delacroix. Adam is worried that Penelope is going to wed the very rich John Shaw, to whom she owes an enormous sum. What two ways does Adam think to get her out from underneath her debt?


Behind Enemy Lines Quiz

Uh oh, you thought you would just buzz by this website and peruse our bad movie ramblings? Think again! You just got shot down by our Bad Movie anti-aircraft missiles and are stuck behind the Bad Movie enemy line. And we’re after you, this ain’t no joke, we’ve been stuck in a Bad Movie prison camp and know how to do one thing well: kill. You best get ready for a quiz!

Pop Quiz Hot Shot!

  1. There are two crewmen in Burnett’s jet, a pilot and navigator. Which is Burnett?
  2. After Burnett (Owen Wilson) and Stackhouse’s flight is cancelled (again, boo!) they go to the mess to get their chow on. What food does Burnett make quite a show of eating?
  3. After getting shot down Burnett and Stackhouse are separated from their cockpit chairs which had homing beacons. Why did Reigart subsequently order Burnett’s homing beacon to be turned off?
  4. After a close escape from the factory booby trapped with mines Burnett escaped to a nearby highway to flag down a very America-loving truck. What kind of music was the truck playing?
  5. The Serbians have a dastardly plan to fake Burnett’s death to prevent the Americans from deploying a rescue mission. What event tips off the battleship that Owen Wilson was alive and well?


Chernobyl Diaries Quiz

Last week we did some Xtreme sports, and this week a quiz about Xtreme tourism. Well, grab your Geiger counter, it’s quiz time.

Pop Quiz Hot Shot!

  1. On their extreme tourism adventure into the heart of Chernobyl country Uri takes our heroes on a tour of Pripyat. Where were they supposed to go instead and why?
  2. How many people went on this 4-hour-turned-4-ever tour with Uri in the Zombie Apocalypse on Gilligan’s Island very special television event (better twist than we actually got, imagine if at the end it turned out to be some Eastern European reality show)?
  3. When exploring an apartment building Uri sees three things that freak him out and cause him to rush everyone back to the van. One point each for identifying what concerned Uri about the town and building in particular that made him cut the tour short?
  4. After getting trapped in the town the group decided to be a bunch of dummies like the dummies that they are and stay around overnight like dummies. How far does Uri say the checkpoint is when they discover the van has been sabotaged?
  5. Ultimately after discovering a large band of mutants roaming the abandoned town the last two survivors, Paul and Amanda, are forced through a series of steam tunnels and into the abandoned reactor itself. Who are the “monsters” of the film, and what is the deal with the town itself? And I’m not just asking because the movie is very confusing … but also my answer might be wrong.

Spooky scary indeed. My bro doesn’t scare easy though and got them all right, how about you?


Smaddies Baddies V

It has become an annual tradition. On the anniversary of the start of Bad Movie Thursday we take a look back at the year in review. With a name that’s just as bad as the films it honors it is ….

Smaddies Baddies! Smaddies Baddies! Ah what a year. We watched all of the Friday the 13th films, smashed some terrible sequels, started the Periodic Table of Smellements, and brought along some friends to see whether we are missing any borderline BMT film genres (we aren’t). And naturally no year would be complete without watching the worst of the worst of 2017 including quite literally the most awkward theater experience possible while watching Fifty Shades Darker. Remember, any film we watched in 2017 qualifies. Smaddies Baddies, what films do we want to bestow the highest honor in all of Hollywood (er … Minnesota and London as least)?

As usual let’s start with the sci-tech awards of the BMT world: the special awards using the 6W’s template. Let’s go!

The I Know Who Killed Me Best Twins Ever Baddie (Who?) goes to Baby Geniuses as the best good twin film of the year, but what’s this?! Superman III comes swooping in with Evil Superman! It’s a tie! What an upset, is Evil Superman even a twin? … Doesn’t matter because if he isn’t then he’s the best out-of-nowhere character in history and single-handedly makes Superman III a hilarious BMT romp. Thank you Evil Superman, you are the best.

The Adam Sandler Memorial Product Placement Baddie Brought to You By Subway, Eat Fresh! (What?) Literally, Beverly Hillbillies has the greatest product placement I think we’ve ever seen. Just, in the middle of a party, Jethro Bodine, orders a six-foot subway sub to chow down on. If you don’t mind horrible quality YouTube videos, here it is. Impressive, I hope they got paid handsomely for this, because it is nuts.

The When in Rome Setting as a Character Baddie (Where?) This year we had a veritable bounty of choices for settings, but only one served as a truly terrible advertisement for an amusement park, city, and state! Jaws 3D took us away from quaint Amity Island and right to Seaworld, where the tourism board promises one thing: you will be eaten by a large shark as a direct consequence of the negligence of the staff who works there. Excellent job Seaworld, I hope you got paid for that shit.

The Marion Cobretti Memorial Secret Holiday Film Baddie (When?) Again, we had a bountiful harvest of bad movie holiday films this year. We had a Columbus Day film (Rings), a Memorial Day film (That’s My Boy … although it was supposed to be a Patriots Day film we think considering it ends during the Boston Marathon), but nothing managed to beat the one-two combination of both I Know What You Did Last Summer and its sequel I Still Know What You Did Last Summer taking place on July 4th. Happy Fourth o’ Juuuuuuuulaaaaaaaaaay.

The Street Fighter Legend of Chun Li Best McGuffin Baddie (Why?) You would be shocked to hear how many ridiculous McGuffins we encountered this year. The Fountain of Youth and Poseidon’s Trident in the Pirates of the Caribbean series, Omicron’s scepter in the newest Transformers film, a world destroying machine in Lara Croft … And yet it all pales in the face of the original Obsidian Dongle (as we call it) in Big Momma Like Father Like Son. A USB drive with, well … something important on it that everyone just needs to get their hands on, what more could you ask for in a bad movie? A plot?! Naw, I’ll take the Obsidian Dongle every time.

The 88 Minutes Starring Leelee Sobieski Worst Twist Baddie (How?) This category was of fierce debate at BMTHQ, but given the amount of play a certain type of twist got amongst the BMT recaps in the last few months, only one terrible twist would do: Don’t help the little girl ghosts! In both Rings and One Missed Call our intrepid protagonists discover a terrible secret: the ghosts haunting them were abused and died in terrible pain, calling out for anyone to help them. So naturally our gifted investigators will help them out, free them from their tormentors, and break the curse! Oops, don’t help the little girl ghosts, idiots. Obviously they are just using you to spread their hate. You dumb. Don’t help the little girl ghosts! We’ve seen this twist three time (!) in the last year, and it never gets any better.

Phew. Now onto the big awards, now officially based on The Good, The Bad and The BMT (plus Live!). And without further ado:

The Freddy Got Fingered Surprisingly Good Baddie (The Good) Nominees: Friday the 13th Part VII: The New Blood, Species, Ghosts of Girlfriends Past, Table 19, Beverly Hillbillies.

And the Winner is: Friday the 13th Part VII: The New Blood. Not only was Friday the 13th the most fun we had with a series in BMT, but the supernatural bent of the seventh film managed to bring the first formidable good guy to face off against the unstoppable force of zombie Jason. Is it the best of the series? Not by a long shot, but for us it was the most fun (while also being one of the spookiest of the (admittedly not very scary) series as well).

The Strange Wilderness Unpleasantly Terrible Baddie (The Bad) Nominees: It’s Pat, Miss Congeniality 2: Armed and Fabulous, Friday the 13th: A New Beginning. Superbabies: Baby Geniuses 2, The Emoji Movie

And the Winner is: Friday the 13th: A New Beginning. This was a close one, but the Friday series pulls off the double! This film is the Halloween 3 of its own series, a reboot of a kind where they try and shed the main antagonist of the series and fail miserably. Is it the worst films we saw? No, It’s Pat is more poorly made and offensive. But this was a slap in the face to Friday fans, and there is no denying that we became fans of the series by the sixth film. That slap still stings A New Beginning, unforgivable.

The Here on Earth Most BMT Baddie (The BMT) Nominees: The Bye Bye Man, Fifty Shades Darker, Superman III, I Still Know What You Did Last Summer, Big Momma Like Father Like Son

And the Winner is: The Bye Bye Man! Don’t think it. Don’t say it. Don’t think it. Don’t say it. What is better than a horror film which makes you laugh? A horror film that posits the twist that you just don’t have to fear him to stop him … and then throws it away right at the end because you can’t think of an ending. I legit laughed at the monster. The movie just never quits being not-scary. Movies like this is what made me become a fan of the genre, making scary things is hard so when it is done well it is amazing, but the margin of error is thin. I think this is the enduring film of the year for us, it is that funny.

The Jack and Jill Worst of 2017 Baddie (The BMT Live!) Nominees: Fifty Shades Darker, The Bye Bye Man, Transformers: The Last Knight, The Emoji Movie, Geostorm

And the Winner is: Fifty Shades Darker. The most awkward theater experience of my life has to be rewarded somehow. Sitting there like a creeper listening to that bomb soundtrack and transcending this mortal plane. What is more Jack and Jill than looking deep within oneself and thinking “this movie is not made for me … and yet I am trapped here watching it … this is unpleasant”? That is why Fifty Shades Darker is the best-worst of 2017. And guess what … we get to watch Fifty Shades Freed this week! Ah, what a terrible life we lead.

Smaddies Baddies, Smaddies Baddies. I love it when a wide array of films gets recognized. As usual, for those of you who fell asleep for the announcements: Watch the Friday the 13th series, Superman III, and The Bye Bye Man. Skip Fifty Shades Darker, Rings, and Jaws 3D. Actually, you know what? Watch them all, you know you want to, become like us, become the monsters you hate!