Saturday, December 13, 2008

One Unfortunate Side-Effect

One Unfortunate side-effect of the way we do Critic-O-Meter is that show's get an initial grade based on the early batch of the reviews and then frequently that grade gets revised, sometimes dramatically. Back Back Back's early Critic-O-Meter posting had a grade around a B- (it might have been a B-/C+, I'm not sure) largely based on Charles Isherwood's review of the play in the Times but later reviews pushed it up to a B+. I know I'm a lot more willing to see a B+ show than a B-/C+ show, especially at MTC prices. Similarly, Rock Of Ages score fell from an initial A- to a B as we began to pick up the stragglers.

This is unfortunate, as it means our initial posts are less accurate than we like and, if Critic-O-Meter becomes as popular and useful as we hope, this could actually affect how shows are received and remembered. At the same time, it might be unavoidable. Many reviews are posted the day after the show opens in the dailies and on several websites. But weekly reviews- which includes TONY, The Village Voice, The New Yorker, New York, The New York Observe and The Wall St. Journal, all of whom are pretty heavy hitters comparatively speaking- weigh in later in the week. Sometimes various press organs don't review a show until later in its run if its a smaller show, or sometimes they'll hold onto a review until they have the column inches to spare. The New York Times just posted a review of Too Much Memory today, almost a week after it opened, that pushed its grade up from a B to a B+.

I'm not sure there's anything we can really do about this at this point. Unfortunately, weekly magazines and theatre columns don't cater to our needs. Jerks. Seriously, though, if you have any ideas for how to solve this particular pickle, please suggest away.

5 comments:

The Director said...

The obvious solution would be to impose a wait period before aggregating reviews -- say, a week after opening? That gives the weekly columns a chance to ring in.

Parabasis said...

Ah,, we thought of that, and here you get the pickle! What if it's an off-bway show with a three or four week run? Usually one of those weeks is previews so... that leaves you with only 2 weeks left in the run and that makes us less useful!

Anonymous said...

Is it possible that you could put with the grade (tentative)? That way, the reader is aware that this is how the grade is at the moment, but it might change.

Rob Weinert-Kendt said...

Getting people to come back and check our site repeatedly--that's a feature, not a bug! Seriously, this is a Broadway vs. Off- and off-Off issue. Typically there are between 15 and 20 reviews of a Broadway show available the morning after opening night, though by the time all the reviews are in it can be as many as 25. And usually the weekly reviews don't affect a Broadway score too much (unless you count Feingold's F for "The Seagull"). It's different with even major Off-Broadway shows, which typically have 5-10 reviews available the next morning, and swell to as much as twice that slowly but surely (with the score changing accordingly). I guess we can hope readers who check in regularly will be surprised and intrigued to see shows' rankings change from week to week, or day to day.

isaac butler said...

Maybe when a show's grde changes dramatically enough we can do a brief post that's like: Seagull goes from B+ to B- and Autobigraphical Vanity Show By Minor Celebrity goes from C- to B and link to them or something. That might be the easiest thing to do.