As you know, I’m an engineer and a scientist by trade. I don’t complain about problems, I design solutions to problems. As such, I’ve been thinking a lot about the problems the Bitter Lemons Imperative created. You may have even read some of my thinking to this point. I have also read through all the published commentary and compliants about what BL did, including the discussions on the pro99 Facebook group. I have also seen, in some of the comments, that there are components of the community that really want the ability to get reviewed when they have been ignored by the media (and they saw this as a way to do so).
I would like to come up with a proposed solution that can be presented to Bitter Lemons that has been pre-vetted by the community — that is, all the kinks have been worked out. I know that a solution is possible: in the accounting business, auditors are paid by the companies they audit, yet have independence in their findings. Corporations also have internal quality control organizations. In my professional life, I’ve done a lot of work with the government’s NIAP/CCEVS program, where commercial labs are paid by vendors to evaluate their products, and validators (such as me) review their work to ensure independence and correctness (before that I was an independent government evaluator for a similar program against the prior criteria). What gives the independence is an agreed upon code of conduct, different reporting chains, independent oversight, and often indirection between the source of funding and the reviewed.
Thinking about this issue, I have come up with the following. Please comment on where additional correction is required and this post will be edited, preserving the changes, until we get to a final solution.
Mission
The purpose of this proposal is to do the following:
- Provide independent and quality theatrical reviews to organizations that are capable of paying for them.
- Provide independent and quality theatrical reviews to organizations that are not regularly reviewed by mainstream theatrical review organizations.
- Provide operating funds for the review manager, and remuneration for the reviewers.
Process
- All reviewers operating under this review proposal must publicly subscribe to the following ethics rules:
[🎭 Note: This provides a basic level of independence and ethics for the reviewer]- The reviewer agrees to knowingly accept nothing from a venue being reviewed other than a pair of complementary tickets, an information packet on the show, and possibly parking.
- The reviewer agrees to disclose any past, present, or future relationships with the theatre or any cast or crew members, and must agree to not let those relationships color their assessment of the show reviewed.
- The reviewer agrees that their assessment of the show will be based solely on the story, performance and presentation, and that their assessment will be honest, even if it is negative.
- Organizations capable of paying for reviews will pay $160 to the review manager to obtain a review. The review manager will retain $25 of this fee to cover operating expenses, and the remaining $135 will go into the review funding pool.
[🎭 Note: The amounts have been selected so that those that can afford to pay provide coverage for those unable to pay.] - For every two paid reviews, the review manager will select, based on an editorial decision of the shows currently running or soon to open that have not received sufficient reviews in the past, a third show to receive a review for no charge.
[🎭 Note: This means that at least one third of the shows reviewed under this proposal get their reviews for free.] - Shows selected for review must agree, at minimum, to provide two complementary tickets per reviewer unless the reviewer refuses. If a show receiving a review for no charge is unable to provide such tickets, the review manager has the option to select a different show for review.
[🎭 Note: Although I’m not in favor of complementary tickets for reviewers, it appears to be standard industry practice.] - Any show that is reviewed agrees not to disclose to the reviewer whether they are a paid or an unpaid review. If a reviewer indicates that a show made such a disclosure, any review will not be published.
[🎭 Note: This addresses the other end: a show disclosing to a reviewer that they have paid for the review in order to try to influence the reviewer. If they do so, they’ve wasted their money and get no review.] - Reviewers are paid $90 for each show reviewed, independent of whether the show paid for a review or received it for no charge. The $90 is based upon an estimated average of 3 hours of writing up the review, 2 hours for the show, and 1 hour transit time at the updated LA minimum wage ($15) for each show.
[🎭 Note: This provides independence of the reviewer from the show. The reviewer has no idea whether the show paid for the review; if they attempt to guess, they have a 1 out of 3 chance of being wrong. [Added 7/2: It also follows the notion of the current idiotic AEA proposal: The reviewer is being paid no more than the actor, who is supposedly getting minimum wage.]] - Reviewers desiring to review under this proposal must (a) subscribe to the ethics rules, (b) apply to the review manager providing samples of recent reviews, and (c) be accepted by the review manager based on the quality of those samples.
[🎭 Note: This ensures that those reviewers accepted into the pool have some level of quality.] - Reviews generated under this approach will be included in the Lemonmeter. Every 6 months, a comparison will be made of the reviewer’s ratings of a show versus the Lemonmeter rating for the show. If the reviewer is outside the standard deviation (i.e., the consensus opinion), they may be dropped from the review team at the discretion of the review manager.
[🎭 Note: This protects against a reviewer who does not appear to be reflective of the opinions of the community — it protects against the “I love everything” or the “I hate everything. It provides the closest one could get to independent oversight, as reviewing is subjects and assessment of whether the reviewer followed an objective process is not possible. The oversight, in a sense, is provided by the community coming to the same consensus view.”] - Ideally, there would be separation between the review manager and the advertising component, so that there is no bias in favor of advertisers in selecting unpaid reviews. This may not be possible for a small website, but ideally the unpaid reviews should be selected based on the theatres regularly receiving insufficient reviews to warrant inclusion on the Lemonmeter (e.g., they are being reviewed, but just not getting enough reviews).
[🎭 Note: The goal here is to prevent bias from the editorial to the selection of the no-pay reviews, addressing the implication that you need to purchase an ad to get a review.]
Let’s address some potential complaints:
- This will be a slippery slope, and all theatres will have to pay. First, this approach includes free reviews based on editorial decisions, so you don’t have to pay to get reviews. Further, there will always be bloggers and others willing to review a show for tickets.
- This will encourage other media to charge for reviews. Other media needs no encouragement. Print media is begging for income. This is an attempt to develop a vetted model that will address that need while still addressing conflict of interest concerns and independence. Hopefully, this might even be a model that those forced to go that direction could adopt.
- How can you charge theatres, and at the same time insist they needn’t pay actors. First and foremost, it is important to remember that no theatre is required to utilize this, and that theatres that don’t use this might still be reviewed for free. Second, I don’t think the position has been that theatres should not pay actors. Rather, the call has been for a tiered system, where creatives (of all types) are paid based upon the theatre’s ability to pay and the budget for the show. If a theatre is budgeting for publicity, this may be part of that budget. If they can’t afford it, they can’t. Nothing is mandatory here; this is just provide an option to those that want to use it. For those that can afford it, budgeting for publicity is part of a production budget, just as is budgeting adequate pay.
- But this looks like a conflict of interest. In an ideal world, there would be no conflicts of interest; reviewers would be paid by journalistic endeavors with no connection between editorial and advertising. But this isn’t an ideal world; there are micro-conflicts everywhere: comp tickets are a conflict, as is any paper or website that accepts advertising at all. Even in such a world there can be independence: look at auditors of major corporations (which are independent), quality control organizations, or the Common Criteria Evaluation and Validation Scheme. They achieve that independence through indirection, strong ethical codes, and oversight. That’s the approach here: have a strong ethical code; get feedback on reviewer quality; and make it so there is uncertainty whether the theatre has paid for the review.
I’m aware that people may not like this idea just because it involves money changing hands. If so, no proposal will be satisfactory unless it fits the model of yore. But our focus should be mission success: getting more people — and younger people and new audiences — to the theatre to see shows, and more reviews help to do that. It should be to improve the viability, the vitality, and the visibility of theatre in Southern California. Let’s work together to come up with a workable approach that satisfies concerns. Let’s be sufficiently open-minded to posit that such an approach is possible if we structure it right.
For those that are willing to make suggestions on how to improve this to address concerns, I’m open to your comments.
Wouldn’t the BL cut would have to be $92.50 to cover operating expenses AND offset every third show being free?
I’m not sure how you get $92.50. I wasn’t thinking BL would get a payment on the free posts; if it was to get one, it should change the $25 to $39.50 ($25 plus 1/2 of $25, so that two shows cover the BL cut for the third show). That would raise the $160 for a paying show to $172.50 — at that point you might as well call it $175 and let BL get a little more (but for you, $174.99).
What do you think of the idea overall? Are there bases I’m missing?
Peter Finlayson, responding to me on Facebook Messenger, had the following comment:
Daniel, Essentially what you’ve created here, is the ethics statement of review procedures is that correct?
I’d love to talk to you about this. However, in the world of journalism, the firewall between editor and Ad sales, has a much better standard of impartiality. This model relies upon the absolute trust that the review manager who is also collecting the funds, sustain absolute impartiality.
The addition of some unsolicited reviews while implying impartiality are still too close to the money and expecting theatres not to mention if they paid for a review or did not pay for a review is a standard that I seriously doubt can be assured.
I guess another point, of curiosity, is how do you see this service as being superior to say Stage Raw?
My response to Peter is as follows:
Essentially what you’ve created here, is the ethics statement of review procedures is that correct?
I think essentially that’s right. I view it more as procedures that has an ethical commitment embedded, but that’s likely just rearrangement of words.
However, in the world of journalism, the firewall between editor and Ad sales, has a much better standard of impartiality. This model relies upon the absolute trust that the review manager who is also collecting the funds, sustain absolute impartiality.
Not being a journalist, of course, I can’t speak to that; my experience is as a government evaluator. But the model of journalism came about in an era where you had sufficient staffing in the enterprise to maintain that separation between departments. In these days of “lean and mean” staffing, that separation is starting to break down — look at the tension that has occurred at the LA times with the special wrappers, and when they turned the LA Times magazine over to the Marketing department. In the online journalism world, it is even leaner and meaner. So we need to try to come up with a process that can provide as much independence as possible (even if not 100% solid firewall) while still working.
The addition of some unsolicited reviews while implying impartiality are still too close to the money and expecting theatres not to mention if they paid for a review or did not pay for a review is a standard that I seriously doubt can be assured.
The addition of the unsolicited review are intended to introduce uncertainty in the mind of the reviewer — they will not know if the review they were doing was paid or not. Expecting theatres not to mention the review is as much enforcable as any other ethics rule — not perfect and more of an honor code, with a penalty if violated. I’m open to suggestions of some other way around the issue. The best might be having the reviewer simply be assigned the review and doing a blind purchase, as any other consumer, of the tickets; the problem is that the theatre might still see their name on the register. The issues might be similar to a well-known restaurant reviewer making a reservation under their name, and the restaurant trying to influence them. We can’t resolve the problem completely, but we can punish ethical violations when they are identified from either side. Perhaps we need to add something to address punishing the reviewer who violates the ethical code or who does not report a theatre that indicated they paid for a review.
I guess another point, of curiosity, is how do you see this service as being superior to say Stage Raw?
Being a cybersecurity specialist and not in the theatre industry, I’m not familiar with Stage Raw’s model. Can you describe it?
In most cases, The publisher controls the dictates to a section editor. The publisher’s responsibility is to meet the community standards of interest, and to assure that revenues are adequate to sustain existence, or hopefully grow.
To accomplish that, a publisher appoints an section editor, someone that presumably knows the field of concern, has connections and understands the players and writers in the field. At no time is that editor responsible for income beyond assuring that community interest are being served.
Advertising which is the traditional income source for journalism, is managed by the sales department. In principal, they have no truck with section editors, and should they have an interest in having a client covered in a news story, the request would have to go through the publisher, and not the section editor.
When a reporter (critic) is assigned a story, there is no potentiality of knowing if the subject has paid for a story. (Should a piece be done that is paid for by the client, that is called an aditoirial, and by law must be identified as such) The reporters job is to witness the event and using the skills and tools of his trade and education, report to the reader a concluding piece.
Now understanding that all of this is a model no longer sufficiently populated, the problem is in how to recreate the lack of influence that sales has upon editorial.
As a business model, What BL has created is unsustainable, if a theatre gets bad reviews, they will be progressively be disinclined to purchase the service. The only way to assure that not happening, is to taint the reviews.
As to The Stage Raw Model, I don’t know of it specifically, but it would appear to sell advertising, solicit donations and has a working editor who meets the qualifications noted above. The firewalls are certainly less protected then in the older model, but they exist by the fact that the product sold is advertising, it’s tangible and definable. Just as importantly, the editor is of such repute that his ethics are beyond reproach, (Despite what some yellow journalist may say on the subject)
I need to mull on this a bit, but it seems like a large part of this would depend on disclosure of the business and management structure of BL, and ensuring that there is sufficient trust in the person who serves in the role of section editor (and publisher). Sales of reviews would need to be on the sales side, and would form a list that goes to the overall publisher above the section manager, who would then add in the free review, and send that assignment list to the section manager (who would not know which was paid or not), and who would be responsible for assigning from the reviewer pool to the show. The section manager should also have the responsibility for providing an editorial and quality review pass on the final review product to provide approval for public posting. To address the advertorial side, the section manager could enforce an appropriate disclaim on the review indicating that it might be a sponsored review, but the reviewer was not provided direct knowledge whether it was sponsored or not. (or some such wording).
As for the business model being unsustainable, I don’t think that is a primary concern. If it fails, we are where we were before the BLI. If it succeeds, we have a way for those theatres that are seemingly excluded from reviews for whatever reason (and can afford it) to at least get the attention of some reviewers.
As I said, I’ll mull some more before making changes, but does this appear to be the right direction. I recognize we may have an incurable in Colin — I’m not sure the community views him “of such repute that their ethics are beyond reproach”.
Reading this again between chapters, I’m not 100% sure of:
As a business model, What BL has created is unsustainable, if a theatre gets bad reviews, they will be progressively be disinclined to purchase the service. The only way to assure that not happening, is to taint the reviews.
That statement would be true if (a) the theatre only wanted good reviews for advertising, and (b) the theatre was guaranteed of getting the same reviewer each time.
If the goal of the theatre is to make the reviewer aware of the theatre in general (which is plausible), (a) might not be true; there’s also the publicist’s magic of being able to find a good pull quote even in a bad review. The theatre might also agree with the problems identified in the review, and just might not highlight it for publicity. They knew the possibility when they paid; they were not paying for a good review, just a review.
As for the second aspect, if you randomize the reviewer assigned and have a sufficient pool of reviewers, then there is a much lower likelihood of getting a bad review the second time, unless all you are putting on is crap (in which case, you arguably deserve it for paying for reviews for crap shows).