Friday 9 November 2012

SIBA Beer Judging – Not Fit For Purpose?

I recently judged at SIBA’s Great Northern Beer Competition at the Mercure Hotel in Manchester.

SIBA is an organisation I respect.

I have considerably more respect for SIBA than I do another beer organisation that possesses a pseudo-Marxist perspective on the beer market, is notoriously dogmatic and intransigent, and promotes the fallacy that all real ale is good beer and its corollary, all non-real ale is bad beer.

I judged at the same SIBA event two years ago. It was a dispiriting experience. I witnessed incidents of spectacular beer-judging cluelessness.

SIBA adopts the position that beer judging should reflect the preferences of the kind of people who consume its members’ beers. No particular credentials are required to be a SIBA judge other than being a beer drinker and being somehow connected with the trades of brewing or selling beers (with a few random “dignitaries” requiring smoke blowing up their arses.)

Two years ago, two particular incidents occurred which stick in my mind. In one round of judging there was a bloke in a faded Pink Floyd t-shirt and mullet who decided he was the table’s expert. He told us he’d been working at beer festivals for thirty years etcetera ad nauseam. He pronounced his verdict on every beer we encountered. I soon realised he was the taste equivalent of cloth-eared when he failed to mark down a beer that was strongly diacetyl.

The second incident was similar. In a later round a chap in a brewery polo shirt (it transpired he was an employee of the brewery) appointed himself as the table loudmouth. In a round of worthy-but-dull pale golden beers we encountered one with a vivid fruity flavour. Mr Loudmouth announced “that’s my kind of beer” (his exact words stick in my mind.) Unfortunately the flavour was the unmistakeable (to me) green apple tang of acetaldehyde.

In both cases I could see the more inexperienced judges re-assess their scores upwards after hearing the mistaken praise from the know-alls. There was one very nice couple on the second table who had won a “win a day judging beer” competition in a pub. I watched them do what a lot of inexperienced judges do – picking up clues from others before forming their opinions. They marked the acetaldehyde beer highly.

I didn't judge last year but this year I was persuaded to judge again by Jon Kyme of the wonderful Stringers Beer of Ulverston. It wasn’t without trepidation that I travelled to Manchester.

My trepidations and reservationswere warranted. I found myself on a table judging premium bitters (if I recall correctly.) We encountered four or five worthy but dull beers. No glaring faults but nothing very interesting. Two of the beers were more vivid,  possessing a hop character that suggested an American influence. They were both well-executed and intelligently conceived without being mouth-puckeringly bitter. A fellow judge declared “they smell, my customers wouldn’t drink them” and presented a histrionic grimace. Her body-language was as if she’d been offered a dog-shit sandwich. As the loudmouth of the table, more timid, inexperienced judges lapped it up and the beers scored badly. I was tempted to offer a spirited defence of the beers and a critique of her ability to judge beer, but I felt it not worth the risk of the red mist descending. I kept schtum.

In a later round I found myself once again on the same table as this Bet Lynch. Amongst the array of worthy but dull beers there was one that was strongly diacetyl. Junior Bet Lynch commented "Ooh, that’s nice, it’s kind of…” Her words tailed off and I filled in "butterscotch.” "Yes, that's it!" This time I had to explain that the flavour and aroma of butterscotch was undesirable. The existence of the unmistakeable oxidised flavour of damp cardboard in the finish confirmed to me this beer was fucked. Bet Lynch deferred to my superior wisdom. Wise.

I insisted the beer was off. Our runner took away the offending jug of beer and returned with a replacement. The runner told us that the brewer of the beer was helping behind the scenes and that he had spotted his own beer being returned. The brewer was told his beer was suspected of being off. We were told the brewer had responded “it’s supposed to be like that – malty.” Our second jug was similarly off. I gave it a rock-bottom score. I don’t suppose it scored highly with my fellow judges, but I might be wrong.

On my final table of the day I found myself judging with a large domineering chap who worked for a brewery, and a couple of his chums. Mr Bellicose chose to chunter audibly through his judging – "eeh, I couldn't drink six pints o' that" or "I could drink that all night" and scored the beers accordingly. The beers he favoured were the blandest; the beers he marked down had more flavour. "Well Mr Bellicose," I wanted to shout, "we don't give a fuck what you would choose to pour down your fat ignorant neck."

After the judging I was talking toWill France of the marvellous Port Street Beer House. He made similar observations about conspicuously bad judging. As manager of a trendy crafty beer catwalk outlet he knew his customers wouldn’t be too impressed by an insipid 3.8% session beer, just as Bet Lynch’s customers wouldn’t be impressed by a 6% hop-bomb. 

In one corner there are judges like Will and myself: familiar with a wide range of styles of beer, knowledgeable about common off-flavours, unperturbed by big flavours or high strength. In the other corner there is Bet Lynch and Mr Bellicose: their judging defined by narrow parameters of acceptability.

This raises a question: which kind of judge better serves SIBA’s desire to award prizes to the best beers by its members? Will France and I or Bet Lynch and Mr Bellicose?

It’s not hard is it?

The profusion of the the Bet Lynch and Mr Bellicose type of judge is letting down SIBA. SIBA’s policy of turning beer judging into a jolly for all-and-sundry is lowering the standard of judging and undermining the validity of its competitions. The results may well reflect the views of many ordinary drinkers, but ordinary drinkers aren't necessarily capable of making sound judgements.


P.s. To whom it may concern in SIBA: biscuits for cheese are NOT suitable palate-cleansers. Sugars and fats of Hovis biscuits and TUCs have the opposite effect.


Ed said...

Interesting, I've never judged at a festival myself.

I'm in two minds though, I quite like the idea judging being done by a mix of people with "sophisticated" palates like yourself as well as "ordinary" beer drinkers.

Baron Orm said...

It's funny that the only two Manchester SIBA events that I've judged at are the same two that you mention. At the one two years ago I was very much a n00b when it came to judging (it was my first time) and so I looked to the table for how it's done.

Luckily I was with Rob from Fyne and a lady (can't remember her name but she was big in the beer scene) both of whom had been judging for years and gave me some great tips.

This year was the opposite, I've tried a lot of beers (mostly bottled) but I'm no expert but I could see people using my comments as a basis for their scores.

Only when I got to a table in round three did we have a few people who had firm ideas, luckily I was happy enough with my judgement to not be swayed this time round.

I think the mix of experienced and new judges is a great idea, I just wonder how SIBA can affect the experiences that we describe above?

P.S. I really wanted some cheese... ;)

beersiveknown said...

waterbiscuits should be sufficient.

Last year at a festival I was helping at a notably hoppy beer was about to be marked down because it wasn't clear until I pointed out it was hop haze. It went on to win beer of the festival.

I think for "beer of the festival" type awards its perfectly acceptable to have a mix of experiences on the panel, but for more prestigious awards there needs to have been at least a basic training undertaken

Erlangernick said...

Sheesh. You lot could take a lesson from a well run homebrew competition over there in Yankley.

StringersBeer said...

Oh, now I feel bad. Sorry Jeff.

StringersBeer said...

Oooh, "the brewer of the beer was helping behind the scenes and that he had spotted his own beer being returned." That certainly shouldn't have been possible. It's more likely that the cellar management team were able to ask the brewer if "he" was happy with the beer being served as it was. Although I'm not sure that this should really have been happening during the competition. Since there's a risk of it un-blinding things.

The runners do a great job and don't get to see the beer being drawn / poured - they'll have no idea what they're carrying. Even the extraordinarily good-looking and talented regional trustees are strictly prohibited from the cellar area during the competition, and not allowed to see the jugs being filled.

Cooking Lager said...

If it is not fit for purpose, what is the purpose?

If it to pick beers which might sell, then asking regular drinkers what they think might be a better approach than asking beer experts.

StringersBeer said...

As it happens, "saleability" is one of the judging criteria, so I think we can assume that it's not the only / primary purpose. Do you want I should get you on the list for next year, Cookie?

Cooking Lager said...

My question was not really of of criteria, but one of the overall purpose of judging beer and giving awards. With that I think you can then question if the criteria match the purpose or even if those doing the judging are fit for purpose.

Clearly Jeff is frustrated by some that appear to be judging, I am not sure my presence would ease that frustration or exacerbate it.

StringersBeer said...

I don't know about the "overall" purpose of beer judging and awards, but we go to the trouble of inviting these judges because we value the opinions of all of them, not just the loudest-mouthed and not least those who are doing it for the first time. What Jeff's pointing to is more to do with the dynamics of small groups than it's about this competition in particular. Of course, knowing what we do about group dynamics and the propensity of empty vessels to make the most noise, we should organise the judging to get the best out of these kind people who are giving us their time for nothing.

Unknown said...

Sorry, late to this, but I do have some thoughts, which I'm not entirely sure how to write without me sounding like a disgruntled brewer, which I'm not.

Firstly, I agree that an element of popularity should influence the outcome. Some people actually like diacetyl, who are we to say they are wrong?

But, I agree that most of the competitions I've actually been involved with have left me feeling that they are not as thorough as I'd like them to be.Even the ones where we won something.

Last years SIBA North competition was most certainly not double blind. I think that this is the very first step to making sure a competition is meaningful. It sounds like this year's was no better.

There is also a variability of seriousness applied to various events. I actually think CAMRA is better than most, although there tends to be problems there too.

I broadly agree with the sentiments here, which is a major reason we didn't enter anything this year, and generally have become disinterested in competitions for most of the reasons given. In particular, training on ensuring that "group think" doesn't effect the judging, which is the key problem Jeff highlights, is absolutely essential.

StringersBeer said...

Dave, I can assure you that the competition was pretty well blinded. Unless there was active cheating going on (and I'm sure you're not saying that). The organisers, SIBA secretariat, and the CAMRA volunteers are committed to ensuring that neither the runners nor the judges can know (as far as possible) what beer is being judged.

On the key biscuit issue: Our specific buscuitary needs were communicated to the venue well ahead of time. We were as shocked as anyone else by the selection which appeared. Urgent and strenuous efforts were made to score a suitable supply of water biscuits which I believe made it to the tables in the later stages of judging. My personal favorite, the "Carr's Table Water Biscuit" (or "Cracker" as I believe they're known in "Yankley") should have been there from the start. A ball was dropped, mistakes made, lessons learned.

On the secondary question of judging: I personally believe that there's a certain amount of work to be done on competitions. It's been argued that there's a serious issue with SIBA competitions in that organising the competition facility (venue / staffing / beer logistics) is a regional responsibility, while the details of recruiting judges / directing judging / scoring etc are down to SIBA central Secretariat. Unless you're in the South West, I'm told, where things are a little different.

The SIBA competitions cover keg and bottled beers, but this competition is mainly for cask beers.
A cask beer competition is enormously different from other competitions. And this competition is very different from other large Cask competitions. At this comp, 10 tonnes of entries had to be got to 140 judges in the space of a few hours. It's not the same as having a sniff of a few hundred bottles of homebrew.

If it was just down to me, both the format and the judging would change rather. There will be changes in SIBA competitions, I hope that they will be significant enough to confirm the credibility of the awards made.

I don't know how interested you are, Jeff, in the kind of folks who rant on "BA" but there was an interesting thread which touched on SIBA comps here:

Unknown said...

Jon, I'm certainly not suggesting that active cheating IS taking place, but in my view there are not enough safeguards to prevent it. That is, unless it's changed since last year. It's not double blind, therefore runners could influence judges.

The fact that a brewer was able to get his thoughts back to the table shows that something is amiss.

StringersBeer said...

A great deal of attention was given to the issue that you (and others) highlighted last year, namely that some of the runners were placed in the position of knowing which beer they were carrying. We stressed this to the volunteers who went into the cellar to fill the jugs and then passed them on to the runners (who were prohibited from the cellar). I'm assured that this "deblinding" was impossible this year.

You'll agree that a simple message being passed (as hearsay) from a brewer to the table (as long as it doesn't identify the brewer) doesn't unblind the judges. I can't help feeling that the runner supplying Jeff's table had misunderstood the situation and was, in any case, saying considerable more than they oughta.
This is certainly something I'll be raising.

Whatever, I suspect that the competition hasn't really kept up with the scale and variety of our members' offering, and a bit of a rethink might be in order.

Unknown said...

Ah, I see, so things were changed this year. This is good.

It was indeed the case last year that "extraordinarily good-looking and talented regional trustees" and even you and me were allowed into the cellar unimpeded. If this has changed them I am very pleased.

StringersBeer said...

Be pleased. It has.

Anonymous said...

So as a member of Siba I can rest assured that if a beer I brew comes last it means nowt oop north.

Boat Inn

StringersBeer said...

Er, No?