Return to Writings
ISU Communication 1327 — The OAC Commission
by Sonia Bianchetti
The ISU has recently sent out Communication Nº 1327 - ISU Judging
System, Evaluation of judging and technical content decisions.
this communication, the ISU explains the rules in detail and gives the
guidelines on the procedure to be followed in the "assessment" of the
judges' activity, which differs from the procedure stated in the
Reading the text, I must confess that I was impressed by
the extraordinary ability of the ISU experts to make things as
complicated and expensive as possible, with very uncertain results.
will go through the text of the communication and make some comments
based on my knowledge and experience in the sport, and particularly on
the matter of the judges' "assessment".
A) Identification of the Officials' Assessment Commission (OAC)
The OAC will be appointed by the ISU Council and shall consist of a
pool of ISU Referees and Judges for Single and Pair Skating,
Ice Dancing and Synchronized Skating "having been trained in the
ISU judging system and having attended the Frankfurt Seminar in July
2005". They shall evaluate evident anomalies based on
predetermined mathematical criteria confirmed by the Council and shall
prepare a report for each competition including all identified cases
of serious errors and/or bias by the judges. After each
segment (e.g., Short program, Free skating, Compulsory Dance), the
assigned OAC members on site will receive the necessary printouts that
include the Grade of Execution (GOE) of every element and the points
of the Program Components from all judges in a "random sequence
without any reference to specific Judges' names".
Two questions come to mind:
- What kind of experience or competence can we expect from an ISU Judge who has perhaps judged a couple of events in the past season and has attended one seminar? Can the judges feel comfortable that they will be "judged" correctly?
- How can the OAC members understand if an "anomaly" is an error or national bias without knowing the name or the country of the concerned judge?
B) Appointment Procedure for ISU Championships and Senior Grand
Prix of Figure Skating Events/Final
OAC Pool members must have the
following skills relating to their OAC work:
a. ability to analyze competition data; ability to work quickly and in an organized fashion;
b. good written English;
c. be familiar with report writing;
d. demonstrate an ability to remain objective in all
officiating evaluation matters;
My questions are: who is responsible for checking that the candidates really possess all these skills?
2) Each Single and Pair/Ice Dance ISU Championship or Grand
Prix of Figure Skating (senior) competition shall be attended by at
least three assigned OAC members, two of which must be
available for Ice Dancing.
Will they be asked orally or in writing to confirm that they meet
these requirements, and will this be considered satisfactory and
enough by the ISU Council? But what I like best is that the candidate
"must demonstrate an ability to remain objective in all officiating
I would really appreciate if somebody in the ISU could explain to me
how anyone can demonstrate this skill in advance. Will they ask the
candidate if he can "remain objective in all officiating evaluation
matters" and expect that, if he cannot, he will reply: "No sir, I am a
This requirement can be justified by the need of the ISU to limit the
costs but is a disservice to the sport.
All one needs to do is to take a look at the ISU list of judges to
- around 20% of the Figure Skating Members do not have ice dance judges at all;
- more or less 50% of the Members having judges in both disciplines do not have any judge who can act both in figure skating and ice dancing (Great Britain, China, Estonia, Israel, France, Finland, Hungary, Italy, Sweden, and Ukraine, just to name some);
- the Judges who can judge both disciplines represent not more than 20% of the judges of each Member, and often they are not the best experts in either one.
The requirement that 2/3 of the OAC members must be qualified to
judge both disciplines limits significantly the possibilities of
choice and there are no guarantees that the referees or judges
selected are really the most competent, knowledgeable, experienced and
honest on the list, as it should be, even if this is more expensive.
C) Assignment of OAC members for ISU Junior Grand Prix of Figure Skating Series and World Challenge Cup for Juniors in Synchronized Skating
Again, sadly for our sport, this appears to be case of a complete lack
of knowledge regarding the best and only accurate environment in which
to evaluate a performance, when the ISU states that "The ISU President
will assign the necessary number of OAC members to perform the
evaluation (of judges) based on the available printouts and based on
video tapes (DVDs). The assigned OAC members will perform this task at
their respective residence".
Now we all know that watching a performance from TV or tape/DVD can
give at times a different view of a performance for the simple reason
that many of the requirements listed under Program Components cannot
be correctly evaluated from a screen!! Pity the poor judges that have
to be assessed by this method.
Yet for ISU Championships, the ISU states "it is recommended
that as may be physically possible in each event, that the OAC panel
is seated as close as possible to the same site line as the judging
panel". By making this stipulation, they are admitting that the
only place to sit and have any fair chance of making a correct
evaluation of a performance is in the rink, at the same sight line as
the judges—and strangely enough, not in their own homes. This
inconsistency leaves me speechless!
This ridiculous idea is no doubt born out of a lack of knowledge plus
a desire to save money. Therefore, perhaps the next money-saving idea
to emanate from the ISU will be to have all the Grand Prix
competitions and ISU Championships judged from a live TV link-up in
the judges' homes!
D) Judges Evaluation at Other International Competitions
The Council has decided that there will be no individual assessment
of Judges' marks at International Senior/Junior competitions other
than the Grand Prix events. The Referee will file a formal report
reviewing the judges' activity in regard to ethics, behaviour,
attendance at Round Table Discussion, and other generalised reporting
areas—but not judging! The reason behind this unbelievable
decision is that in such competitions there will be no guarantee for
the availability and consistency of technology as available at ISU
events (e.g., video replay, DVD analysis, on-site OAC personnel);
there will be no checks and balances evident to assess these judges'
I wonder if anybody in the ISU has considered the impact that such a
decision will have on the whole judging system, aside from being an
insult to the hundreds of skaters who will take part in these
competitions. With this decision, the ISU has officially stated that
they could not care less about how these events are judged! Based on
my experience I can affirm that this is just another disgraceful
decision of the ISU
These so-called "second class international competitions" have always
been and are the lifesblood of the sport, not only for the skaters but
also for the judges. Without them the sport would die. They are the
grounds where the skaters and the judges grow and develop their
experience. Nobody is born a champion or an experienced judge, but if
we cut the roots out there will be no champions or judges in the
Unfortunately, not all ISU Members have a well-thought-out education
program for their national judges; therefore, in some cases the
technical committees had to trust the Members when they first
nominated judges for international competitions as to their level of
knowledge. In the former comprehensive evaluation system, the weaker
judges were soon identified. But now, the whole educational system
of the judges, which has been so successful so far, will collapse if
there is no monitoring of the activity of the judges in these senior
and junior international competitions. How will the ISU Technical
Committees know which judges are good and competent enough to become
ISU Championship judges and meet the requirements to pass the Judges'
In the past, all International competition judges were assessed at
every event, and had to act successfully at a stated number of
international competitions over a certain period of time to be able to
participate at the ISU Judges' examination, but now this
well-thought-out educational and monitoring/assessing of the Judges
At present, the ISU is still profiting from the hard work carried out
over the last 50 years by the Technical Committees to instruct and
train the judges to guarantee the best possible quality of judging to
the competitors at the highest level. They can avail themselves now
of a lot of good judges who were educated in the past. But in the
future, without all this background work, where will the best judges
As with the New Judging System, it seems that nobody in the ISU
leadership cares about the future of the sport. "After me, the
deluge" seems to be Ottavio Cinquanta's motto.
The ISU states that since there will be 6 Grand Prix events and
four ISU Championships reviewed by the OAC, the "activity of most
judges in the season would 'enter the ISU radar' and come up for
review in at least one of these events". I may be wrong, but I
sincerely doubt that most of the judges who will officiate in
the 29 international competitions (other than the Grand Prix events or
the ISU Championships) organized in the next season will also judge at
least one of the major events. Normally the judges selected by the
Members for these prestigious competitions, with so much money
involved, are the best experienced judges—not beginners! This is
nothing new; it has always been like this. And, in any case, one
event is not enough to decide on the competency of the judge.
E) Mathematical Criteria to Identify Potential Anomalies
The system is very simple—and very simple minded— at the
same time. For each Judge the computer calculates the deviation in
elements, which is an absolute value of the difference between entered
GOE and the trimmed mean of the GOEs. The total deviation points in
elements is the sum of the deviation of the individual elements.
A general "corridor of an acceptable number of deviation points" is
given to each individual Judge. The "corridor" is based on the number
of elements performed, e.g., in the Short Program there are 8 required
elements to perform. Each judge may vary in his/her decision with one
(1) step as an average per element, which are 8 steps in total for the
Short Program (8.0 is the deviation points maximum). More than 8 is
considered an anomaly and might get an "assessment".
For the five Program Components, the Judges' "corridor" will be 15% of
the maximum total points of 50 points, which means 7,5 deviation
points. If the Program Component scores of the OAC members is located
outside the "corridor", the "corridor" can be enlarged accordingly.
Of course, this proposed enlargement of the famous "corridor" still
allows marks that are wrong, but in the original "corridor" before
enlargement, to stand unchallenged. Unfortunately, over the past
three years, the ISU has had the ill-informed philosophy that the
majority, or "corridor", will always be correct. This system sets us
back 50 years, when "to be in line" was the goal of all the judges and
it was a tragedy for the skaters who were judged on reputation or on
the results of the year before.
we witnessed the same situation at the major ISU events last year.
The judges do not judge any more! They must return to the system when
a judge that was "out of line"—now known as an "anomaly"—could
possibly be the only correct judge, and the others who were "in line"
or in the "corridor" could possibly be wrong—or the instances of
poor judging we all witnessed last year will continue unabated.
To prove how wrong this philosophy is, I can quote a case that speaks
for itself. In the past season, a well known American judge who is
probably one of the most educated judges in the evaluation of the
Program Components received an Assessment of being out of the
"corridor" in his specialty! Knowing the person, by instinct, I would
guess that he might well have been the only correct judge in the
At the 2005 European Championships in Torino, from the top skaters to
the very last one in the classification list, both in men and ladies,
the GOE for a given element was more or less the
same—0—all the way through. For the GOE points assigned by
the judges to the triple Axel jumps in the men's event in Torino, for
instance, except in case of a fall or a particularly severe error for
which a deduction had to be applied, they were the same from Plushenko
down. But this was not correct! The difference in the quality of the
jumps was there as bright as the sun! And how is it possible that
there were so few elements deserving pluses? Plushenko skated an
outstanding program; nevertheless, the only +2 and +3 were awarded to
his step sequence. What about all the other elements that were
excellent as well? Are the judges really judging? What are they
looking for to give a +1 or a +2?
Well, the explanation is simple: they are wisely and safely giving the
same marks to everybody to make sure to be within the famous
"corridor". On the other hand, this problem was raised and discussed
during the last season. Many judges and officials were of the opinion
that this assessment system is wrong and must be changed.
But the ISU is of a different opinion and Cinquanta has proudly
announced in a press release that the ISU Council is pleased with the
second season using the ISU Judging System. In particular, the ISU
Council highlighted that the System has introduced a new level of
accountability through the Officials Assessment process and: "Contrary
to previous seasons, the evaluation process and Assessments were
monitored as the season progressed rather than at the end of the
season. Judges were informed of specific deviations which caused an
Assessment to occur, this proved to be a key factor contributing to
error prevention and education throughout the season. The number of
Assessments declined steadily throughout the season. The vast majority
of 'assessed' Officials were subject only to an Assessment 1 (1st
Warning), which showed the deterrent effect of the system".
To me, the "deterrent effect of the system" seems just the
opposite of what Cinquanta claims. The Assessments declined steadily
throughout the season after the judges were informed during the season
of specific deviations which caused an Assessment to occur, not
because the judges improved in their judging but because they
understood what they have to watch for: stay within the "corridor"!
And this is a tragedy for skating.
G) On "Assessments" the ISU Communication reads:
"If an 'Assessment' is deemed necessary by the OAC and the
Technical Committee, the 'Assessment' will be communicated by the ISU
Secretariat to the Officials and the Members concerned without delay",
and: "In the case that an accumulation of Assessments (at present when
reaching 'Assessment 4'), results in the demotion or suspension for
the Officials concerned, they shall be notified by the ISU Secretariat
of the possible outcome. The Official will have the right to ask
within 5 days upon receipt of the notification for a meeting to be
held as soon as possible, to give his/her explanation to the relevant
"Assessment"—in front of at least 3 members of the respective
Any travel, board and lodging or other expenses
incurred by the Official concerned relating to the explanation meeting
will be for the Official's account if all the 'Assessments', despite
the explanations received, are confirmed by the respective Technical
Committees after the meeting. The ISU shall only reimburse such
expenses if at least one of the 'Assessments' would be revoked".
I quoted the whole text because, again, to me it is beyond belief.
a) The ISU decides that an "Assessment" is deemed necessary, and
informs the concerned official of the reasons why it was given, but no
explanation is asked of the concerned judge, who might be right in his
deviation from the "corridor".
b) Only when the concerned Official reaches "Assessment 4" (which
might occur after two or three years for an accumulation of
"Assessments 1"), and is going to be suspended, is the judge
generously offered by the ISU the possibility of being asked to
furnish his explanation (within the time limit of 5 days!), not in
writing, but in person by travelling from somewhere, maybe on the
opposite side of the world at his own expense—not to meet a
neutral committee, above the parties, but the same three Members of
the Technical Committee which had decided to impose the sanction!!! I
wonder how many judges will travel from Japan or Australia, the US or
Canada, or even simply from Moscow to meet the same panel of people
that condemned him, hoping to get relief! And what kind of
explanation can a poor judge give, maybe three years later, of the
reason why, in the short program of skater Smith he exceeded the
maximum points deviation of 0.2?
Perhaps the ISU should ask the Wige - Data experts to invent another
of their marvellous mini computers—to replace the brains of the
judges while they are officiating. In this way, judging will be
perfect and there will be no "deviations" from the corridor any
longer. Then all judging problems will be really solved once and
forever, and this idea, if taken on board, could be another "miracle"
from the ISU President.
This article , as well as some interesting comments, can be seen on