No ancient authority has left us a clear account of how the judges at the Athenian dramatic festivals operated. We can therefore never know for certain what happened. But it may be possible to improve the reconstruction normally given, which does not look as if it could ever have yielded acceptable results.
One thing that is very clear (from Isocrates 17.33–4) is that the choice of judges was taken seriously. Not only did it involve the Council, the Prytanies, and the Treasurers, but any tampering with the panel from which the judges were eventually selected seems to have been punishable with death. Even the physical arrangements were quite complicated. Names approved by the Council were deposited in ten jars, one jar for each tribe but each jar containing several names, sealed by the Prytanies, held in safe-keeping on the Acropolis, and eventually brought down to the theatre, where the archon publicly broke the seals and drew out one name from each jar.
At this stage of the process there emerged ten judges, one from each tribe. Their quality had been guaranteed by the Council's prior vetting of the panel, but their actual names were unpredictable because randomly selected from it. All had been done democratically and with equal representation of tribes, in perfect accord with normal Athenian practice.
But now comes the difficulty. The paroemiographers record a proverb (or rather perhaps a joke since it seems to be a kind of parody of a Homeric verse) about judgement being in the lap of five judges — ⋯ν π⋯ντε κριτ⋯ν γούνασι κεῖται; Lysias tells us of a judge whose vote was not counted; and Lucian, who likes to disguise a very careful antiquarianism behind an apparent casualness of style, says that at festival competitions the many know how to clap and hiss but that judgement is in the hands of seven or five or however many.