The annual earthquake predictions of the China Seismological Bureau (CSB) are evaluated by means of an R score (an R score is approximately 0 for completely random guesses, and approximately 1 for completely successful predictions). The average R score of the annual predictions in China in the period 1990–1998 is about 0.184, significantly larger than 0.0. However, background seismicity is higher in seismically active regions. If a ‘random guess' prediction is chosen to be proportional to the background seismicity, the expected R score is 0.123, and the nine-year mean R score of 0.184 as observed is only marginally higher than this background value. Monte Carlo tests indicate that the probability of attaining an R score of actual prediction by background seismicity based on random guess is about . It is concluded that earthquake prediction in China is still in a very preliminary stage, barely above a pure chance level.