How do you measure how good someone is at predicting binary outcomes? Let's say the person predicts what will happen (A or B) with a certain probability each time. An obvious measure is the average probability that he assigns to the outcomes that happen. But this is complicated by the fact that he might assign a lot of events a probability close to 1/2, therefore his total average will be very close to one half, even though on the events he was confident about he was mostly right.
Is there another measure to quantify his predictive ability?