Is it annoying because it is loud?
Experiments on other noise sources show that louder sounds are more annoying. Our hearing is an early-warning system: we listen out for sudden changes in the sounds around us which might indicate threats, and ignore benign, persistent noise. When noise becomes as loud as a vuvuzela, however, it becomes impossible to habituate to the sound.
What else about the sound makes it annoying?
The droning quality makes it more annoying – the fact it has a distinct pitch or note. Investigations into many noise annoyance problems have demonstrated this. Indeed some noise standards and regulations have corrections to allow for the additional annoyance from such sound. Droning sounds are harder to ignore and more alerting than broadband noise such as the hiss of a badly tuned radio. This might be because tones can carry useful information in the vowel sounds of speech. But it might also relate to threat detection – because predator sounds like a lion's roar has tonal components – but I'm speculating.
\n
The worst thing about these cacophonous horns, perhaps, is that they drown out all the festive football chants and songs. If you want to hear those, by the way, check this site, which has tons of recordings. The chants for the Aussie socceroos are especially fun.
Image: Vuvuzela Day, a Creative Commons Attribution (2.0) image from dundasfc's photostream