I've used both. About 15 loaded days with the trailer, and about 70 loaded days with the paniers. I started out with the BOB attached to a mountain bike. It handled very poorly downhill, especially when heavily loaded. The front end was light and wobbly feeling - kind of swoopy. In fact, I crashed it (or it crashed me) - got a wobble going downhill at speed, and it highsided me. I then switched to front and rear paniers on the same bike - boy what a difference! It just felt like a normal bike, only heavier. I am definitely a fan of paniers. The organization is a little harder at first, you have to balance the load front/rear and side to side, but everything seems to find it's home after a few days, and then you know where everthing is. I don't have waterproof paniers, just regular ones that I put dry-bags or trash compactor bags inside.
Agree with the drafting comment - you can't get much of a draft off a companion who is towing a trailer.
Plus, the trailer is a total pain to park. YOu can jack-knife it, but take a look at the lateral strain that puts on the rear dropouts. Ick.
Also, the attachment system with the little cotter pins is way lame (they may have fixed this in later-model trailers?) - the pins like to fall out, and the trailer likes to get part-way detatched, which is terrible for the rear dropouts, not to mention dangerous.
Oh, and another thing - flat tires on the trailer seem pretty common, and you have to carry a different size spare tube (and spare tire, if you do that).
I'm really liking my Trek 520 with racks and paniers.
RE: side wind issues - well, yes, with the paniers on the bike you do get pushed around by side winds, but i wouldn't trade that for the full-time bad handling of the trailer.
Anyone out there? If so, see you on the road - I'm northbounding the pacific coast from santa barbara to santa cruz starting tomorrow.