Rather than bike shorts and jerseys for tours, I prefer clothes that are comfortable on or off the bike and that fit into the local scene. I've settled on riding briefs (available from Performance or Nashbar) that give the same riding comfort as riding shorts, worn under nylon zip-off pants. The pants are good for hot or cool temperatures, have pockets, wash easily, work as a swim suit, and with regular skivies, are fine for non-riding times.
Check the baggy mountain bike shorts before buying; many have thick padding that feels like a diaper.
For shirts, I use anything that's bright(for visibility) and not cotton (for ease of washing/drying) -- tee-shirts, button-up, even aloha shirts when the mood fits.