There are a number of different options if you only want to carry a small load. Perhaps the most popular one now is a seatpost rack with a small trunk. These are very easy to find and can generally carry about 20 lbs. They work best with trunks that are not very tall; the new Arkel trunk looks perfect for them. There are number on the market and I can't say for certain which one is lighter than the others. Make sure that you are using a normal seatpost (not carbon) and that the rack you buy fits your bike. Some racks will push the trunk into the back of your seat or set it on the brakes. With a small load, they work well and do not effect handling significantly. They will become noticable if you like to stand a lot. I have used these to great effect on shorter trips with my ultralite load.
Another option is to use a transverse saddle mounted bag (Carridice bag). These can carry all your gear and come in many sizes so your not stuck with a bag twice the size that you actually need. You saddle most liekly does not have the loops needed to mount on of these, so you will also need a seatpost mounted bag lifter that has the needed loops. These are generally larger than the rack and trunk, but are less secure and may brush against your thighs while riding. Rivendell, Harris Cyclery, and Wall Bike all carry these bags.
A third option is to look back to the French bikes and to use a small fork mounted rack with a small bag. The hard part with this is finding the rack and finding a spot to mount it on a modern bike. The old bikes used to mount these to the brake bolts when they used sidepull or cantilever brakes. If you can put it all together, it is a very elegant solution because it is very secure, will not effect handling (assuming a light load), and is easily accesible while riding. Momovelo is probably the best place to start looking for these. The rack will also be lighter than a seatpost rack.
Of course, you could always carry your gear in a small backpack, also.
Speaking about weight, one of the things I like about bicycle touring is that an extra pound or 10 has alot less affect than an extra pound while backpacking. When touring using the "Ray Way" I do not go significantly faster or use much less effort than I do when I tour fully loaded. Instead, I tend to lay in my sleeping bag thinking about the chair and big thermarest that I left at home. A light bike is nice in cities and when sightseeing, though.