Touring bikes generally have front geometry similar to a road bike so they don't feel luggish when loaded in front. So they will steer briskly when used unloaded. But with the stiff and heavy frame, and long chainstays, you might miss some of those ethereal qualities like 'liveliness' that people use to describe bikes. I have a Long Haul Trucker with road wheels that I use for most of my riding around town and for club rides in the mountains. It is a wonderful and comfortable bike, but it just doesn't display an eagerness for speed. I somewhat unkindly describe it as the Ford Taurus of bicycles; It'll get you there, but with no panache.
A good choice for an 'everything' bike might be a cyclo-cross bike. They ride pretty briskly and are heavy enough for touring. I've toured on a Bianchi Volpe and another Bianchi cross bike and I see tourers with Surly Cross-Checks. The cross bike should have the same brisk steering with a little livelier frame. You'll also probably have a slightly higher bottom bracket and shorter chainstays ( and consequently wheelbase), both good traits for zipping around town.