My first tours were with rear panniers alone (Ortliebs). And a roll on top. With that very simple and basic setup my bike handles completely stable even at 45 mph descents.
A while ago I did another tour in a more cold climate: It was necessary to carry a heavy sleeping bag and cold weather clothing. For that reason I got the Ortlieb front panniers and the rock solid Tubus front rack. From day 1 and the next 30 days the bike behaved badly. Lots of vibrations and flexing of the frame especially on descents when riding faster than approx 18 mph. I had to literally brake on the descents. I tried all kinds of variations with the panniers: Shifting weights etc but nothing helped.
My next tour will be with my trusted Ortlieb rear panniers and a roll on top - just like in the old days :-)