You pay the going rate if you don't have the time or ability to shop competitively. Last October we paid $4.00/gallon for regular gas in California while it was only $1.78/gallon in Tennessee. However, since I needed to fill the car right then, I couldn't wait to take advantage of the potentially lower cost.
It's a long walk from CA to TN and back to save a few $ on gas. I'm not sure it's an appropriate comparison to the OP's situation.