Agree - the life expectancy of a rechargeable battery has gotten better, but 5 years is pretty good. If expensive, a replacement battery is the way to go, if it can be done without disrupting the electrical shielding for the speaker. In the case of your $30 speaker, I'd recycle the old one and buy the new technology; the cost of procurement and installation of a replacement battery is too high.
This thread is a reminder that wind noise from cycling can take a toll on our hearing.
I recall that wind noise at 15 MPH is good for 85 decibels (dBA) - pretty loud and sufficient to cause damage.
As a musician, I've been sensitive to the noise issue. Since the 70s, I have routinely sported ear plugs while training. In my dotage, as I try to avoid eventual need for hearing aids, I've added noise cancelling headphones with microphone ($25) for downhills, headwinds (which always seem to be more prevalent than downhills), and the rare phone call or need for a tune to keep the cadence up.
I'll note that a lot of the fellows with whom I ride motorcycles prefer custom fitted, noise cancelling ear plugs, but I always found the sweat on a bicycle to limit my comfort with this rather expensive soluton.
(And like a lot of us, I just can't keep those expensive little earbuds in my ears if I start to grimace on an uphill - no matter how well they are fitted while relaxed. Right - nice thing about a speaker attached to the bike - it's not going anywhere.)
That's not to say I don't occasionally carry a high quality speaker for some mood at day's end. I simply recommend one consider whether they want to add to road and wind noise while riding.
(And before the safety police jump in, consider that with a 30 dBA reduction in noise, I still have plenty of room to distinguish between a Jeep and a Chevy coming from behind - which I confirm with my large-ish helmet mirror - now a must for detection of smaller EVs.)
Ride on.