I've never heard of a commercially available aluminum pannier, so maybe you will need to make your own. It might be easier to buy a commercial pannier and add aluminum reinforcement at places where you feel it might be necessary.
Although most panniers do contain plastic parts, most of the pannier is not plastic. They are typically a coated fabric. Yes, sometimes the plastic hooks or brackets might break, so you could replace those with metal if you want. Aluminum, however, isn't very strong either, and it can break. A high-quality plastic would likely be stronger than aluminum. So if I was going to replace hooks and brackets, I might opt for a stronger metal.
I've been touring with the same Ortlieb panniers for over 20,000 miles, and absolutely nothing has gone wrong with them. Some people, however, carry spare hooks. It might be easier to just go with a commercial pannier and carry some spare parts if you want additional protection.
If you really do want an all-aluminum pannier, you'll probably need to make your own. Don't use too thin of aluminum, however, or it will probably fail. And you'll have to coat all joints and fasteners carefully to make the panniers waterproof. I hope you come back here and post some information and pictures when you get them made.
