Shimano Nexus Teardown.

[zerodish]

There are many variations of this hub so this may not apply to every variation. First of all the axle is 3/8 inch and is flattened. The flattened portion is to keep the axle from rotating. This is not robust enough to allow for serious torque. I clamped down on it with vise grips and tried to rotate it and the axle spun in the vise grips. There is no way in hell any bicycle can prevent the axle from spinning if it locks up. I would say this would destroy an aluminum frame the first time it happens. In the past 11 mm axles flattened to 10 mm have been made for tandems and Bullseye made 12 mm axles flattened to 10 mm. Even with these I would not rely on the axle and would insists on a reaction arm like coaster brakes used.  It was difficult to get apart you will need either a 17 mm or 18 mm cone wrench. Shimano is not going to sell you a cone. The hub was packed with grease that had dried out. The cones and races had pitted. They like to imply the hub is weather sealed but actually it is worse than Shimano's regular hubs. Don't buy.

[canalligators]

It looks like you’ve had an extreme experience with an abused hub.  If the hub is locked up, I agree that the flats and keying washers won’t hold the axle in place.  It shouldn’t be expected to.

In normal service with a reasonably maintained hub, the axle flats and keyed washers work fine.  Nexus and Alfine hubs enjoy a reputation as reliable, durable components, and in my experience, it is deserved.  I don’t think that its failure to work when the internals are gummed up that badly, is a reason not to use one.