The bridge across to Astoria can be a challenge but is not as difficult as people say. Just take your time and ride when traffic gets lighter. You don't really need to buy any special books or maps for Oregon either. The very good Oregon Coast Bike Route Map is free and is all you will need. Go to the following site http://www.oregon.gov/ODOT/HWY/BIKEPED/Pages/maps.aspx#ODOT_Maps Click on the link to order your hard copy. Or you could probably get the map at a tourist info center in Astoria.
Further down the coast the Coos Bay Bridge can be a bit of a problem but a local man has posted an alternate route you may want to look at. Check the following link. http://www.crazyguyonabike.com/doc/?o=1&doc_id=12467&v=2h
For the California part of my trip down the coast I used the ACA maps and they were excellent.