I've spent over 40 years balancing my passions for family, biking, hiking and climbing. When the kids are very young, it's tough. There were times a 45 minute ride or quick hike or ski was all I could get after work. (In fact, I often went out crosscountry skiing or biking at night, after thee kids were in bed.)
My wife is very supportive and we've kept open communication through the years. That's necessary, if you are both going to work it out so you can have a healthy family, marriage and personal life. Believe me, it gets weird if you don't have that. I definitely can't say it was always easy.
As the kids grew up, they became part of these activities, and we've had many family hikes and bike rides. Now, I've got sons who carry some of my stuff. 25-30 years ago, I was carrying them! It's great to see them enjoying active lives doing pretty much the same things I was doing so many years ago, and still do, albeit on a more relaxed level than they do. And it's wonderful to see them excel at these pursuits.