TBL- I was fortunate to be able to go on long and remote canoe trips into the Quetico with my dad until he was 85.  Check out my trip report LacLaCroix August 13-23 2009 for a recounting of this year's BWCA trip with older campers.  We purposely avoided portaging and still had a wonderful trip.  Perhaps even more so because no one was feeling physically over worked.
What I have learned is that as the younger, and theoretically fitter, member in the group YOU will get to do the brunt of the physical labor.  I think the greatest issue of older campers is physical strength and stamina; they can and do keep up and stay enthusiastic, but the body wears out quicker and takes longer to bounce back.  After a while they are utterly exhausted.
How well does dad sleep on hard ground and rocks on a cool damp night?  Being able to rest comfortably has a lot to do with anyone's energy level the next day.
IC