Wouldn't do it, and have never seen it done.  Take the time to set up camp, it just makes sense.  No one likes to fish more than me, and I would always set up camp before going fishing.

Twice I've had to share campsites with trippers who weren't on the ball all that well.  They got into areas at dusk where the only campsite around was occupied by us.  Both instances were in Lake Superior PP, not the Q.  Both went well, no problem.  Not ideal, but you roll with it.

Mark