This year we started hanging the base camp and letting it rest for 30 minutes to let whatever sediment settle, before opening the line.  This seemed to help.  The bag tends to sag down around the filter housing, creating a low spot for sediment to keep it away from the filter.  Usually there isn't much, as we usually fill it well away from shore.

Also, be careful to not let the water level get below the filter when filtering - you'll get air in the filter and this will slow down the gravity feed badly.  Not a problem with a hand pump, but definitely is for the bag.