Have always gone in very late August or in September.  For the past 5 years it's only been mid-September for the reasons stated by others.

Mid-September can be warm and beautiful or cold and windy (or sleety even as it was last year) but there are very few people. You never have to rush to get to a particular site (other than for weather-related reasons) and the bugs are virtually non-existent.

The days are shorter but the stars at night are brilliant and that first sip of brewed coffee on a cool, crisp September morning is exquisite.