I use a scrubee which doesn't have any sponge component.  I think the sponge is where the bacteria really thrive.  An all plastic scrubbee is hard to find, but I finally found some at Cub Grocery Store.

Plus as mentioned, earlier, just boil the scrubbee and that will kill all the bacteria.

Westwood