I have no idea why that would happen, but since   I was planning a similar set-up with my Ryobi battery, I hope you/someone finds out what happened?
I wonder if the issue could be that the battery when not in the "proper" receptacle on a powertool, somehow goes into some kind of "mode"?

My Ryobi battery for my new string trimmer has held a charge thru a couple of long triiming sessions and still has plenty of juice left. Can't beleive a fish finder would deplete a battery faster than the effort it takes to cut weeds. :-?

I haven't bought a fish finder yet, after reading this, I might do a mock-up here at home and see if I get the same result while I have time to make a AA holder.