Another thought...did you win some scary pots early on? Maybe this allowed you to steal some pots and scare away drawers.

You must have had them running away with your aggression factor, and the ones that came back at you must have picked the wrong time!