I think it's more probable that he's got you crushed under a bigger pocket pair - nits don't usually make such moves holding air.
You're in a tough spot here - if you call and don't hit a set on the flop, you will still probably be behind in the hand - even if there are no overcards. You say you've played the last hands very aggressively, so your folding equity is low as well.
It might be better to just fold this one.