I think you either want to call or fold here... Pushing is not getting called by most weaker hands, but will get called by QQ+.

If he is just betting with something like AK I think he will slow down if you call and the turn is a blank and you can turn up the heat there.

If you call and he fires again I think you'd fold most the time.

But I also don't see a problem with folding where you did..