PokerStars No-Limit Hold'em, $0.02 BB (4 handed) - Poker-Stars Converter Tool from FlopTurnRiver.com

Button ($6.18)
SB ($2)
BB ($5.12)
Hero (UTG) ($8.02)

Preflop: Hero is UTG with ,
Hero bets $0.08, Button calls $0.08, 2 folds

Flop: ($0.19) , , (2 players)
Hero bets $0.18, Button raises to $0.50, Hero calls $0.32

Turn: ($1.19) (2 players)
Hero bets $1.02, Button raises to $5.60 (All-In), Hero folds

Total pot: $3.23 | Rake: $0.15

Opponent is loose aggressive (48/16/4) with a vendetta against me for beating him in a big pot, and also bluffy.

I called his raise because of those reasons above.

I think it's pretty clear I made the right decision in the end, but should I have folded after his initial raise?