When he went all in after the turn, he was trying to chase you off a gut shot straight draw. Coincidentally, that's exactly what you were on and didn't even realize it.

You have to be careful on a broadway flop when it was raised preflop. Often with heavy betting, top pair is a dog. This is just one of those things you'll have to learn how to fold against.