It all depends on whether or not you think you can push him around.

If its a $5 home game then

a) He probably doesnt care about it all that much.
b) He probably knows you, and knows you will try to push him around.