How do i fight when i need to?

its not that i don't know how, i work out and im strong and fast and can take a punch amazingly well for some reason. my problem is that i cant fight, i dont know why, today a friend invited me over and mugged me, he tried to take my money and when i wouldn't give it over he punched me in the side of the head and tried to take it, i could have easily overpowered him and beaten him to a pulp but i just stood there, he wasn't strong enough to pry it from my clenched fist but i feel pathetic after not standing up for myself after that, he did it because he was messed up on pills and he was screaming complete nonsense the whole time. he didn't have a knife or gun or anything and fighting him would have been the best solution because he stole 4 bucks from my wallet as i was trying to leave. How do i fight someone when i need to, i don't wanna freeze up like that ever again