Calling someone a retard is a reference to those with learning difficulties. It's exactly as offensive as calling someone a mong (reference to down's syndrome) or a spastic (reference to cerebal palsy).

retadred - less advanced in mental, physical, or social development than is usual for one's age.
It's offensive to people with learning diffculties. Not that I have a problem with the word "retard", it's just that to assume it's less offensive than "mong" or "spaz" is stupid.