To me, A off-suit rag (lets say anything below 9, even though I typically fold A9 as well), always look like a piece of shit hand to me, even if I'm down to 4 or 3 players. Heads up, ace high is a slight favorite over a random hand, so its pretty good there. But still I'd probably rather steal the blinds than play the hand out (assuming its late in a tourney and the blinds are huge).

With 4 or 5 players like you're talking about, still anything below AT offsuit should be an automatic fold, in my opinion, unless you're on the button and there's no limpers (where I'd raise), or in the blinds, where you can play the hand if there's no raises. AT at least the added bonus of being within straight range, and you're nearly sure to get action (and possibly outdrawn, as people will probably have outs) if a K, Q, and J are on the board. But it still a pretty iffy hand.