The physically tired part isn't generally a bad thing; being tired physically can help you focus because you won't be as easily distracted and your body is more at rest. The mental part can be learned.

I went from breaking even at 10nl to killing 100nl 6max last year while working a 48+ hour/week framing job so that I'd have the option to play poker while going back to school. If you really want it, you'll take measures to compensate for being tired while you play.

Edit: Just noticed you said you get off work in the mid-afternoon so figured I'd throw this in since I don't think anyone else has. If you choose to play in the early morning, you'll likely be missing out on some really good games. If you could manage to play decent poker in the evenings, this would be best for game selection, which will give a noticeable difference in winnings in the long run.