really mild book stuff, not spoilery, but has to do with differences between the show and books:

Spoiler:
I could buy that in the book world-- and the totality of it would be that LF is really the extremely pragmatic good guy while Varys is his foil, being the ostensibly good shadowy character, but ultimately he's driven by the trauma of his youth which has his goals being revenge against those who he sees as being the same as those who have wronged him.

But I think the show has failed to give us the same depth with Varys, and there's just not enough time to pull this one off on screen.