Maybe you were. But the best manipulators look for something they can use, more likely it was some quality that made you a vulnerable target. A need to please, a will to fight...

[which, actually, segues nicely into his own tale]

Not so much mind-controlled, but manipulated, used by someone else for his own needs. He used my anger and nihilism to groom me to work willingly with him. I might have become his right hand if not for Cora-san.

[...sigh. Compulsion to speak or not, it's still a bit unpleasant to dredge up.] I had a terminal illness, and he wanted to try to find someone to cure it, so he took me away. We searched for six months to no avail. But, in doing so, I was away from Doflamingo's influence and had someone full of hope to look after me instead. It really wasn't any strength of my own, just not being around the manipulating asshole.