It isn't them making a decision for your Steem power. They are making a decision with theirs.

Look, I don't think people should use their down voting power willy-nilly. But it's in no way unfair.