Genetic Engineering Is Controversial

I get that, but the part that worries me is where the line gets drawn. If we start with preventing diseases, that seems reasonable. But once the technology exists, what stops people from wanting to edit other traits like intelligence, height, or appearance?