Those worms on your geraniums are The larvae of a small, beige moth called Heliothis virescens or Helicoverpa virescens, which feeds especially on Nicotiana species. The budworms are one of the most destructive pests of tobacco and cotton crops. Some of the common hosts are also petunias and geraniums, and occasionally, roses. The color of the larvae can differ from brown to green with yellow stripes.
If these pests infest your containers, control is only achieved by changing the soil and sterilizing the pots before bringing the potted plants inside. Hand picking is also an option. Insecticides don’t have much affect on them, but it can help. I have sprayed insecticide on all my infested geraniums, especially on their buds, and it worked for a while. Maybe regular spraying can help more. Prevention can be done by checking buds and leaves, to eliminate possible larvae.