Mix the yogurt and lemon juice. Both of these ingredients need to be at room temperature before mixing.
Preheat the oven. Put the oven racks in the middle position in the oven and preheat to 425 degrees.
Spray each muffin area in the muffin pan with nonstick spray.
In a smaller bowl, add the sugar and lemon zest. To get the lemon flavoring throughout, rub the sugar and zest together between your fingers to get that lemon oil to really stick and mesh with the sugar.
Add the room temperature butter to the sugar mixture and cream together. Slowly add in the room temperature eggs one at a time.
Add the yogurt and lemon mixture together in the bowl and mix well.
Using a different bowl, whisk the salt, baking powder, poppy seeds, and flour. In the middle of the mixture, make a well (a hole).
Slowly add the wet mixture into the well you just created in the middle of the dry ingredients. Stir just enough to make the mixture moist, but don't overmix. This will make the muffins dry and tough. (There will be small lumps - that's okay!)
Do your best to divide the batter equally to the 12 prepared muffin cups. If you want to smooth the tops before baking, do so now.
Let the muffins rise. This step is optional, but if you want to have slightly raised muffin tops, letting them sit in the pan for about 20 minutes before putting them in the oven helps! You can skip this step if you want.
Put the muffin pan in the oven and bake for 5 minutes at 425 degrees. Once that time has passed, turn the temperature down in the oven to 350 degrees and bake for 12-15 minutes.
As the muffins are cooling from baking, whip up the glaze. Whisk the milk, powdered sugar, and lemon juice together in a bowl.
Give the muffins about 10 minutes or so to cool down before removing them from the pan and glazing them. Dip a fork into the glaze and drizzle it over the top of all the muffins.