Preheat your oven to 350°F (175°C). Line a 12-cup muffin tin with cupcake liners.
In a medium bowl, whisk together the flour, baking powder, cinnamon, nutmeg, and salt. Set aside.
In a large mixing bowl, beat the softened butter and granulated sugar until light and fluffy, about 3-4 minutes.
Add the eggs one at a time, beating well after each addition, then mix in the vanilla extract until well combined.
Gradually add the dry ingredients to the wet ingredients, alternating with the eggnog and milk. Begin and end with the dry mixture, mixing just until everything is combined.
Divide the batter evenly among the cupcake liners, filling each about 2/3 full.
Bake in the preheated oven for 20-25 minutes, or until a toothpick inserted into the center of a cupcake comes out clean.
Let the cupcakes cool in the tin for 5 minutes, then transfer them to a wire rack to cool completely.
For the frosting, in a large mixing bowl, beat the softened butter until creamy. Gradually add the powdered sugar, one cup at a time, mixing well after each addition. Add the eggnog, vanilla extract, and nutmeg, and beat until the frosting is light and fluffy.
Once the cupcakes are completely cool, frost them with the eggnog buttercream using a piping bag or a knife. Sprinkle with a bit of nutmeg if desired.