Preheat the oven to 350° F. Line two cupcake tins with cupcake liners.
To make the cupcake batter, combine the flour, cocoa powder, baking soda, and salt in a medium bowl; whisk together and set aside. In a liquid measuring cup, stir together the sour cream, milk and vanilla extract. In the bowl of an electric mixer fitted with the paddle attachment, combine the butter and sugar, and beat on medium-high speed until light and fluffy, about 2 minutes. Beat in the eggs one at a time, scraping down the bowl as needed. With the mixer on low speed, alternately add the dry ingredients and the sour cream mixture, beginning and ending with the dry ingredients. Mix just until incorporated.
Divide the batter evenly between the prepared cupcake liners, filling about 2/3 full. Bake for 18-22 minutes, or until a toothpick inserted in the center comes out clean. Allow to cool in the pan for 5-10 minutes, then transfer to a wire rack to cool completely.
To make the frosting, in the bowl of an electric mixer fitted with the paddle attachment, combine the cream cheese, butter and peanut butter until smooth. Slowly mix in the confectioners’ sugar, beating until smooth and well blended. Mix in the whipped topping until smooth and fluffy. Frost cooled cupcakes as desired.