Preheat an oven to 325 F. Line a 9x5 loaf pan with parchment paper so that there's some overhang. This makes pulling the cake out of the pan much easier. Lightly spray the pan with nonstick spray.
Use a hand mixer on medium speed to cream the butter and sugar together in a mixing bowl until very fluffy. Scrape down the sides of the bowl as needed while mixing. The process should take about 4-5 minutes.
Add the eggs, sour cream, and vanilla and mix for 30 seconds or so until really well mixed and smooth.
In a small bowl, whisk together the flour, baking soda, baking powder, and salt. Fold the dry ingredients in with a rubber spatula until no streaks of flour remain. Fold the milk in with the flour.
Spread half of the batter into the prepared loaf pan and top evenly with the brown sugar-cinnamon mixture. Top with the remaining batter and spread in an even layer. Top with the streusel. Bake for 60-70 minutes, or until a cake tester comes out clean. Cool the cake in the pan for 10 minutes before removing. Allow to cool for 20 minutes before slicing and serving.