Bingo. Brownies are different.

Butter is a unique mix of water, fat, and solids that separates and strengthens layers in dough. Because it's solid ish at room temp it traps tiny air bubbles in the dough when kneading. It's also a bit acidic which works with the alkali in flour to make it a bit lighter and fluffier. When you sub olive oil for butter you get a greasy slab instead of a light flaky roll.

Brownies don't have this problem because they have a dense, oily, sugary crumb that's relatively uniform. They're also loaded with chocolate which has the same pH as butter and can serve some of the same role.

Chemistry in baking is fucking fascinating.