If you wear a freshly washed pair of trousers every single day without fail, then I actually think it's fine. But underwear creates a barrier which prevents any contamination (sweat, wee drips, poop if you're not wiping properly) from making contact with your trousers. This means you can wear trousers for multiple days while swapping out clean underwear every day. We can't wear the same underwear every day (or the same trousers without underwear) because constantly exposing our skin to a build up of that contamination will cause discomfort at best and skin infections at worse. In other news, whoever was meant to teach you this stuff as a kid failed you, badly.