Em dashes are extremely common in books and articles, which likely form the majority of the training data of most models since they're generally high quality (compared to social media).

"It's not x, it's y" doesn't show up much on places like Wikipedia or social media, but contrastive rhetorical devices like that have been been staples of public speech and marketing copy, basically ever since the Ancient Greek Sophists, whose whole schtick was to master a set of rhetorical techniques that could be reliably leveraged to convince people of their argument, regardless of which side it was arguing for or whether they had any real knowledge of the subject, and "antithesis", utilizing contrast, was one of the main things they emphasised.

The RLHF/DPO stage of training basically does exactly what they practiced. It teaches the model the most "effective" (preferred) way to talk, regardless of it's content or truth value. I imagine it's more likely to be that process that leads these models to use the same set of speech patterns so much, not the base dataset.

I imagine the fact that these RLHF datasets are often annotated by low-cost (likely ESL) workers in the Global South has an impact on what is "preferred". Cheaper rhetoric that sounds cliché to native speakers is probably more impressive to those less familiar with it.