The truth is that genitals and chromossomes do matter, but not in 99% of social interactions.

Right, but we're not really talking about casual social interactions where we can only operate on assumptions - we're talking about a deeper philosophical point of whether a trans woman is actually theoretically a woman, per above.

And I don't think the progressive position is actually as rooted in outward gender expression as you're implying here. Consider two hypotheticals:

1) Consider a cis man who identifies strictly as a man. He dresses up as a woman as a practical joke, and manages to pass convincingly.

Progressives would agree that he is a man, regardless of his outer appearance.

2) Consider a trans woman who has made almost no efforts to pass. Full beard. But she genuinely considers herself to be a woman and identifies as trans.

Progressives would insist that she is a woman, regardless of her outer appearance.

And therein lies the philosophical issue - the progressive position is ultimately rooted in subjective intent rather than any sort of definition of man or woman or male or female.

These terms may not have perfect clear definitions in all fringe cases, but the rest of society outside of progressive spaces doesn't view these things as something that is rooted in subjective intent.