And they're right.

Kinda but not really. Here in Tokyo it's common for people to vomit or urinate in public, then everyone has to walk around to get to their train station. In Japan, you sit on the floor, eat on the floor, and often sleep on the floor.

There is a very real risk you'll step in vomit during your commute and then track it onto the floor where you eat dinner.

In the US, we don't puke and piss in public, we aren't stepping in that, we aren't eating on the floor, we aren't sleeping on the floor. There is a zero percent chance you will track vomit onto your dining room chairs in the US.

Even then, the Japanese custom of taking shoes off at the door is as much about ritual cleanliness - if you live here long enough, you'll notice that there are absolutely situations where outdoor shoes are worn inside, e.g. a school garden will be outside but you wear your indoor shoes.

So Japan has pretty much the same standard we do - if you know you won't be stepping in vomit and you won't be sitting on the floor, it's not a big deal to wear indoor shoes outside. If the outside area just counts as inside, it's allowed. 

And even then, "Americans wear shoes inside" mostly just means we take our shoes off in the living room, sitting on a chair - not standing in the doorway. It's frankly overblown by people who just want to portray others as dirty and themselves as superior. Oh no, you take your shoes off in a different part of the house.

Like, sorry, but eating dinner on the floor of a restaurant is not magically more hygienic.

before you even talk about bidets

The Toto Washlet was invented in America; Toto only bought the patent.