Yes, it's clear why it fails. An LLMs alphabet is not comprised of letters, but letter combinations called tokens. Yet, the test is still significative, because it shows a limitation of the current model architectures.

People don't mess up at providing an answer to a question regarding the number of times a letter shows up in a word. LLMs do.
Irrespective of the reasons why this might be the case, since we want the models to be as correct as possible (for them to be as useful to us as possible), this entails them having the ability to do simple tasks like these correctly.

Any mistake that it makes may lead to an interesting realization about the best model architecture.
Here you could consider: "Is tokenization really a good idea?", maybe you could build a model that uses letters as tokens and cleverly design it, such that it scales and performs well. Just because people have failed to make such a model perform well so far, doesn't mean it is not possible with a unique approach.
But maybe tokenization is indeed necessary for good performance. Here you could maybe consider a hybrid approach. "Is it possible to design a model that has the ability to vary how it dissects words into tokens depending on context?".

There are many interesting possibilities. To be able to think of them you have to find problems with current LLMs. My comment showed one such problem.

To clarify, I didn't mean my comment as any criticism. I don't like when people criticise. I meant my comment purely as an observation.