The Shannon Limit defines the maximum theoretical rate at which error-free data can be transmitted over a communication channel with a specific bandwidth and signal-to-noise ratio (SNR).

That is not directly applicable here, except for how much information can be transmitted in a single embedding.

The limitations are closer to the entropy of information and Kolmogorov complexity.

The models have to learn some specific facts, but in general, facts are a specific case of a combination of general underlying patterns and principles. Basically, specific details are noise that we assign meaning to.

Finding the generative functions of information means being able to compose and interpolate that information.

This is the whole thing about "generalization".

If you learn the rules of logic, the rules of a million billion things become dramatically easier to understand, because you don't have to memorize everything, you memorize key points, and apply logic.

If you know the sine function, then you can generate all kinds of sine waves and find points to an arbitrary level precision, you don't have to memorize infinite points.

A single 16 bit vector with 4096 dimensions can represent (2^16)^4096 states. That is approximately 10^19728 distinct values. You could give an address to every atom in the observable universe with that number. Then when project that vector up 4x, and store values in that space. The values in the FFN are typically (2^16)^16384 space, which means that embeddings map into a very high dimensional volume.

The capacity of AI models is wildly underutilized, and there's insufficient pressure to make the models use all that space.
The models can often just memorize everything. Under-parameterization forces the model to generalize in order to make efficient use if the space, while over over-parameterization causes generalization, because all those neurons have to be doing something useful without causing the loss to go down.

The underutilization/undertrained observation is what led to the "super massive data" shift, where training went from the low hundred billion tokens, to 10+ trillion tokens.

The models also have to learn a whole lot indirectly, via frequency ana adjacency, which is a big reason why their latent spaces can be a mess.

The cross entropy loss function is useful for training early generation models, but ultimately it's insufficient for any kind of data efficiency on complex data where there is not singular correct answer.
We have Kullback–Leibler divergence, but don't usually know what the actual distribution should be.
The models eventually learn a distribution, and it's probably a decent one. So you can use KL to distill knowledge from one model to another.
If you've got multiple expert models, you can distill the experts into a single student, which then potentially has a better structured latent space.
This is at least partially why we can have 2~7B models today that are better than the 100B models from a few years ago.

Then you have the quantization issue: if we can consistently quantize a 16bit model to 4bit, that means the model was significantly over-precise.
The model could have held 4x~ the information.

So, yeah, we have at least one huge parameter efficiency breakthrough that's going to happen. I'm thinking at least an order of magnitude in terms of weights, and another order of magnitude, in just having a model that is a domain expert, and which doesn't have every digital thing ever in its parameters, but instead is properly trained on the distributions and generative functions of the data.