Hi guys, I was digging into Linux memory management and came across an interesting optimization, when a page in memory is dropped(assuming it's clean) the kernel doesn't immediately zero out the contents rather it unmounts the page, does TLB shootdowns and puts the page in the page pool. Now when another process needs it the kernel zeros out the page and mounts the page to that process Virtual Memory.

Now the interesting thing is that if the page requested was by an user process the zeroing out is done mandatorly as to not violate isolation rules but if the page requested is by the kernel itself say the kernel needed it for its internal buffer or something then zeroing out isn't usually done as the kernel space is treated as trusted boundary and anyways the kernel will overwrite the contents of it so as to save time and bandwidth it avoids it.

This got me into thinking could it be missued. Like i did learn the other day that external devices like NIC, GPU, PCIe devices if they need to write to Main Memory they usually don't directly DMA to user mapped memory rather they DMA write to kernel buffer and copy from kernel space to user space happens.

I thought of situation where say a NIC card is DMAing to kernel buffer page this page was previously was allocated to some process and wasn't zeroed so old contents still exist. For example the NIC writes only 64 bytes but reports it as written 128 bytes So when the kernel sees this it interprets as NIC written 128 bytes as valid bytes and copies the 64 bytes actual content+64 bytes of stale left over bytes into respective process receive socket and the process then can call read on the socket and it reads the other process data.

But as i dig little deeper into the working of NIC I came to conclusion that this to happen is very highly unlikely and would need a bug at NIC's frimware level or the driver itself because NIC can't just like that lie about the bytes received, they track how many bits recieved at the phsyical level and writes a metadata about the exact length it wrote to the DRAM. So unless the frimware didn't count the recived bits properly or the driver failed to interpret the metadata it's highly unlikely to occur.

Another place where this could possibly happen is with GPU especially if followed the pipeline of GPU(DMA to)->kernel buffer(driver)->copy->user space.

As far as i have seen GPUs don't exactly report how many bytes it has written it usually signals after completion. and the driver acknowledges it and even if an explicitly mentioned the bytes written like using an counter it's usually managed by the software.

So when an user space application uses APIs like CUDA/DirectX to request a GPU compute with the expected output size, the driver in the kernel space then validates the request, allocates the required buffer size, sends GPU commands for execution and memory descriptiors for DMA. The driver then expects the GPU to fill the buffer with the expected size here say 128 Bytes was requested. But the GPU actually wrote only 16 Bytes and doesn't report the size written and just signals the completion the driver then copies the 128 Bytes from the kernel space to user space assuming the GPU has filled 128 Bytes where as in reality that wasn't case so if that page that was allocated to that buffer wasn't zeroed out those remianing bytes copied could contain the data of other process and the malicious application reads it.

Since GPUs are programmable today, is this possible if not, what exactly prevents this scenario from happening.