you are welcome

_________________________________

# Vulkan Pinned Memory Fallback


## Summary


`ggml-vulkan-nopinned.cpp` disables Vulkan pinned (imported host) memory allocation and replaces it with plain CPU-aligned memory. This prevents driver crashes on systems where Vulkan host memory import is unstable.


## Problem


The original `ggml_vk_host_malloc` / `ggml_vk_host_free` functions allocate Vulkan device-visible coherent host buffers via `ggml_vk_create_buffer` with `HostVisible | HostCoherent | HostCached` flags. On certain GPU drivers, this pinned memory path triggers crashes or hangs during model inference.


## Changes


### `ggml_vk_host_malloc`


**Before**
 (vulkan.cpp): Creates a Vulkan buffer with host-visible memory properties, registers it in `device->pinned_memory`, and returns the device pointer. Falls back to `nullptr` if the buffer is not host-visible.


**After**
 (nopinned.cpp): Skips Vulkan entirely. Allocates plain CPU memory with 64-byte alignment using `_aligned_malloc` (MSVC) or `posix_memalign` (POSIX). Prints a warning indicating pinned memory is disabled.


### `ggml_vk_host_free`


**Before**
 (vulkan.cpp): Searches `device->pinned_memory` for the buffer matching the pointer, calls `ggml_vk_destroy_buffer`, and erases the entry from the vector.


**After**
 (nopinned.cpp): Calls `_aligned_free` (MSVC) or `free` (POSIX) directly. No Vulkan cleanup is needed.


## Diff


```diff
 static void * ggml_vk_host_malloc(vk_device& device, size_t size) {
-    GGML_UNUSED(device);
     VK_LOG_MEMORY("ggml_vk_host_malloc(" << size << ")");
-
-    if (size == 0) {
+    vk_buffer buf = ggml_vk_create_buffer(device, size,
+        {vk::MemoryPropertyFlagBits::eHostVisible | vk::MemoryPropertyFlagBits::eHostCoherent | vk::MemoryPropertyFlagBits::eHostCached,
+         vk::MemoryPropertyFlagBits::eHostVisible | vk::MemoryPropertyFlagBits::eHostCoherent});
+
+    if(!(buf->memory_property_flags & vk::MemoryPropertyFlagBits::eHostVisible)) {
+        fprintf(stderr, "WARNING: failed to allocate %.2f MB of pinned memory\n",
+            size/1024.0/1024.0);
+        device->device.freeMemory(buf->device_memory);
+        device->device.destroyBuffer(buf->buffer);
+        return nullptr;
+    }
+
+    std::lock_guard<std::recursive_mutex> guard(device->mutex);
+    device->pinned_memory.push_back(std::make_tuple(buf->ptr, size, buf));
+
+    return buf->ptr;
+}
+
+static void ggml_vk_host_free(vk_device& device, void* ptr) {
+    if (ptr == nullptr) {
+        return;
+    }
+    VK_LOG_MEMORY("ggml_vk_host_free(" << ptr << ")");
+    std::lock_guard<std::recursive_mutex> guard(device->mutex);
+
+    vk_buffer buf;
+    size_t index;
+    for (size_t i = 0; i < device->pinned_memory.size(); i++) {
+        const uint8_t* addr = (const uint8_t*) std::get<0>(device->pinned_memory[i]);
+        const uint8_t* endr = addr + std::get<1>(device->pinned_memory[i]);
+        if (ptr >= addr && ptr < endr) {
+            buf = std::get<2>(device->pinned_memory[i]);
+            index = i;
+            break;
+        }
+    }
+    if (buf == nullptr) {
+        fprintf(stderr, "WARNING: failed to free pinned memory: memory not in map\n");
+        return;
+    }
+
+    ggml_vk_destroy_buffer(buf);
+
+    device->pinned_memory.erase(device->pinned_memory.begin() + index);
 }
 
-static void ggml_vk_host_free(vk_device& device, void* ptr) {
-    GGML_UNUSED(device);
-    if (ptr == nullptr) {
-        return;
-    }
-    VK_LOG_MEMORY("ggml_vk_host_free(" << ptr << ")");
-
-#if defined(_MSC_VER)
-    _aligned_free(ptr);
-#else
-    free(ptr);
-#endif
 }
```


*Note: The diff above shows the nopinned version as removed lines and the original vulkan version as added lines for readability.*


## Trade-offs


| Aspect | Pinned (original) | CPU fallback (nopinned) |
|--------|--------------------|------------------------|
| Stability | May crash on some drivers | Stable |
| Transfer speed | Zero-copy from GPU | Requires explicit memcpy for GPU transfers |
| Memory overhead | Counts against VRAM budget | Uses system RAM |
| Compatibility | Requires Vulkan 1.1+ host query | Works on any system |


## Files Modified


- `v4-support/ggml-vulkan-nopinned.cpp` — Pinned memory disabled (safe fallback)
- `v4-support/ggml-vulkan.cpp` — Pinned memory enabled (original behavior)