MI Történik?

Security firm Trail of Bits has looked at how secure LLM sessions running on GPUs are and found that for some GPUs it’s possible for a hacker to be able to read the outputs of an LLM running on that hardware. As of mid-January, the attack worked on some AMD systems and may work on some Apple and Qualcomm systems; NVIDIA and ARM seem to not be vulnerable. The attack, called LeftOverLocals, “impacts the security posture of GPU applications as a whole, with particular significance to LLMs and ML models,” according to Trail of Bits. It works by “recovering local memory… we were able to build a PoC where an attacker can listen into another user’s interactive LLM session (e.g., llama.cpp) across process or container boundaries”. “The attacker only requires the ability to run GPU compute applications, e.g., through OpenCL, Vulkan, or Metal,” Trail of Bits writes. “Using these, the attacker can read data that the victim has left in the GPU local memory simply by writing a GPU kernel that dumps uninitialized local memory. These attack programs, as our code demonstrates, can be less than 10 lines of code. Implementing these attacks is thus not difficult and is accessible to amateur programmers… given the lack of comprehensive patches across impacted GPU vendors, LeftoverLocals can be defended by modifying the source code of all GPU kernels that use local memory.”