Anyone thinking about the security side of Gemma 4 on phones?
Posted by Ok-Virus2932@reddit | LocalLLaMA | View on Reddit | 16 comments
Seeing Gemma 4 run locally on phones is really cool, but I feel like most of the discussion is about speed, RAM, battery, privacy, etc.
I’m curious what people think about the security side once these models get more capable on mobile.
Things like:
- model tampering
- malicious attacks against models
- local data leakage
- tool use going wrong if mobile agents become more common
Do you guys think running locally is actually safer or more private overall, or does it just open an new attack surface?
tvall_@reddit
if the security of your app depends on the output of an llm, either cloud or local, you're doing it wrong
Ok-Virus2932@reddit (OP)
Exactly, but the on-device AI deployment just gives attackers more room for app security.
tvall_@reddit
oh and on the stealing the model bit, if your secret sauce in your app is how you trained a model, if you hand out that model then obviously you've handed out your secret sauce. only options there are lock things down, make sure model weights never leave your well secured servers. or reconsider your business model because that's not much of a moat to keep competors from surpassing you when a new model comes out the job and zeroshots your task
Ok-Virus2932@reddit (OP)
But on-device AI means you already got rid of your secured server for model protection. Moreover, this is the trade-off between privacy-preserving and security.
tvall_@reddit
exactly. if its a task can handle, then on device generic llm could save you some money. but you have to treat untrusted inputs from users and the llm as untrusted. if you have some secret sauce special llm, then you still have to mitigate for when it hallucinates anyway even if you keep your secret sauce in the cloud, and you still have the untrusted user input problem.
can you describe the scenario where this would actually a problem? if you are more specific instead of coming up with an incredibly vague potential risk in some unimagined edgecase, maybe we can figure out where an actual problem could lie
Ok-Virus2932@reddit (OP)
A practical case is when the on-device model is hooked into phone context or tools. Phones are full of untrusted inputs like notifications, app UI, webpages, and overlays. If the model can read that stuff and also take actions, the risk is unintended actions or leaking local data. That seems like a pretty real issue, not some imaginary edge case.
tvall_@reddit
that's still a vauge hypothetical with nothing specific, and the answer there is you're dealing with untrusted inputs, so harness around the llm needs to sanitize inputs and sanity check outputs no matter where the llm is running
tvall_@reddit
yes, instead of the llm being guaranteed to output garbage or nearly guaranteed to be jailbroken within hours, it's guaranteed to output jailbroken-like responses in hours. nearly no difference, santize your inputs
Ok-Virus2932@reddit (OP)
Then what if malicious users modify the llm from an app, repackage the llm to the app, and redistribute such the compromised app. Sanitizing input won’t work anymore.
tvall_@reddit
well if they only modify the llm, then thats exactly what sanitizing your inputs is for.
and if all input is handled on device and never touches your servers, then its in the users hand anyway, nothing you can do. if you need to be 100% sure the app isnt tampered, apks are signed, so theres no way a malicious 3rd party could replace the app with a tampered one unless its the first install and the user went out of their way to install it from somewhere other than where you distribute. and if thats a risk, theres always play integrity.
StupidScaredSquirrel@reddit
This is an unrestricted agent problem. Not a model problem. You can have all these issues with a cloud provided model.
Ok-Virus2932@reddit (OP)
I don’t think malicious users can modify or steal cloud modes like those deployed on-device, as on-device ai allows all end-users to access them.
StupidScaredSquirrel@reddit
I'm not suggesting people can swap the models for malicious ones. I' suggesting that malicious actors won't bother with the models themselves but will focus on the apps and agents that link to a model.
Ok-Virus2932@reddit (OP)
That is a good point, but it is really up to how adv actors want to play the game.
FusionCow@reddit
this is a repost of a post that got removed
Ok-Virus2932@reddit (OP)
Man, this is a new post.