Google, GovTech developing first-of-its-kind AI model to flag scams in real time for Android users

[u/Extension-Finance179]

https://www.straitstimes.com/tech/google-deepmind-govtech-developing-first-of-its-kind-scam-detection-ai-model-for-android-users?ref=top-stories

[u/worldcitizensg]

I thought the same google and govtech did the scamsheild sometime back. Not working ? or now need "AI" for everything ? Why can't the same "ScamShield" can be improved with AI in the backend ?

[u/hotgarbagecomics → u/worldcitizensg]

Google provides more context here:

On-Device AI Against Scams: Google DeepMind & MDDI — Google DeepMind

Scamshield already uses AI to classify scams, but this approach is suggesting a two-step process, where scams can be detected using citizen reporting, AI classification based on known signals etc, AND more importantly, using a local on-device model to analyze private conversations to detect manipulation tactics within chat.

Having tried out Gemma, this is a fantastic example. It can run on mobile, and people won't have to worry about chats being sent to gov. servers just to identify scams.

Also, I hate that ST never links to the actual source. The Deepmind article was so much more informative!

[u/gently_into_the_dark → u/hotgarbagecomics]

Yeah remember trace together? How will this be governed any better....

[u/hotgarbagecomics → u/gently_into_the_dark]

Part of the anger about tracetogether was about the privacy concerns, and iirc it was exacerbated when it was revealed that tracetogether data was used in a police investigation, despite promises that data wouldn't be used off-purpose.

If anything, this on-device analysis is moving away from the tracetogether approach, so they HAVE learned from their mistakes. Obv proof needs to be in the pudding and all that

Keen to see how it plays out

[u/gently_into_the_dark → u/hotgarbagecomics]

So hear me out.

Google already has your data. Govt issues an app that actively reviews your data including messages. Whats to stop govt from saying that the app allows them access to data? And how many people would read the fine print?

Only diff is they mandated trace together. This one is to catch all the fools

[u/worldcitizensg → u/hotgarbagecomics]

That's true but again I am not "excited". Who are the target audience ? mostly the not so well informed and if we look at the phone usage, majority of our elderly, children or in general mid market phones. Not every one uses samsung ultra or high end ones. If we run Gemma 2B the token speed is 10-15/sec and okiesh. Anything above is a serious drag; Then comes the performance of the device (be it 'hot' or battery or cpu resources).

Still need to send the chats to Gov servers. The "LI" anyway can read every single activity on the phone at NW level. IMHO overcomplicating things with technology / AI because we can.

[u/hotgarbagecomics → u/worldcitizensg]

I can tell you this: the target audience is EVERYONE. Scammer tactics have gotten so good, anyone can fall for them. No really, anyone. Scam victims aren't scam victims because they're older, dumber or less tech savvy: victims become so because scammers have just gotten THAT GOOD. They're quick on the uptake, thrive on the bleeding edge of technology, and they're extremely motivated. Nowadays, they work in coordinated teams, and there is no smart individual/MIT genius on earth that can match the knowledge, skill and speed of a motivated scammer group.

Collective intelligence incentivized by easy gains will run circles around even the most diligent, informed, security-focused, paranoid individual. It's a statistical reality, unfortunately.

AI tools are going to be only way, honestly. I'm guessing they're gonna have a fine-tuned or quantized version of Gemma specifically for this usecase, which is the right way to go. Curious to see how this is applied. That said, even a half-baked version of it will be better than relying on more human-centric approaches, like reporting, training or education.

[u/lawlianne → u/worldcitizensg]

Different person’s KPI to pad of course.

[u/Wide_Open_Buttcheeks]

Yeah, no way am I giving full access of my daily life to the government and google, fucking insane how Singaporeans have such a low sense of privacy

[u/HidingCat → u/Wide_Open_Buttcheeks]

Isn't it on-device though?

Though I am wondering how well it'll perform on phones, running Gemma4 12B on a decent PC already causes the fans to spin up.

[u/SYLOH → u/HidingCat]

Supposedly, but even if it's running totally on phone, it would be difficult to tell what it's sending back.
Most apps and devices with an internet connection send telemetry back all the time, and it would be hard to figure out if and how much of the conclusions it makes stay on the device, and if those are just limited to scams.

[u/gently_into_the_dark → u/SYLOH]

Plus it has to learn no? How to update latest scam trends

[u/anakinmcfly → u/[deleted]]

Just because he’s willing to give full access to his buttcheeks doesn’t mean he’s willing to give full access to his daily life.

[u/Serious_Attitude_882]

scammers getting better at avoiding such protection

and scammers are also using ai for their own scam works

[u/SYLOH]

The AI model analyses text exchanges on messaging apps such as WhatsApp and Telegram in real time

So the thing will be reading all your text messages, got it.
I am never installing this thing. If it's mandatory, all my messaging apps are going into a wall garden away from where it can read.

But oh, they really really promise they won't send it all to a server! Sure! That makes it all better! /s

[u/anakinmcfly → u/SYLOH]

If the people you message have the app it would still read those messages.

[u/byeoxg]

Curious what is the false negative rate for this supposedly lightweight model? I suspect it's an LLM based implementation so I guess the model would be Gemma?

[u/SuzukiSatou]

All fun and games until tis AI learn to start scamming people instead

[u/lawlianne → u/SuzukiSatou]

AI is already being abused by scammers to scam people en masse.