Has anyone actually figured out how to track what AI tools employees are using?

Posted by Narrow-Layer-7500@reddit | sysadmin | View on Reddit | 35 comments

So I've been dealing with this for months now. Our security team keeps finding random AI tools popping up across the org — people signing up for whatever new ChatGPT wrapper pops up on Twitter, pasting source code into random LLMs, you name it.

We tried blocking domains but it's basically whack-a-mole at this point. New tools pop up faster than we can add them to the blocklist. And half the time engineering pushes back because they "need it for productivity."

I started working on something that sits at the network level and monitors AI tool usage across the org — basically gives you a dashboard of what's being used, by who, and flags when sensitive data (PII, source code, credentials) is being sent to these tools. Think of it like a CASB but specifically built for AI tools.

Still early stage but curious — is this something other security teams are struggling with too? How are you handling shadow AI right now? Would love to hear what's working (or not working) for others.

[-]

BoilerroomITdweller@reddit

We have Applocker. It blocks anything running out side of Program Files or Windows.

For the web tracking we have Defender and previously Crowdstrike. It monitors everything.

Our security team just made a policy and threatened everyone with getting written up if they used anything except co-pilot which we have with our Tenant.

zed0K@reddit

Why threaten writing people up when you can just block it? And applocker doesn't work for webapps, where most AI services live. Or even downstream from another app that you may allow, like VSCode, or similar. These need blocking at the proxy level.

TYGRDez@reddit

Why threaten writing people up when you can just block it?

How are you blocking every AI tool in existence?

Web filtering, proxy and firewall, application control, etc.

Thanks for being so specific

Half of the leading enterprise suppliers of firewall / proxy have a baked in AI web filtering content library, you just need to enable it and patch any further holes. Such as Google AI, block the UDM.

Application Control, Applocker, WDAC, ThreatLocker, pick one.

thortgot@reddit

Web filtering

That is security not me. Trying to block AI is like trying to find a needle in a stack or needles.

Applocker does block the apps like Claude from being downloaded.

It is more efficient to just tell people they could be fired for privacy violation rather than trying to block it.

Ketalon1@reddit

Implement a tool like cisco secure endpoint, that mixed with N-Able and you'll be able to monitor everything, including blacklisting programs. MS has a similar feature with their Defender if you have 365 licenses for it.

Kumorigoe@reddit

Sorry, it seems this comment or thread has violated a sub-reddit rule and has been removed by a moderator.

Do Not Conduct Marketing Operations Within This Community.

It is not acceptable to advertise a product, service, Blog or FOSS Project within this community outside of authorized threads.
It is not acceptable to perform product research or market research within this community without permission.
The Reddit advertising system exists to help you reach out to new or existing customers.
Product Representatives are free to discuss their product in the context of an existing, naturally-occurring discussion. Astroturfing is not permitted.
As always, users must disclose any affiliation with a product.
Content creators should refrain from directing this community to their own content.

Your content may be better suited for our companion sub-reddit: /r/SysAdminBlogs

If you wish to appeal this action please don't hesitate to message the moderation team.

bitslammer@reddit

We're using multiple sources to do this such as Defender, looking at Zscaler logs, Tenable data, etc.

SlyCooperKing_OG@reddit

Firewall reports

krattalak@reddit

Umbrella tracks (and can block) both Generative AI content and as an application.

OneRFeris@reddit

Mimecast Incydr lets us see whenever content is uploaded to an AI site, or even clipboard copy/paste.

The Incydr agent even captures the content (including clipboard) for admin review.

Ask for a demo, its pretty neat.

rainer_d@reddit

Then Mimecast has all your source code…

So sue them if something bad happens.

CPAtech@reddit

So you're actually just marketing a tool and not really here with an honest question, got it.

automounter@reddit

I don't see a lot of marketing just OP saying what they are doing.

Here's another, they are a cancer to this sub:

https://www.reddit.com/r/sysadmin/comments/1sm9vsc/at_what_point_does_google_drive_stop_being_enough/

Valdaraak@reddit

I started working on something that sits at the network level and monitors AI tool usage across the org

There's the marketing part. Another guy who's "making a tool" and "wants to know what others are doing" so he can vibe code his own and try to sell it.

These all follow the same pattern and always include "just curious" or "wanted to ask." 4 year old account with no karma nor post history. Old accounts are either purchased to use for marketing or they create brand new accounts but hide their comment history.

"is this something other security teams are struggling with too" = is there interest in my tool.

Woofpickle@reddit

sHaDoW aI

BK_Rich@reddit

The other sites are blocked. We can only use copilot.

xaeriee@reddit

Barracuda or Zenity

JCochran84@reddit

We use Cisco Umbrella, it has a section for tracking 'Generative AI' apps. We can also request items to be tagged if they are not.
Generative AI Content Category - Cisco Community

sazzer@reddit

Education might be more useful than tooling here.

People need to understand the dangers of pasting company-confidential (e.g. source code) content into remote services that you don't have suitable agreements with.

They (hopefully) wouldn't post company source code as-is onto Stack Overflow, or Bluesky, so why would they do the same into Claude or ChatGPT?

And the same is true of using tools that work with those public LLMs. Downloading and using e.g. Claude Code without a suitable agreement is no different to sending the code to the LLMs directly.

That needs to be understood by the senior leadership, and they need to agree to either fund some LLM agreements for productivity gains, or ensure that their staff are not using those tools, making it clear what could happen if they do.