just got put in charge of SOC2 compliance at my company, trying to get ahead of the credential generation piece before it bites us
Posted by Fresh-Obligation6053@reddit | sysadmin | View on Reddit | 17 comments
previous role I was just implementing stuff, now I'm the one who has to make sure we can actually prove it during an audit and its a different feeling lol
first thing I'm trying to nail down is credential generation evidence because I've seen it catch people off guard. we generate correctly, right functions, complexity enforced, but I have no idea if we could actually show an auditor what entropy settings ran on a specific credential six months ago across all our environments
don't want to be the person scrambling to reconstruct evidence two weeks before the audit
for people who have been through this what are you actually using to capture generation time evidence? built something internal, leaning on your secrets manager, third party tool?
also what killed you during the audit that you didn't see coming, and what do you wish you had set up way earlier
trying to avoid as much drama as possible before we get there
thortgot@reddit
SOC 2 is an attestation. Meaning it isnt against a strict standard. Its meant to be a discussion about your controls.
Auditors will have different requirements but they commonly like screenshots over logs for some reason.
CrumpetNinja@reddit
Screenshots are harder to fudge for non-techncial people generally.
A log file is just a string of text, and anyone can edit that. A screenshot requires some decent effort to fake it in a believable way.
Is it any more secure or trustable in reality? No.
Does it feel way more secure for people who don't live and breathe tech? Yes.
thortgot@reddit
You can modify the log, screen or whatever before taking the screenshot.
Its theatrics, especially because you don't actually have an audit log which you would have with original data sources.
SageAudits@reddit
the auditor is getting reasonable assurance that what management of the company says is accurate as far as how the system is designed and operating. It’s more than just the screenshots, though it’s interviews with people who operate the controls, understanding the current policies, observations of the control operating, and understanding how changes are being monitored to understand compensating controls etc.
thortgot@reddit
Actual technical control policies alongside non modifiable audit logs (ex. Azure, AWS admin logs) are vastly superior.
Interviews are good to have an understanding of scope and approach especially to compensation. Policies are tested against not taken as part of the actual controls.
SageAudits@reddit
At the end of the day it’s management report and the auditors are going for reasonable assurance. If something went wrong and the system they audited were questioned they may always need to provide their evidence under oath and it’s very likely many are more comfortable explaining their understanding of the risks and coverage via showing screenshots to the 70yr old judge vs json configs. Putting all that aside, I’m comfortable with configs, you have to document where it’s coming from and details around the process either way. The process and governance understanding is Much more important. Continuous auditing is another thing to grab onto (eg. How does auditor consider sampling for it) audit procedures and sampling around a control are things firms commonly interpret differently.
thortgot@reddit
Modern auditors want read access and pull their own evidence of configuration.
SageAudits@reddit
Not exactly. It depends on where the in scope data is. Modern IT environments are not on a flat network where you can just grab everything in one scan. For example, I see SOC reports from some mills that are only looking at a system like AWS and doing an infrastructure scan from a GRC tool. Those are cheap audits that don’t give a lot of value because you’re not exactly seeing what’s going on on in endpoints, databases, application layer, and the internal GRC processes involved.
AICPA will be cracking down on GRC tool audit partners soon enough.
thortgot@reddit
Those audits are mostly pointless.
Im talking about big 4 audits of medium complexity (what Ive seen) where they will get read access into everything and are validating the entire scope after after having the attestation on how it works.
Its a much better solution then assuming they get good data.
ErrorID10T@reddit
Every auditor is different, and most of them know very little about technology, they're usually just accountants that have taken a couple technology courses.
That being said, usually they will very happily tell you what they want for evidence, and often they have explicit requirements because the auditors really don't know what they're doing, and instead are just following a guide they've been given.
If you don't know how do demonstrate something, just ask, they'll very likely have a concrete way they want you to do it.
andrew_joy@reddit
Most of them are useless , this is an exact quote from one asking for more evidence for .... something.
ErrorID10T@reddit
That is both vague and expansive enough to cover about 50% of your infrastructure. What did they end up wanting for this?
_Blank-IT@reddit
I feel that in my bones
SageAudits@reddit
It’s more about asking questions and being able to identify the business risks invoked. That being said.. some of them do very much know quite a bit about technology, we are few but strong! 💪
Gunny2862@reddit
Manual evidence collection will kill your time. People outside your team give zero mental bandwidth to doing this. Makes automated evidence collection with Secureframe or another GRC platform basically mandatory.
MyThinkerThoughts@reddit
Do you use a GRC platform?
Wild-Ostrich1205@reddit
For credential evidence specifically, most teams I've seen handle this with scheduled exports from their identity provider (Okta, Azure AD) showing password policy enforcement settings, combined with screenshots of the actual config at a point in time. The gotcha is that auditors often want to see it was consistent over the audit period, not just on evidence collection day, so building that into a quarterly snapshot habit early saves pain later.
The thing that I've seen kill teams most often is off-boarding. Not the password policy, not the firewall config. Someone left six months ago and still has read access to a production system. Auditors pull user access lists and cross reference against HR termination dates. That one shows up constantly.