How to setup Logs for windows
Posted by Top_speed_@reddit | sysadmin | View on Reddit | 68 comments
Hi just joined a company as IT support, how do I setup Logs for windows systems (11, 10) for general troubleshooting and see what updates are happening and what caused the issue. To get a bird's eye view of the office environment.
What might be the optimal way to achieve this.
30yearCurse@reddit
Decide on important things you want to get, Update information, Security logs, what equipment do you have now? laptops, servers. Once you decide some metrics you want to capture you can use AI to help narrow what you want.
There are a lot, a lot of services out there, pricing varies a lot. Zabbix, ManagedEngine, MSP products, Action1.
You will figure it out, if you are dealing with laptops, System Logs and Security logs. Most of your issues will be end user driven.
Siems will be expensive and break your company's budget pretty quick.
Get some free network scanners, start poking around the corners of your network. run scans. You have Win10, that is pretty much end of life, what server OS's are you running... 2016 and before are EOL.
Remember to breath.
Top_speed_@reddit (OP)
There are no servers. All the data is either stored in end users laptops or via a shared folder in desktops.
Currently spun up zabbix to monitor the AP's, switches, firewalls, vpn, printers. To see if the devices are up and running via SNMP.
30yearCurse@reddit
Not sure of the number of users, but having data shared via their laptops is a problem. You have no central control, no backup of data. I am guessing no one travels anywhere. Action1 also gives you an RDP client to connect to the users laptop. Not sure if Zabbix does.
I would look at getting a central NAS, SuperMicro any number of dealers. Get the data centralized and then backed up to cloud, probably some immutable storage of some kind.
Do you have a budget? What does management want, besides not to be bothered, and zero cost?
GeneMoody-Action1@reddit
We do indeed have remote access, and appreciate the shoutout!
We are a patch management solution, but we have additional tools to make it more effective in that market, such as scripting and automation, reporting & alerting, and as you mentions full unattended remote access. Some people use us for RMM if the need is light, but we ware not a n aspiring RMM, it is just that some people's RMM needs can be covered by a robust patch management solution.
It is hard to beat Wazzuh for the price tag, for SIEM and centralized logging, Security Onion is a treasure trove of information for free as well.
So Wazuh = Free
Security Onion = Free
Action1 = Free for first 2 endpoints (perpetual and fully functional)
Can go over to to spiceworks and get their free helpdesk ticketing as well, or go to turnkey linux and get a preconfigured VA to just load up and go onprem or in the cloud... (As well as all the other helpful things at turnkey)
That's a LOT of utility for free...
LowMight3045@reddit
talk to a software services vendor / reseller. there are several software vendors that can sell you products to help parse the many log files that windows can create. Controlup , Splunk are a couple of good ones that come to mind. your reseller may have others. Talk to Microsoft too. I dont have experience with it but Azure Monitor Agent (AMA) might also be good. stick to Microsoft products if you can.
dvr75@reddit
You can have ELK for free to collect the logs from all your systems.
To analyze the data to actionable actions you need either make custom dashboards on the ELK or have 3rd party tool like SIEM or log analytics like datadog for example.
GhostandVodka@reddit
Windows servers can genterate 300,000 logs per second. The naivety of this post is endearing. Youre going to figure out what services and software your work does and document the corresponding event IDs or what section of eventviewer they are in.
It sucks. It's not fun. There is a whole industry devoted to SIEM and software that interprets logs and makes them more easily searchable.
Some things are easier than others. For example connecting to a wireless network is logged under wlan-autoconfig in eventviewer. Youre starting an exciting journey. goodluck!
FU-Lyme-Disease@reddit
Why can’t he just read like half of them! 150,000 logs would give a good feel of what’s happening on day to day windows!
GhostandVodka@reddit
Thats a second not a day lol
FU-Lyme-Disease@reddit
Yeah, that’s why he would only read half of them! I’m not a monster!
Top_speed_@reddit (OP)
So I have been thinking to implement wazuh in the environment.
dreniarb@reddit
i found it to be disappointing it what it actually did. this was a few years ago so maybe things have improved.
would love to hear what you think of it if you do try it out.
Top_speed_@reddit (OP)
Ok are you using any SIEM now. I want to have SIEM to integrate the logs from firewall, switchs, AV, and windows to track updates and app changes.
GhostandVodka@reddit
Our security company used a modified WAZUH
AdInevitable8483@reddit
That's the reason log management solutions exist you filter only important logs
GhostandVodka@reddit
Notice how I said "There is a whole industry devoted to SIEM and software that interprets logs and makes them more easily searchable"
Your comment added nothing. Kindly step on a lego.
Due_Peak_6428@reddit
first rule of IT is you need to learn to google stuff before you start asking for help
Top_speed_@reddit (OP)
Google and GPT has been my best friend.
sammavet@reddit
Don't use chatgpt. That shit will have your deleting necessary files, then say "you're right, there's no way to get those files back."
Sobeman@reddit
Second rule of thumb, all LLMs make shit up.
Arudinne@reddit
Trust but verify.
I often ask Claude to cite its sources.
missed_sla@reddit
LLM is a place to start but be aware that it'll give you a lot of bad info that sounds correct. Because at the end of the day, it's just a program to figure out the next word in a string of words. When prompted with anything moderately complex, I have yet to encounter a 100% correct and factual response from any of the models. There's always correction, and at some point it becomes more efficient to just learn the subject you're working on.
theEvilQuesadilla@reddit
Adding to this: adding "make no mistakes" or anything like that in fact does not prevent the LLM from just making up whatever the hell it wants.
Due_Peak_6428@reddit
Oh shutup you're boring
theEvilQuesadilla@reddit
Then you have failed.
Due_Peak_6428@reddit
evidently not!
MARS822a@reddit
Go back to shoveling manure for a living. You're WAY out of your league.
Some_Team9618@reddit
Setting up OpenTelemetry (otelcol-contrib) and having an aggregator server such as another otel instance or even fluent-bit then output to graylog or victorialogs or Loki / grafana.
Doesn’t replace a full SIEM but can get you part of the way there for visibility.
Alternatively use something like Wazuh as well.
Some_Team9618@reddit
To add:
Windows logs can generate a ton of stuff so be intentional on the audit policies you enable with GPO. Use the logging agent or aggregator to drop all the noise or process the logs to filter out noise.
A good place to start is also what Microsoft lists as minimal / common if you were to ingest into Sentinel. And their appendix L
https://learn.microsoft.com/en-us/azure/sentinel/windows-security-event-id-reference#event-id-reference
https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/plan/appendix-l--events-to-monitor
The appendix L is more server specific but can help orient yourself.
I find ingesting on-prem first before you go down a cloud SIEM will give you a really good idea on what’s being logged so you can then save on ingestion costs by only shoving stuff with forensic value into the cloud.
hkusp45css@reddit
Wazuh, have your logs ingested into the SIEM and learn to search for your events that way.
patient-engineer-656@reddit
Try Velociraptor! Free and can be tuned to look for things that would actually be worth looking at.
Adam_Kearn@reddit
All of this can be achieved via the built in event viewer.
I would recommend collecting a load of IDs for things like windows updates applied/error/actioned status.
You can then make a powershell script to show all the events that you care about in one table.
But tbh in my career I’ve never really needed this.
If I get a feeling it’s a windows update that’s broken something (normally when more than one user is getting the same problem) I’ll just check the update history in control panel.
The best option I would recommend to you is to automate your workflow of deploying operating systems.
Use tools like OSDCloud or FOG to deploy windows with all the installed software and drivers automatically.
When I get something that’s just acting weird I just reimage it now as it only takes 20mins for the whole computer to image and be ready to use again.
Also makes onboarding easier for you and your team.
420GB@reddit
You do not collect logs from client/office machines, only exception if you want to call it that is the security related data that your EDR collects.
Your MDM or asset Management Tool will give you this, not a database of billions of event logs.
Top_speed_@reddit (OP)
We currently do not have any asset management tool or MDM. Currently looking at snipe-it for asset management.
Adam_Kearn@reddit
I think this guy was referring to an RMM tool instead of asset management.
But I also do recommend snipe for doing your asset tracking.
I’ve made a load of powershell scripts to automate everything in snipe for me.
It will create and deploy assets across my org automatically from multiple systems.
I’ve even connected it into netdisco to create assets for printers/desk phones/switches automatically.
BlackCodeDe@reddit
Activated the Embedded Sysmon Feature at Windows 11 March Update and use maybe SwiftOnSecurity Sysmon Config.
jainesh3271@reddit
If you have enough budget you can try EDR solution.
It will give you everything that is going on you network + to you end device.
Indiesol@reddit
From what you're saying, it sounds like event viewer is all you need. Start there with the default settings.
When you're reviewing logs but don't know what you're really looking for, look at the options on the right. Often time I'll start by filtering, and only showing warnings and errors and critical, or I'll filter by application, service or event IDs.
Want to see who logged into a computer, go to security log, then google the event ID for a logon, then filter by that event ID and user. Want to check status of the hard drive, go to System and filter by critical, warning and error, and look for hard drive errors.
Just a tip, windows disables auditing on files and folders (who deleted what) by default because it can cause performance in disk space issues, so you have to manually configure that when you need it (like, when files are coming up missing). My SOP there is to start auditing when a problem arises, review the logs until we've isolated the issue, then turn auditing back off once it has been addressed.
"The previous IT people left the company," is a concerning statement. Does that mean the whole internal IT staff quit at once, or did they use an MSP and get rid of them? It sounds like you've been thrown into an active wildfire.
Siritosan@reddit
Welcome to the rabbit hole of logs. Event viewer will get you started. Get use to event ids like what shutdown the PC. Errors and critical. Remember to come out for air once you dive in.
AdInevitable8483@reddit
For single pc its fine. But if there are more than 2 pc then you require logging solutions like loggly clickhouse ELK etc. And firward logs there.
Siritosan@reddit
So a bigger rabbit hole
hihcadore@reddit
Uhhhh this would be posted in tech support, I think. If you’re gonna make it in this field you have to do your own research first.
Without knowing your environment have no way to know in which direction to point you. At this point we’d mark the ticket return to requester, no enough information to support.
Top_speed_@reddit (OP)
So the current role is mix of IT support, sysadmin, network admin, soc etc.
GhostandVodka@reddit
I don't believe they hired you a role that does all of that and you don't know what event viewer is
Top_speed_@reddit (OP)
I'm never mentioned that I don't know about event viewer, I just wanted to know how IT works in enterprise network. To get an overview to troubleshoot issues and keep everything stable.
Inquisitor_ForHire@reddit
I'd recommend setting up an ELK stack (Elasticsearch, Logstash, Kibana) and installing the client to push logs to it. Don't push a bunch... push ONE system to ELK until you get it working. Take note of how many logs you're ingesting - that determines how fast you're going to consume your space.
Set your retention up. I'd recommend extremely short at the start - like 7 days. Take that data and you can then extrapolate how much to keep 14 days or 21 or 30, etc etc.
You can literally set up an ELK stack for zero cost - just the system resources. It's not the absolute easiest thing ever, but there's a lot of stuff online (Youtube) on how to set this up.
420GB@reddit
Start learning the basics first then before you implement bad ideas like this
hihcadore@reddit
U just described a in-house sysadmin. You’re also asking basic tech support questions.
Do you and your company a favor and go find an MSP that will support you.
Sorcerious@reddit
Not knowing about Event Viewer is like having a network admin who doesn't know what a subnet is.
Wait, do you?
Hale-at-Sea@reddit
If your clients are in a domain, then enable the various Audit event policies for things like account logon. Most of the useful events will be on the server side for these things though
Most RMM tools for clients will collect some events for you, but client side events are not very useful. We only actively track one windows event ID on client OS (System:1001 for bluescreen crashes)
Lonely-Buyer440@reddit
you should try RMM tool
Top_speed_@reddit (OP)
Any RMM tools suggestions.
Lonely-Buyer440@reddit
tactical RMM
Top_speed_@reddit (OP)
Currently implementd zabbix to monitor all the AP's, switchs, firewall, VPN's. This is just a basic implementation to monitor the up time, temperature. Using snmp templates.
Lonely-Buyer440@reddit
how many endpoints are you managing?
Top_speed_@reddit (OP)
Around 40 at current location and 20 at different location.
Lonely-Buyer440@reddit
You managing this solo right now or with your colleague?
Top_speed_@reddit (OP)
With colleague.
Lonely-Buyer440@reddit
great
AdInevitable8483@reddit
Setup elk on Linux PC or server and forward all logs to elk using file beat. Best solution.
Top_speed_@reddit (OP)
Ok I'll try this.
AdInevitable8483@reddit
You can dm if you need any help
Kahless_2K@reddit
Sounds like you are hopelessly underqualified for your role.
Good news: You are going to learn a lot!
Bad news: keep your resume up to date.
The first thing you are going to need to learn is how to obsessive Google every problem that comes up.
use -ai in your search to eliminate AI responses. Its not that they cant be useful, but you don't have the experience yet to spot when they are hallucinating bs ( which is often )
Don't even think about trying to vibe code, you will at best build something unmaintainable.
Dangerousfish@reddit
Have you tried event viewer?
Top_speed_@reddit (OP)
Yes, when doing troubleshooting I take a look at reliability history and event viewer.
Plane_Parsley9669@reddit
Get me out.
hihcadore@reddit
Yea homies in the wrong subreddit. Should go to tech support.
nullbyte420@reddit
Have you tried anything at all?