User-oriented audit logging vendor?

Posted by BattleBrisket@reddit | ExperiencedDevs | View on Reddit | 8 comments

I've developed about a dozen apps in various industries over my career, and every one of them had a different solution for app-level event logging. I'm talking about tracking business-relevant events within the app, with the express intent of those logs being viewed and utilized by the end users of said app. For example, if my app handles blog posts, the users might have an audit log that track edits to those posts (who made, them, when, what changed, etc.)

Each time this need came up, the ultimate solution we picked fell into one of two camps (or a little of both): open source libraries (usually tied to language ecosystem), and/or roll-your-own minimalist solutions tailored to the specific logging needs.

Over the years I've looked for a vendor-based solution to this need, but I always wind up looking at the same heavyweight "log & monitor any/every thing" approaches, such as Splunk, Dynatrace, ELK stack, New Relic, etc. etc. These allow tremendous logging capability, but cost a fortune (time, money, or both) and are ultimately overkill to deliver on a user-facing feature for audit log visibility.

I'm looking for a solution I can carry from one project to the next, and ideally offload the deep audit log research/reporting to a third party. Am I alone in this need? Has anyone found solutions that fit the bill described above?

[-]

Dangerous-Sherbet160@reddit

Hey OP! I have some free time, and this sounds like an interesting side project for me. If you're open to it, DM me and I can send some ideas your way

pachumelajapi@reddit

If youre using amazon, S3 has a built in versioning feature. You can also use any key value store for this purpose, whenever a new version of your item is saved, just save the older version as a nested item within your new version. Just make sure you keep a finite number of versions in your primary DB. That finite number might depend on how big your data is, I wouldnt do this for blog posts but for small objects sure.

BattleBrisket@reddit (OP)

Yeah used that functionality before, but in this instance I'm talking about tracking changes within the data stored in my database. If you're familiar with the Ruby ecosystem, the Paper Trail gem is an example of the functions I'm talking about.

Rea-301@reddit

I’m not sure if I understand why this isn’t simply part of your data model. If you’re tracking change history in a db level - and again maybe I’m just not getting it - why isn’t it a required element in your data store. I.e a blogpost has a db entry with the author and last modified user. Every change I would give it a history and transaction table that gets written to with some age of requirements.

I'm talking about tracking each change, over time. In this example, the blog entry would have a one-to-many relationship with a table that notes who made a change, when, and what the change entailed. If you create an blog post, that would add a log entry. If I then edit that entry, it would create another entry.