Spent a bunch of time choosing between Loguru, Structlog and native logging

Posted by rm-rf-rm@reddit | Python | View on Reddit | 28 comments

Python's native logging module is just fine but modern options like Loguru and Structlog are eye-catching. As someone who wants to use the best tooling so that I can make my life easy, I agonized over choosing one.. perhaps a little too much (I'd rather expend calories now rather than being in production hell and trying to wrangle logs).

I've boiled down what I've learnt to the following:

Read some good advice here on r/Python to switch to a third party library only when you find/need something that the native libraries can't do - this basically holds true.
Loguru's (most popular 3rd party library) value prop (zero config, dev ex prioritized) in the age of AI coding is much less appealing. AI can handle writing config boiler plate with the native logging module
What kills loguru is that it isnt opentelemetry compatible. Meaning if you are using it for a production or production intent codebase, loguru really shouldnt be an option.
Structlog feels like a more powerful and featured option but this brings with it the need to learn, understand a new system. Plus it still needs a custom "processor" to integrate with OTEL.
Structlog's biggest value prop - structured logging - is also now trivial through native logging with AI writing the JSON formatter classes.

So my recommendation is:

Hobby/Personal projects: where you want to spend the least amount of effort on logging, use loguru. An ideal print() replacement
Production projects: Use native logging but ensure you do structured outputs - offload to AI to take care of this - its well within its wheelhouse and is capable of doing a solid job.
Use structlog only if and when you need complex processing logic on your logs.

The one trade off is that loguru/structlog have good exception/stack trace handling capabilities built in. With native logging, you'll need to write more code and for this case, AI coding may get hairy.

P.S: Im yet to integrate into a log aggregation service (aiming at Signoz) so we'll have to wait and see how this decision pays off.

[-]

fenghuangshan@reddit

i never feel it's hard to native logging module , even you don't know logging module much , you can ask AI wirte it or just copy it from a famous project

You can just write it once , and copy to every porject

I have a log.py file with only one logger variable to export , with all necessary config , from other py file just use

from log import logger

logger.debug('')

then just use this logger for all logging funtction

[-]

rm-rf-rm@reddit (OP)

agreed. any recommendations on good projects to use as reference?

[-]

fenghuangshan@reddit

Logging — The Hitchhiker's Guide to Python

Python logging template

here are some examples, just take it as a start, adjust to your project , later you can have a stable logging module , then you can use it in all your projects

[-]

roerd@reddit

I find your reasoning that avoiding boilerplate is much less valuable now because LLMs can generate boilerplate very flawed. The main reason to avoid superfluous code has IMHO never been because it has to be written, but rather because it has to be maintained.

[-]

ANautyWolf@reddit

This

[-]

UloPe@reddit

And read and understood

[-]

omg_drd4_bbq@reddit

Loguru is nice for hobby projects, i would not use it for prod though. I highly recommend whatever you use, use one compatible with the std logging library. Custom logger classes are annoying for interop

[-]

JamzTyson@reddit

i would not use it for prod though

Why not?

[-]

rm-rf-rm@reddit (OP)

true - structlog is compatible with native logging and thus the recommendation to use that as the next level when more processing or additional tooling is needed to manage tracebacks, exceptions etc.

[-]

knobbyknee@reddit

Standard library logging is really the only sane option if a sysadmin or devops role is going to run the code in production. It can control logging with config files, which is what these people know and use.

Otherwise loguru is probably the simpler and better option.

[-]

UloPe@reddit

That reads like a take from 2002

[-]

cbarrick@reddit

I'm an SRE. I can setup whatever logging infrastructure you need. Doesn't matter to me if the feature devs choose a third party library.

[-]

syklemil@reddit

Using an LLM to get JSON output I would interpret as the developer not being able to find their own nose. People have been doing it for ages before LLMs showed up; it's not that hard.

[-]

dusktreader@reddit

Why can't you use loguru with open telemetry? It's a little more setup, sure, but it's pretty straightforward. Just wrap the mess up in an app. logging module with an init_logs() and you're golden.

I've used loguru across a lot of codebases and it's my preferred approach.

structlog has some sharp edges.

logbook is painful and (was) broken in a couple of places.

standard logging works... it's just not so friendly.

[-]

rm-rf-rm@reddit (OP)

Why can't you use loguru with open telemetry? It's a little more setup, sure, but it's pretty straightforward. Just wrap the mess up in an app. logging module with an init_logs() and you're golden.

why take on the task of bridging these gaps when the tool isnt offering you any sufficiently concrete benefit over native logging?

[-]

hmoff@reddit

How are you getting json out of native logging?

[-]

rm-rf-rm@reddit (OP)

Custom JsonFormatter, here's what I had Claude write me:

class JSONFormatter(logging.Formatter):
    """Custom formatter that outputs logs as JSON for file logging."""

    def format(self, record: logging.LogRecord) -> str:
        """Format log record as JSON.

        Parameters
        ----------
        record
            The log record to format

        Returns
        -------
        str
            JSON formatted log message

        """
        log_data = {
            "timestamp": self.formatTime(record, self.datefmt),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
            "module": record.module,
            "function": record.funcName,
            "line": record.lineno,
        }

        # Add extra fields if present
        if hasattr(record, "extra_data"):
            log_data.update(record.extra_data)

        # Add exception info if present
        if record.exc_info:
            log_data["exception"] = self.formatException(record.exc_info)

        # Add stack info if present
        if record.stack_info:
            log_data["stack_info"] = self.formatStack(record.stack_info)

        return json.dumps(log_data)

[-]

fnord123@reddit

This doesn't do structlogs bindcontextvars. E.g. a request to your service comes in and Middleware adds some trace id, endpoint stuff to the context. Then all other logs in that context will have the trace id info added.

[-]

dusktreader@reddit

this is the same guy that doesn't want to take the time to configure loguru with open telemetry 🙄

[-]

rm-rf-rm@reddit (OP)

can you share the code for that?

[-]

Orio_n@reddit

Loguru offers convenience. "Just use ai" isnt convenient when setting up all that boilerplate for logging

[-]

jazz_the_dinosaur@reddit

It does seem like you're cherry picking a bit saying AI can just handle setting up maybe logging, but then asking things like why bother bridging the gaps with loguru. Why not just let AI handle that?

I have used Python for long enough now and I have never loved the native logger. Can I fumble my way through it? Sure. Is it a bit convoluted? I think so.

The first time I used loguru it was like a breath of fresh air. It just worked and it worked well. Sure I can get AI to set up the native logger, but I will still have to use it and at that point I'd just rather use loguru.

[-]

rm-rf-rm@reddit (OP)

the tool isnt offering you any sufficiently concrete benefit over native logging

[-]

mattl33@reddit

What's the sharp edges with structlog? I recently added it to a cli tool at work, so it's not really "production" but so far I like it. Wondering what land mines to avoid.

[-]

Orio_n@reddit

Just use loguru. Unless you dont want external dependencies then logging

[-]

xcbsmith@reddit

Solve: use OTel and do tracing w/attributes instead of structured logging.

[-]

EarthGoddessDude@reddit

If you’re on AWS, aws-lambda-powertools is amazing. It gives you structured logging out of the box plus a whole bunch of other nice things. It even works on Fargate, though not all features.

[-]

Uncle_DirtNap@reddit

Logbook has a pretty rich set of features and is fully compatible with the standard logger. The key feature it provides is that you can add nested metadata at the following levels:

application
thread
greenlet
context block
message

and everything cascades nicely, so that you can, like, add a run_id at the application level when you start up, add thread info to all of your worker threads, then add various contextual values in the body of a function, and then when you say logger.info(“foo”) the resulting record object has all of the additional data.