I ran into something recently that changed how I think about ML evaluation.

I trained a fraud detection model and got \~99% F1.
At first, it felt like a huge win.

But after digging deeper, I realized it was caused by data leakage — the model was learning patterns it shouldn’t have had access to.

What surprised me was that nothing warned me.

No errors.
No obvious issues.
Just a “perfect” score.

Since then, I’ve been much more careful about:

• checking for leakage
• watching for “too good to be true” metrics
• validating features more strictly

It made me realize how easy it is to build a model that looks great but fails in real-world use.

💬 Curious:

How do you usually detect data leakage early?

Do you rely on:

• manual checks
• validation strategies
• or specific tools?