Do you pay for curated datasets, or is scraped/free data good enough?
Posted by Lost_Transportation1@reddit | LocalLLaMA | View on Reddit | 18 comments
Genuine question about how people source training data for fine-tuning projects.
If you needed specialist visual data (say, historical documents, architectural drawings, handwritten manuscripts), would you:
a) Scrape what you can find and deal with the noise
b) Use existing open datasets even if they're not ideal
c) Pay for a curated, licensed dataset if the price is right.
And if (c), what price range makes sense? Per image, per dataset, subscription?
Alena_laistea@reddit
depends on the stage. for quick experiments, scraped/free data is usually enough. but once you move toward something production-like, the trade-offs start showing up
natasa_nattes@reddit
yeah especially with noise and bias. you spend so much time cleaning that it kind of defeats the free part
lelaniey_karoline@reddit
we ended up using a hybrid approach.
Sporta_narres@reddit
depends on the stage. for quick experiments, scraped/free data is usually enough. but once you move toward something production-like, the trade-offs start showing up
Mundane_Concept_5196@reddit
You use all three, but at different stages.
a) Scrape: good for early prototyping. Fast, cheap, messy. Fine to test if the task is learnable. Not great for production.
b) Open datasets: best starting point. Useful for baselines and benchmarking, but rarely aligned enough for real product performance.
c) Curated, licensed data: what you use when it actually matters (customers, reliability, defensibility). Cleaner distribution, better labels, fewer samples needed to reach performance. The price is usually per dataset with subscription for continuous updates if you need them. Some observability capability for evaluation is a huge plus for iterations. Start with a small paid pilot dataset.
If we’re building something customer-facing, we almost always end up paying for curated data.
TheRealMasonMac@reddit
Most open datasets are poopoo. Make your own if you can.
Cool-Tell3963@reddit
Commissioning experts sounds expensive af but probably worth it if you're doing something actually important and not just experimenting
TheRealMasonMac@reddit
Yep. Garbage in -> Garbage out
MoistRecognition69@reddit
Scraping is nice, but you reach a point where you simply run out of HQ sources.
Also, time = money - sometimes it's cheaper to just buy a pre-curated, HQ one than to make one on your own.
GatePorters@reddit
If you are making a model, you should personally curate the data you collect, not matter how you collect it.
I’m not saying check every single word, but any effort you put into refining your dataset for your use case will pay off more than scraping a million useful things blindly.
Ready-Interest-1024@reddit
Scraping is almost always more than enough (if not better). You just need to make sure you are picky with what you are giving to the model. Clean scraped data is incredibly powerful and can also be really challenging to get.
You just want to make sure you pull out exactly what you want (I prefer just raw scraping vs. the LLM powered stuff) and none of the fluff. You'll need to keep browser fingerprinting and proxies in mind for most sites now.
toothpastespiders@reddit
Visual historical data complicates things. With standard digital text I'm 100% on handling everything myself. But historical documents that 'only' exist in image format? I've been intentionally avoiding anything with them for the most part.
I'm basically just waiting around for things to advance far enough that some kind of automation could cover the bulk of the process. I don't know how bad the documents you're interested are. But for my particular focus it's generally a total nightmare. Messy, time faded, writing combined with archaic grammar is rough going for automation.
If I needed it, like really needed it? I'd probably pay if that was an option simply because it'd be too time prohibitive to do it myself. And if I had the money to spend. I mean each 'letter' in one of those things can be a chore. But at the same time I doubt it'd be an option. I'd need assurance that the people doing it were familiar enough with that place and time to understand the context they were working with in order to compensate for ambiguity.
So personally? I'm just waiting in hopes that tech will solve this issue for me eventually.
cosimoiaia@reddit
Curated on different levels. Everyone with a bit of experience knows that every dataset you download needs refinement for production purposes. There is no pay and use, there are ml engineers and data scientist who do the work, so it's always paid for production but in salaries.
Vegetable-Second3998@reddit
Scrape and curate your own. You want both clean AND the noise - you need to teach the bot to differentiate. So, your training set needs to include "noise" that you can show how it becomes structured (think before/after of cleaned up visual data). If all you do is train on perfectly clean data, then any variance in the real world causes hallucination or isn't recognized at all. You need to train not just on the outcome, but the process to get to the outcome.
Caryn_fornicatress@reddit
depends on goal and risk tolerance
for quick experiments or MVP scraped and open data is fine even noisy, for anything production or domain critical curated wins every time, cleaning scraped data costs more than people expect
most people pay only when data quality directly impacts output or legal risk matters, pricing that works is usually per dataset not per image, low four figures for niche sets, subscriptions only make sense if updates matter
Lost_Transportation1@reddit (OP)
What would make a niche dataset worth five figures instead of four?
Whole-Assignment6240@reddit
What quality threshold makes scraped data worth the noise?
Mythline_Studio@reddit
Scraped.
Curated only makes sense if the domain is narrow, label/licensing justifies.