Sentiment Analysis Model Guidance
Posted by Reno911-07078@reddit | LocalLLaMA | View on Reddit | 8 comments
What would be the best model to analyze text sentiment (Positive/Neutral/Negative) as part of a daily workflow analyzing 25,000-500,000 snippets of text (1-3 sentences).
I am looking for accuracy and speed. I tried some cheap methods of FinBERT/RoBERTa w VADER but got mixed results.
Added llama 3 8B to the flow but it’s slower than I expected and I’m honestly new to this in general, so not sure which model would be best or most appropriate for this use case.
I’m on apple silicon but in between hardware so I don’t have the specs. Will mostly land around 64-128 GB memory.
Thank you 🙏
Jolly-Gazelle-6060@reddit
Have you tried fine-tuned models from HF? there are quite a few...
Unless you have a very niche use case, in that case fine-tuning would be right path.
Reno911-07078@reddit (OP)
Thank you. Like a fine tuned llama version?
I’m using 3 RoBERTa models trained on different content: Stocktwits, Reddit, and Twitter. And a similar model for FinBERT. Trying to get them to learn from each other but now thinking maybe drop them all and just switch to a single true llm.
Jolly-Gazelle-6060@reddit
that's what I had in mind :)
I think you might get better results with a larger model if your hardware allows for it. You can always fine-tune different adapters for different use cases afterwards in case you want to further optimize.
If you don't have the data distilling a small model is also an interesting option, I have had quite some success with it for my tasks at least.
DunderSunder@reddit
Use a strong LLM to label 5-10k samples. Then finetune a BERT/Roberta/ModernBert model on them.
Constant_Leg_4107@reddit
In the paper "A multi-label text sentiment analysis model based on sentiment correlation modeling", they provide a table for the SOTA for sentiment analysis methods (as of Dec 2024):
Reno911-07078@reddit (OP)
Thank you!
Shivacious@reddit
are u looking for personal usage or professional op?
Reno911-07078@reddit (OP)
Personal use but treating it like a professional operation. As much as I can.