Meta releases SAM Audio for audio separation

Posted by umarmnaq@reddit | LocalLLaMA | View on Reddit | 19 comments

>SAM Audio separates target and residual sounds from any audio or audiovisual source—across general sound, music, and speech. [https://ai.meta.com/samaudio/](https://ai.meta.com/samaudio/) [https://huggingface.co/collections/facebook/sam-audio](https://huggingface.co/collections/facebook/sam-audio) [https://github.com/facebookresearch/sam-audio](https://github.com/facebookresearch/sam-audio)

Reply to Post

19 Comments

[-]

MrAlienOverLord@reddit

needs 33-gb in vram - needs to be chunked in 30 sec intervals otherwise it overfills a 48g gpu its very "picky" what works and what doesnt .. the samples a very cherry picked

[-]

Aggravating-Coder@reddit

Have you tried the smaller models as well? What version of CUDA are you running (I wasn't able to get it running on 13) on a 50XX series card.

[-]

MrAlienOverLord@reddit

i found the large one to be unreliable with separation tasks ( could be my prompting skills ) ..and the small done's did way worse .. my problem is i have a corpus of many TB to go throw and had hopes it would replace cleanup actions with rx11 for me

[-]

SrijSriv211@reddit

I'm not sure but didn't they use this in Instagram? I saw a reel which was in different language originally but Meta AI changed the Audio Track to english and it was so convincing & realistic that I learned it was actually a translated video through the comment section. If that's true Meta is definitely cooking something. I hope they are cooking something with Llama 5 as well.

[-]

Lazy-Pattern-5171@reddit

Yes. I see posts that are part of my “recommendations” now that are in foreign languages with translated by Meta tag. It’s not perfect yet it just mutes parts of audio sometimes but it’s an amazing feeling to know that we might soon be brought closer with our recommendations without being divided by language or background.

[-]

SrijSriv211@reddit

Yeah. Meta is definitely cooking something very hot.

[-]

LocalLLaMA-ModTeam@reddit

Rule 1. Posted 2 days ago already. See link in top comment

[-]

Few_Painter_5588@reddit

I wonder if this can be useful for generating a dataset for voice diarization.

[-]

Barry_Jumps@reddit

Rather than training diarization, it seems like this IS the diarization. Isolate the speakers into separate tracks, extract a transcript and done.

[-]

Few_Painter_5588@reddit

It's a bit to expensive to be used just for diarization. If it's trained in FP16, the large model is probably 30 billion parameters in size. Most diarization models are probably a hundredth of that

[-]

xadiant@reddit

This makes no sense, fyi. Any size model can be trained in any precision like fp64, fp32, mixed, fp16, bf16...

[-]

po_stulate@reddit

I think they were trying to say that given the size of the model, it is probably a multi billion parameters model, while an average model for this job usually only needs way less parameters.

[-]

mikael110@reddit

No, it's not remotely that big. The sizes are 500M for the small, 1B for medium, and 3B for Large.

[-]

Few_Painter_5588@reddit

My bad, I meant to say 8 billion if it's at FP16. But if it's 3B, that means the model is at FP32 which still uses a lot of VRAM.

[-]

Gretian15@reddit

Time to boot up FL Studio

[-]

MrPecunius@reddit

MLX wen?

[-]

Qwen30bEnjoyer@reddit

Haven't played with this yet, would it be any good at seperating the voices of multiple speakers for diarization?

[-]

mikael110@reddit

Previous discussion can be found here: [Meta announced a new SAM Audio Model for audio editing that can segment sound from complex audio mixtures using text, visual, and time span prompts.](https://www.reddit.com/r/LocalLLaMA/comments/1po7i0c/meta_announced_a_new_sam_audio_model_for_audio/)

[-]

HareMayor@reddit

More like ASM - Artificial Surveillance Model, lol. But, hey, now everyone can have it.