small cli REPL for interacting with apple foundation model on macOS
Posted by claykos@reddit | LocalLLaMA | View on Reddit | 0 comments
I think you already know that macOS has a builtin llm ...
it is a small model , 4k context... but maybe fun to play with it .
below is a repo , but you don't need to use this .
you can interact with it in python :
python3 -m pip install apple_fm_sdk
python3 -c "import asyncio, apple_fm_sdk as fm; print(asyncio.run(fm.LanguageModelSession().respond('What is the capital of Romania?')))"
brew tap kosminus/fm && brew install fm