Oh, I can get smaller models to run reasonably fast but I'm very interested in tool calling and I'm having a hard time finding a model that runs fast and is good at calling tools locally (I'm sure that's due to my own ignorance).
I decided on openai api for now after setting up so many differnt methods. the local stuff isn't up to snuff yet for what I am trying to accomplish but decent for basic control.
I use a combo of Anthropic and OpenAI for now through my bots and my chat UIs and that lets me iterate faster. My hope is once I've done all my testing I could consider moving to local models if it made sense.
