How disable the default LLM for Voice Chat Mode

While using local LLM in voice chat mode the along with local LLM the default LLM is also getting called. Now when I give a voice input first the default LLM is getting called (and the result is displayed) and I interrupt it and stop it from the avatar speaking. And then later the Local LLM result is spoken by the Avatar. So how to stop/disable the default LLM