Recent LLMs for local Estim AI

Ainavirin

The speed at which new AI models evolve is insane.

The list of suggested LLM models on the guide (https://eu.daimonia.app/articles/estimAI-start) is getting quite old.
Do you have any recent model suggestions for 16GB VRAM ?

Also, do you plan to support Ollama? It seems to be used by more and more sites for its ease of use...

KairosJS

I'm not planning to support Ollama, as I think KoboldCpp is light and easy to use. But I will keep an eye on it to check if the integration can be done quickly. Last time I checked (a few years ago), their API was different from other software.

I haven't tested local models for quite some time, but the recently released base models to look for abliteration/finetunes are:

Gemma 4 (12B for smaller GPUs)
Qwen 3.5 & Qwen 3.6 (there is a Qwen 3.5 9B for smaller GPUs)

Otherwise, it's probably still going to be some Mistral finetunes.

yngmstr

Hi,
where can we find these finetunes?

KairosJS

yngmstr I've answered to a similar request here:
https://space.daimonia.app/d/21-ai-model-comparison-thread/12

estimer

how can i connect a local llm to eudaimonia?

KairosJS

estimer You can find a tutorial here:
https://eu.daimonia.app/articles/estimAI-start

It's outdated, but the koboldcpp installation is the same.