fancyfeast

/

llama-joycaption-beta-one-hf-llava

like 57

I will try out to make it work, maybe the recent changes in llama.cpp make it possible (;

bad news currently there is more work needed

I have managed to create a Q8_0 gguf and a mmproj gguf, now i need to test inference

Is there a way that we can use in LMStudio as GGUF Qs?

are you currently online?

Is there a way that we can use in LMStudio as GGUF Qs?

are you currently online?

Sorry, I was sleeping.
Ready to test stuff!

I created a FP8 version for vLLM inference, should work on 16GiB VRAM cards.

Edit Misread your post, nevermind.

Is there a way that we can use in LMStudio as GGUF Qs?

are you currently online?

Sorry, I was sleeping.
Ready to test stuff!

didnt get it working yet, ill need to implement stuff for that in llama.cpp, if that will be successful idk xD

Hi all, I have converted Joycaption beta one to GGUF including the mmproj file.
You can get it at https://huggingface.co/concedo/llama-joycaption-beta-one-hf-llava-mmproj-gguf

Hi all, I have converted Joycaption beta one to GGUF including the mmproj file.
You can get it at https://huggingface.co/concedo/llama-joycaption-beta-one-hf-llava-mmproj-gguf

Nice thank you! There were way to many model releases to get this working too for me 😅

Hi all, I have converted Joycaption beta one to GGUF including the mmproj file.
You can get it at https://huggingface.co/concedo/llama-joycaption-beta-one-hf-llava-mmproj-gguf

works flawlessly in lmstudio too (;
Mind looking into ovis 2? Those multimodal models are also insane (especially for agent use i think, they are perfect to understand websites etc), but i didnt get them to work as ggufs yet

Ovis2 also works good for captioning positioning etc better than joy, but joy can do more detail and nsfw stuff, so a combination would probably be best