My humble opinion about experimental v3 release
I hope, my opinion is useful or at least is interesting.
In comparison to v2.1 of Lemonade-Unleashed, the v3 I do not like. The output of v3 feels more dry in comparison. It feels like the model goes over a checklist in its head, like, "This is first part, check. This is second part.. err... it's fine if I write a bit. Third part? Whatever. Fourth part, check.", does not even follow it well, forgets some parts of the prompt immediately, or perhaps ignores. Which would be fine, if it was selective, but it feels just... fractured? Maybe even broken, like, actively developed holes in its understanding. In addition, its understanding of the character seems degraded compared to v2.1, more akin initial version of Lemonade - the v3 is confused.
I checked that sha256 of my download is correct, so, the gguf file is not damaged. I tried both my favorite sampling settings (written in discussion of v2.1) for prior Lemonades as well as "stable baseline" settings (even without context quantization nor flash attention), and the result is largely same. Bad.
That being said, I am just one person. Maybe for someone else (be it due to style of prompting or personal preferences) this model is good.
Thank you for your time.
I was considering changing the name actually due to how much the model had changed, so that makes sense. I don't think there's a one size fits all for any 70B currently so it's fine to like the older versions. I've even seen some people prefer my original versions which were just merges.
The intent of this one was to burn out some of the Llama issues like inconsistently small replies and fix some instruction following, but I think I overdid it a little bit. I've seen other people mention fragmented sentences as an issue and I know it's got a bit of repetition in there.
Once I get my data in order, assuming nothing better releases I'll probably push another version through to try and fix these issues, as they're caused mostly by my own inexperience training.
Just as a quick side note, you could actually try raising the temp on this model. Someone mentioned they had a better experience with the temp higher and now that I think about it, dry prose and linear responses can be a symptom of a low temp. I just gave it a go at Temp 1.2, minp 0.03 with no other samplers but DRY and it felt a bit more similar to the older versions. Could be worth a shot.
Just adding on, I've just requested quants from mradermacher for an older version of this model.
https://huggingface.co/ApocalypseParty/L3.3-GeneticLemonade-Unleashed-v2.2-70B
I don't plan on making a full release of it, but this one might be more up your alley once the quants come out and should have a similar feel to 2.1.
It's this model, but without the DPO training to reinforce the instruction following, narrative style. It was a release candidate also, before I started messing around with other types of training. It has better prose than this version and a bit less repetition too I think, but you've gotta reign it in a bit more.
I tried your suggestions, thank you for 'em. The v2.2 of Lemonade felt like something in-between v2.1 and v3. Which makes perfect sense, I suppose! To my humble experience, v2.2 I did not like initially, felt a bit dry. But, I did like it more at higher temperature, tho, results were some great and some meh. I think that v2.2 is a peculiar middle ground, and that some people may like it. I also tried v3 at a higher temperature (1.2, as you suggested)... and liked it lots, to my surprise! Now what it outputs is pleasant for me to read! Also, seems to be more coherent/understanding, which is extra surprising. I am not yet sure if I'll use v2.1 or v3 (at higher temp), or both... but at the least, both are now excellent candidates as best mainly-RP model for me! With v2.1 being more chaotic and v3 being more logical, at least such is my impression so far, good impression for both. Once again, thank you - for good models and good advice!
(such is my opinion; opinions and experiences of other humans can differ)