Files changed (1) hide show
  1. README.md +28 -28
README.md CHANGED
@@ -71,7 +71,7 @@ to MOE/Mixture of expert models - both GGUF and source.
71
  (Updated: "INDEX", and added "Generation Steering" section ; notes on Roleplay/Simulation added, Screenshots of parameters/samplers added in quick reference section.)
72
 
73
  This document includes detailed information, references, and notes for general parameters, samplers and
74
- advanced samplers to get the most out of your model's abilities including notes / settings for the most popular AI/LLM app in use (LLAMACPP, KoboldCPP, Text-Generation-WebUI, LMStudio, Sillytavern, Ollama and others).
75
 
76
  These settings / suggestions can be applied to all models including GGUF, EXL2, GPTQ, HQQ, AWQ and full source/precision.
77
 
@@ -94,9 +94,9 @@ The settings discussed in this document can also fix a number of model issues (<
94
 
95
  Likewise ALL the setting (parameters, samplers and advanced samplers) below can also improve model generation and/or general overall "smoothness" / "quality" of model operation:
96
 
97
- - all parameters and samplers available via LLAMACPP (and most apps that run / use LLAMACPP - including Lmstudio, Ollama, Sillytavern and others.)
98
- - all parameters (including some not in Lllamacpp), samplers and advanced samplers ("Dry", "Quadratic", "Microstat") in oobabooga/text-generation-webui including llamacpp_HF loader (allowing a lot more samplers)
99
- - all parameters (including some not in Lllamacpp), samplers and advanced samplers ("Dry", "Quadratic", "Microstat") in SillyTavern / KoboldCPP (including Anti-slop filters)
100
 
101
  Even if you are not using my models, you may find this document <u>useful for any model (any quant / full source / any repo) available online.</u>
102
 
@@ -186,7 +186,7 @@ DETAILED NOTES ON PARAMETERS, SAMPLERS and ADVANCED SAMPLERS:
186
  - DETAILS on PARAMETERS / SAMPLERS
187
  - General Parameters
188
  - The Local LLM Settings Guide/Rant
189
- - LLAMACPP-SERVER EXE - usage / parameters / samplers
190
  - DRY Sampler
191
  - Samplers
192
  - Creative Writing
@@ -466,7 +466,7 @@ Source files / Source models of my models are located here (also upper right men
466
 
467
  [ https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be ]
468
 
469
- You will need the config files to use "llamacpp_HF" loader ("text-generation-webui") [ https://github.com/oobabooga/text-generation-webui ]
470
 
471
  You can also use the full source in "text-generation-webui" too.
472
 
@@ -474,16 +474,16 @@ As an alternative you can use GGUFs directly in "KOBOLDCPP" / "SillyTavern" with
474
 
475
  <B>Parameters, Samplers and Advanced Samplers</B>
476
 
477
- In section 1 a,b, and c, below are all the LLAMA_CPP parameters and samplers.
478
 
479
  I have added notes below each one for adjustment / enhancement(s) for specific use cases.
480
 
481
  <B>TEXT-GENERATION-WEBUI</B>
482
 
483
- In section 2, will be additional samplers, which become available when using "llamacpp_HF" loader in https://github.com/oobabooga/text-generation-webui
484
  AND/OR https://github.com/LostRuins/koboldcpp ("KOBOLDCPP").
485
 
486
- The "llamacpp_HF" (for "text-generation-webui") only requires the GGUF you want to use plus a few config files from "source repo" of the model.
487
 
488
  (this process is automated with this program, just enter the repo(s) urls -> it will fetch everything for you)
489
 
@@ -491,17 +491,17 @@ This allows access to very advanced samplers in addition to all the parameters /
491
 
492
  <B>KOBOLDCPP:</B>
493
 
494
- Note that https://github.com/LostRuins/koboldcpp also allows access to all LLAMACPP parameters/samplers too as well as additional advanced samplers too.
495
 
496
- You can use almost all parameters, samplers and advanced samplers using "KOBOLDCPP" without the need to get the source config files (the "llamacpp_HF" step).
497
 
498
  Note: This program has one of the newest samplers called "Anti-slop" which allows phrase/word banning at the generation level.
499
 
500
  <B>SILLYTAVERN:</B>
501
 
502
- Note that https://github.com/SillyTavern/SillyTavern also allows access to all LLAMACPP parameters/samplers too as well as additional advanced samplers too.
503
 
504
- You can use almost all parameters, samplers and advanced samplers using "SILLYTAVERN" without the need to get the source config files (the "llamacpp_HF" step).
505
 
506
  For CLASS3 and CLASS4 the most important setting is "SMOOTHING FACTOR" (Quadratic Smoothing) ; information is located on this page:
507
 
@@ -518,7 +518,7 @@ In Sillytavern you can then adjust parameters, samplers and advanced samplers ;
518
 
519
  Currently, at time of this writing, connecting Silly Tavern via KoboldCPP or Text Generation Webui will provide the most samplers/parameters.
520
 
521
- However for some, connecting to Lmstudio, LlamaCPP, or Ollama may be preferred.
522
 
523
  You may also want to check out how to connect SillyTavern to local AI "apps" running on your pc here:
524
 
@@ -527,21 +527,21 @@ https://docs.sillytavern.app/usage/api-connections/
527
 
528
  <B>Lmstudio, Ollama, Llamacpp, and OTHER PROGRAMS</B>
529
 
530
- Other programs like https://www.LMStudio.ai allows access to most of STANDARD samplers, where as others (llamacpp only here) you may need to add to the json file(s) for a model and/or template preset.
531
 
532
- In most cases all llama_cpp parameters/samplers are available when using API / headless / server mode in "text-generation-webui", "koboldcpp", "Sillytavern", "Olama", and "LMStudio" (as well as other apps too).
533
 
534
- You can also use llama_cpp directly too. (IE: llama-server.exe) ; see :
535
 
536
  https://github.com/ggerganov/llama.cpp
537
 
538
- (scroll down on the main page for more apps/programs to use GGUFs too that connect to / use the LLAMA-CPP package.)
539
 
540
  Special note:
541
 
542
- It appears "DRY" / "XTC" samplers has been added to LLAMACPP and SILLYTAVERN.
543
 
544
- It is available (Llamacpp) via "server.exe / llama-server.exe". Likely this sampler will also become available "downstream" in applications that use LLAMACPP in due time.
545
 
546
  [ https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md ]
547
 
@@ -1119,9 +1119,9 @@ Paper: https://arxiv.org/abs/2007.14966
1119
 
1120
  CRITICAL:
1121
 
1122
- If you activate Mirostat when using "LLAMAcpp SERVER" and/or some LLAMA_CPP based apps this will VOID/DISABLE all parameters (excluding "penalties", "logit_bias" ) AND all other SAMPLERS except "temp" parameter plus the following:
1123
 
1124
- V1: n_vocab(model) (this is set internally by llamacpp), seed, mirostat_tau, mirostat_eta
1125
 
1126
  V2: seed, mirostat_tau, mirostat_eta
1127
 
@@ -1133,7 +1133,7 @@ For Text Generation UI:
1133
 
1134
  No blocking occurs. Note that ONLY Mirostat 2 is available. (other parameters/samplers should work without issue)
1135
 
1136
- Note this is subject to change by LLAMAcpp, Koboldcpp, Text Generation UI and other AI/LLM app makers at any time.
1137
 
1138
  ("seed" is usually a random value. (default) ; this parameter can be set in some AI/LLM apps to control Mirostat output more closely.)
1139
 
@@ -1259,7 +1259,7 @@ You can use these samplers via Sillytavern IF you use either of these APPS (Kobo
1259
 
1260
  Other Notes:
1261
 
1262
- Hopefully ALL these samplers / controls will be LLAMACPP and available to all users via AI/LLM apps soon.
1263
 
1264
  "DRY" sampler has been added to Llamacpp as of the time of this writing (and available via SERVER/LLAMA-SERVER.EXE) and MAY appear in other "downstream" apps that use Llamacpp.
1265
 
@@ -1273,9 +1273,9 @@ https://github.com/oobabooga/text-generation-webui/wiki/03-%E2%80%90-Parameters-
1273
 
1274
  ADVANCED SAMPLERS - PART 1:
1275
 
1276
- Keep in mind these parameters/samplers become available (for GGUFs) in "oobabooga/text-generation-webui" when you use the llamacpp_HF loader.
1277
 
1278
- Most of these are also available in KOBOLDCPP too (via settings -> samplers) after start up (no "llamacpp_HF loader" step required).
1279
 
1280
  I am not going to touch on all of samplers / parameters, just the main ones at the moment.
1281
 
@@ -1446,7 +1446,7 @@ For additional details on these samplers settings (including advanced ones) you
1446
 
1447
  https://github.com/oobabooga/text-generation-webui/wiki/03-%E2%80%90-Parameters-Tab
1448
 
1449
- (NOTE: Not all of these "options" are available for GGUFS, including when you use "llamacpp_HF" loader in "text-generation-webui" )
1450
 
1451
  Additional Links (on parameters, samplers and advanced samplers):
1452
 
@@ -1464,7 +1464,7 @@ The Local LLM Settings Guide/Rant (covers a lot of parameters/samplers - lots of
1464
 
1465
  https://rentry.org/llm-settings
1466
 
1467
- LLAMACPP-SERVER EXE - usage / parameters / samplers:
1468
 
1469
  https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md
1470
 
 
71
  (Updated: "INDEX", and added "Generation Steering" section ; notes on Roleplay/Simulation added, Screenshots of parameters/samplers added in quick reference section.)
72
 
73
  This document includes detailed information, references, and notes for general parameters, samplers and
74
+ advanced samplers to get the most out of your model's abilities including notes / settings for the most popular AI/LLM app in use (Llamacpp, KoboldCPP, Text-Generation-WebUI, LMStudio, Sillytavern, Ollama and others).
75
 
76
  These settings / suggestions can be applied to all models including GGUF, EXL2, GPTQ, HQQ, AWQ and full source/precision.
77
 
 
94
 
95
  Likewise ALL the setting (parameters, samplers and advanced samplers) below can also improve model generation and/or general overall "smoothness" / "quality" of model operation:
96
 
97
+ - all parameters and samplers available via Llamacpp (and most apps that run / use Llamacpp - including Lmstudio, Ollama, Sillytavern and others.)
98
+ - all parameters (including some not in Llamacpp), samplers and advanced samplers ("Dry", "Quadratic", "Microstat") in oobabooga/text-generation-webui including Llamacpp_HF loader (allowing a lot more samplers)
99
+ - all parameters (including some not in Llamacpp), samplers and advanced samplers ("Dry", "Quadratic", "Microstat") in SillyTavern / KoboldCPP (including Anti-slop filters)
100
 
101
  Even if you are not using my models, you may find this document <u>useful for any model (any quant / full source / any repo) available online.</u>
102
 
 
186
  - DETAILS on PARAMETERS / SAMPLERS
187
  - General Parameters
188
  - The Local LLM Settings Guide/Rant
189
+ - Llamacpp-server - usage / parameters / samplers
190
  - DRY Sampler
191
  - Samplers
192
  - Creative Writing
 
466
 
467
  [ https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be ]
468
 
469
+ You will need the config files to use "Llamacpp_HF" loader ("text-generation-webui") [ https://github.com/oobabooga/text-generation-webui ]
470
 
471
  You can also use the full source in "text-generation-webui" too.
472
 
 
474
 
475
  <B>Parameters, Samplers and Advanced Samplers</B>
476
 
477
+ In section 1 a,b, and c, below are all the Llamacpp parameters and samplers.
478
 
479
  I have added notes below each one for adjustment / enhancement(s) for specific use cases.
480
 
481
  <B>TEXT-GENERATION-WEBUI</B>
482
 
483
+ In section 2, will be additional samplers, which become available when using "Llamacpp_HF" loader in https://github.com/oobabooga/text-generation-webui
484
  AND/OR https://github.com/LostRuins/koboldcpp ("KOBOLDCPP").
485
 
486
+ The "Llamacpp_HF" (for "text-generation-webui") only requires the GGUF you want to use plus a few config files from "source repo" of the model.
487
 
488
  (this process is automated with this program, just enter the repo(s) urls -> it will fetch everything for you)
489
 
 
491
 
492
  <B>KOBOLDCPP:</B>
493
 
494
+ Note that https://github.com/LostRuins/koboldcpp also allows access to all Llamacpp parameters/samplers too as well as additional advanced samplers too.
495
 
496
+ You can use almost all parameters, samplers and advanced samplers using "KOBOLDCPP" without the need to get the source config files (the "Llamacpp_HF" step).
497
 
498
  Note: This program has one of the newest samplers called "Anti-slop" which allows phrase/word banning at the generation level.
499
 
500
  <B>SILLYTAVERN:</B>
501
 
502
+ Note that https://github.com/SillyTavern/SillyTavern also allows access to all Llamacpp parameters/samplers too as well as additional advanced samplers too.
503
 
504
+ You can use almost all parameters, samplers and advanced samplers using "SILLYTAVERN" without the need to get the source config files (the "Llamacpp_HF" step).
505
 
506
  For CLASS3 and CLASS4 the most important setting is "SMOOTHING FACTOR" (Quadratic Smoothing) ; information is located on this page:
507
 
 
518
 
519
  Currently, at time of this writing, connecting Silly Tavern via KoboldCPP or Text Generation Webui will provide the most samplers/parameters.
520
 
521
+ However for some, connecting to Lmstudio, Llamacpp, or Ollama may be preferred.
522
 
523
  You may also want to check out how to connect SillyTavern to local AI "apps" running on your pc here:
524
 
 
527
 
528
  <B>Lmstudio, Ollama, Llamacpp, and OTHER PROGRAMS</B>
529
 
530
+ Other programs like https://www.LMStudio.ai allows access to most of STANDARD samplers, where as others (Llamacpp only here) you may need to add to the json file(s) for a model and/or template preset.
531
 
532
+ In most cases all Llamacpp parameters/samplers are available when using API / headless / server mode in "text-generation-webui", "koboldcpp", "Sillytavern", "Olama", and "LMStudio" (as well as other apps too).
533
 
534
+ You can also use Llamacpp directly too. (IE: llama-server.exe) ; see :
535
 
536
  https://github.com/ggerganov/llama.cpp
537
 
538
+ (scroll down on the main page for more apps/programs to use GGUFs too that connect to / use the Llamacpp package.)
539
 
540
  Special note:
541
 
542
+ It appears "DRY" / "XTC" samplers has been added to Llamacpp and SILLYTAVERN.
543
 
544
+ It is available (Llamacpp) via "server.exe / llama-server.exe". Likely this sampler will also become available "downstream" in applications that use Llamacpp in due time.
545
 
546
  [ https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md ]
547
 
 
1119
 
1120
  CRITICAL:
1121
 
1122
+ If you activate Mirostat when using "Llamacpp SERVER" and/or some Llamacpp based apps this will VOID/DISABLE all parameters (excluding "penalties", "logit_bias" ) AND all other SAMPLERS except "temp" parameter plus the following:
1123
 
1124
+ V1: n_vocab(model) (this is set internally by Llamacpp), seed, mirostat_tau, mirostat_eta
1125
 
1126
  V2: seed, mirostat_tau, mirostat_eta
1127
 
 
1133
 
1134
  No blocking occurs. Note that ONLY Mirostat 2 is available. (other parameters/samplers should work without issue)
1135
 
1136
+ Note this is subject to change by Llamacpp, Koboldcpp, Text Generation UI and other AI/LLM app makers at any time.
1137
 
1138
  ("seed" is usually a random value. (default) ; this parameter can be set in some AI/LLM apps to control Mirostat output more closely.)
1139
 
 
1259
 
1260
  Other Notes:
1261
 
1262
+ Hopefully ALL these samplers / controls will be Llamacpp and available to all users via AI/LLM apps soon.
1263
 
1264
  "DRY" sampler has been added to Llamacpp as of the time of this writing (and available via SERVER/LLAMA-SERVER.EXE) and MAY appear in other "downstream" apps that use Llamacpp.
1265
 
 
1273
 
1274
  ADVANCED SAMPLERS - PART 1:
1275
 
1276
+ Keep in mind these parameters/samplers become available (for GGUFs) in "oobabooga/text-generation-webui" when you use the Llamacpp_HF loader.
1277
 
1278
+ Most of these are also available in KOBOLDCPP too (via settings -> samplers) after start up (no "Llamacpp_HF loader" step required).
1279
 
1280
  I am not going to touch on all of samplers / parameters, just the main ones at the moment.
1281
 
 
1446
 
1447
  https://github.com/oobabooga/text-generation-webui/wiki/03-%E2%80%90-Parameters-Tab
1448
 
1449
+ (NOTE: Not all of these "options" are available for GGUFS, including when you use "Llamacpp_HF" loader in "text-generation-webui" )
1450
 
1451
  Additional Links (on parameters, samplers and advanced samplers):
1452
 
 
1464
 
1465
  https://rentry.org/llm-settings
1466
 
1467
+ Llamacpp-server - usage / parameters / samplers:
1468
 
1469
  https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md
1470