Sillytavern best context template reddit. SillyTavern is a fork of TavernAI 1.
- Sillytavern best context template reddit It's a different experience from running models locally with kobold/llamacpp where I can run models past the "official" context size limit. As far as Sillytavern, what is the preferred meta for 'Text completion presets?' And for 'Advanced Formatting?' I've had success with the roleplay and Simple proxy presets, but open to hearing more! A place to discuss the SillyTavern fork of TavernAI. e. 05, as that's the best setting so far in my observation. I use dynamic temperature (I think I'm currently at 0 minimum and 4 maximum or 0. Edit your cards so that they are similar. Ultimately what makes a ‘good’ character is entirely subjective, you’re better off learning what you want in a character and experimenting with card building so that you can then apply your own ideas to characters - that imo is the most satisfying approach. 84 votes, 66 comments. SillyTavern as front end icefog72 model, quant of choice (I'm using exl2 8) ooga text gen Web Ui as back end In my experience I have had extremely immersive roleplay with Mythalion 13B 8tgi-fp16/8k context size from Kobold Horde (with an average response time of 13/20 seconds and no more than 50) and I must admit that it knows how to recognize the anatomy of the characters in a decent way without the need to use formats such as: Ali:Chat + Plist (which are not very much to my liking). 1 - 0. 7. Infermatic's service is pretty okay for $15 unlimited but the context size limits on a lot of their models can be so annoying. This field is a template for pre-chat character data (known internally as a story string). As far as the [ ] bracket goes. So i have a amd 580 with 8vram but so far nothing works correctly with my card and i dont think it has support for rocm. 8 which is under more active development, and has added many major features. Find your model's native INSTRUCT TEMPLATE. The quality of the roleplay would drop dramatically past 4k context. Very fast, and I think they're at least as good as the 13b models I mentioned (and better in some ways - larger context sizes and better at following instructions, generally). json file. If you want to modify it, I advise creating your own preset (+ button), to avoid messing the default ones. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. Odd little glitch in Windows 11: the Terminal window, which I have set to have the title "SillyTavern" gets changed to "Windows Powershell" when 'node server. Hey everyone, Since my last review on Steelskull/L3-Aethora-15B has generated some interest, I've decided to give a smaller 8B model a chance to shine against it. But going through the summaries it generates, I can see a lot of errors and it misses a lot of key details from the conversation. More than one model described the situation in such detail and what the characters see and do. Hi again! :^D I made this post yesterday, and after playing around with models (more or less), it seems that for my use, Mistral 8x7B is a good choice. Open-Orca/Mistral-7B-OpenOrca Synthia for, you guessed it, e. To the advanced users: What do you consider best practice? Especially related to mixtral and miqu (as those are currently the strongest long context models). json, the 'max_position_embeddings' Is 4k, which means without rope scaling(A process of extending context beyond its limits) model's context limit is 4k, I use 'alpha_value' flag (a rope scaling method) to extend beyond context limit to 6k, I calculate value of 'alpha_value' by using a formula(Yes there is a formula), or copying the recommended optimal value from wiki Hi everyone,I don't understand why ContextShift doesn't seem to work when I use SillyTavern with koboldcpp. I don't want to jeopardize everything for anyone else just That model appears to include two . I use alpaca with it and it works fine. Before, I usually preferred to inject the author's note at a depth of around 1-3 depending on my needs. The most important thing is that the character card is set up properly. tell it what details I'm sharing a collection of presets & settings with the most popular instruct/context templates: Mistral, ChatML, Metharme, Alpaca, LLAMA This field is a template for pre-chat character data (known internally as a story string). Since he used OAI setting, I assume he's using gpt. That could be a big reason. I turned off my jailbreaks and it does really well up until around 8k context, then starts getting squirrelly. You might have noticed I have no idea what I'm doing. I like my writing to reflect complexity of characters, getting into emotions and thoughts, and need the model to show understanding of this, separate the emotions/thoughts/actions of different characters, and if possible even recognize non-standard methods of communication such as telepathy. i tried big names like lm studio and ollama but they require rocm i belive becasue they seem to only want to use my cpu and not my amd A place to discuss the SillyTavern fork of TavernAI. It could be my story string / instruct template, I have no idea what Orca-Vicuna is so I just used something in the story string i. I would like to hear what you are using and what your thoughts and experiences are with the other SillyTavern Presets Usage Make sure to grab both context and instruct templates. js' executes. Notes, notes, notes. using command r plus on chat completion gives me widly different responses compared to text completion. A place to discuss the SillyTavern fork of TavernAI. best model for NSFW? Currently I am using neversleep/noromaid-mixtral-8x7b-instruct and it's a bit A place to discuss the SillyTavern fork of TavernAI. I've connected SillyTavern to ooba and had a few interactions that worked well. Pandora from MistralAI has opened a PR for SillyTavern to add corrected templates that properly accommodate all of Mistral's models (as their tokenizer behavior is slightly different) but for Nemo in particular, these should be functional. Hello again! Instead of another LLM comparison/test, this time I'll test and compare something very different On the model card for Mixtral-8x7B-Instruct-v0. A realistic context length to aim for imho is 32k. You might double check that you have SillyTavern set to send the right amount of tokens to the AI, or the “extra” tokens may just get discarded, and they will no longer instruct the chat (ie if you load a llama 1 or older Erybus model, which support 2048 token contexts, but you have Sillytavern set to output 4096 token contests, then as your chat gets longer, anything past the 2048th Ooba for ExLlamaV2 Kobold for GGUF You definetly can run small models with large context, at least by using exllamav2, there is bratowski (huggingface exl2 cool dude) who marks how much VRAM the model would eat for each quant/context, but with full cache, so you could even run something larger. This will open up Mixtral 8x7b as an option. When it stops generating mid-sentence due to hitting a target max token limit, the leftover part should be removed from the output automatically. I would like to hear what you are using and what your thoughts and experiences are with the other template and preset settings. 0 Release! with improved Roleplay and even a proxy preset. Import and use them when using 13B, they perform much better on average. SillyTavern Text Completion presets: Deterministic (I don't like randomness ;)) Response (tokens): 512. The depth level is just space, and only 10 messages have passed to understand this. This is somewhere that example messages help (by giving the model a lot of tokens to work with from the start), but a lot of the magic of NovelAI takes a fair number of tokens to really get going. What settings could I change to make the role play experience in SillyTavern be the most like Replika? Or if that’s not possible, just the best options I should roll with? Many thanks in advance! Hello! I have a question regarding my use of prompts, trying to understand if I'm doing something wrong or not. This works by comparing a hash of the chat template defined in the model's tokenizer_config. I remember llama3 8b models still working past 8k context. All I did is posting my context template from context template into the main prompt on chat completion. 32k context, it's fast, and it can RP with the best of them provided you actually put some time into the character card. These mistral based ones also work with 32k context. 1 and repetition penalty at 1. 5), etc and tried all Vicuna context templates. From using a combination of Character Description, Author's Notes, and Word Lore template, I write very short bullet-points about details but not like an event. I have detailed character (2000 tokens) and it works fine in Ooba's WebUI, I decided to try SillyTavern to get opportunity to use group chat, but while just testing chatting I can't get answer from my character, it just retells me it's story or tries to continue it from third person view. 11. . SillyTavern is a fork of TavernAI 1. N00b here who’s kinda overwhelmed with all the options available in the app. Ability to stay in character — it perfectly does so, even in group chats, remembering lore details from its card with practically zero issues. Read the SillyTavern documentation that describes how to use NAI and import the two example characters. Thank you for all your hard work. Make sure you have instruct mode turned on as well. While the { style bracket is what NovelAI instruct uses, the memory settings (which the preamble seems to be utilizing, Why Everything In Openrouter, even they have good models (Except OAI, claude) Like Mythomax 13B, Noromaid, it doesn't react to instruct, it gets out of context, force option it makes it more bad, always impersionating {{user}}'s I primarily write fiction that goes into lewd, but is not the primary purpose. ) and the advanced formatting tab (context templete or instruction templete). You also have access to the instructions sent that precede your messages. I normally start with around 12-13k since that's how much I prefilled it to bypass the filters. Fixed rare server crash on writing to stats. Click on default and you'll see a big list of setting to choose from. Click the big A on the top bar of silly tavern and under context template you'll find a most of settings to choose from as well including lighting 1. So far, it's been a mixed experience: the 32k context is nice, the responses are generated very fast, and it's not bad in **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. I regret paying 20 bucks for asha,I tried many different prompts and tweaks but none of them gave me satisfactory results,I prefer the 13b model mercury (mythomax) and found that it gave much more pleasing results and didn't go OOC as often as asha did,and asha also tends to be extremely flowery which some may prefer,but I dislike personally. - Gemini Pro API is probably the best free model. In the config. This format must be strictly respected, otherwise the model will generate sub-optimal outputs. app/usage/api I use oobabooga textui to make characters. Refer to official docs for more details: https://docs. 1, MistralAI writes regarding instruction format: . Any time I try to run a decent sized model I wind up seeing several seconds of "prompt evaluation" (viewed from the CMD window) before it even starts, and then when it does start, it may be 10 tokens per second slower than an exl2 file for the same quant/filesize/model. I'm personally running the Noromaid finetune of Mixtral at 20k context and that's good Brucethemoose RPMerge has 200K context, based off Yi-34b. 2. What are most of you using for the Context Template? Default or something else? I've been using the Default template and using the "Alpaca" preset with instruct enabled. And since I structure my lorebooks to have the entries come either before or after the author's note, it basically allowed me to control when lore entries or recent messages were more important. Important! Edit out the lines like on the example screenshot below in your SillyTavern -> public Best of Reddit; Topics; There is a lot of information floating around regarding prompting. I noticed a lot of these settings are shared with ooba. Claude does not actually run this community - it is a place for people to talk about Claude's capabilities, limitations, emerging personality and potential impacts on society as an artificial intelligence. I'm actually running it on CPU with KoboldCPP and 32k context, and while the speeds are slow, I don't find them intolerable, probably because of Context Shifting (which works quite well when you don't have any dynamically injected prompting, like World Info or Author's Note). SillyTavern Docs. In SillyTavern you can edit the "context Template" which allows you to tell ST what order your world info, character info, and other data gets sent in and also insert text in between. 1-Mixtral-8x7b-Instruct-v3-4. Temperature Feel free to play with this one, lower values are more grounded. Where to get/understand which context template is better or should be used. you can use another character template and ask the AI to use that as the format or simply tell it to make a character template. Also here Huge LLM Comparison/Test: 39 models tested (7B-70B + ChatGPT/GPT-4) you can find some other models beside 13b. What I would actually recommend, however, is Fimbulvetr-v2 or Kaiju. And a lot of the other role playing models on HF are 4k context. I'm quite sure that the "Story String" is source of the problem (what are the parameters that I can leave?). I may add that one in and see what I get. so yes it **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. Templates under "Advanced Formatting" matter a ton. It is a new type of model - some new class between creative and grounded - and that's most likely what a majority of people like about I've been using the Default template and using the "Alpaca" preset with instruct enabled. sillytavern. Currently, I How to use: Advanced Users: Enjoy, modify, forget. The context is being built and sent from a blank state during every generation, everything that exists outside of the context range is not considered for a generation. So I notice that on SillyTavern the default is that templates send the character card together with the system prompt. I haven't used WizardLM2, but I do enjoy Command+ via their own API through SillyTavern. If everything seems ok, install Sillytavern. For context and instruct templates, I used 2 different ones. There you can see the System Prompt. Silly tavern has presets, so use them. So, I use the Mistral small/medium and Mixtral 8x7B Instruct (beta)* (context of 32k), and my system prompt in advanced formatting is very long (2798 characters) + another prompt in the author's note (260 tokens), leaving the "main prompt" section in the slider(?) completely People like to believe that some custom context template or instruct mode settings will be a silver bullet that will make NAI work like chatgpt/Claud on every card and that is far from the truth. With some work, I got a model that was trained on 32k to get around 5500k. 8 which is under more active development, and has added many major The default chatml preset included in sillytavern should have all that if you're up to date. 3 of SillyTavern to save and import Novel ai presets (I was using 1. I get feeling weird about ERP with a Google chatbot but I can guarantee that they already know about all your weird porn interests Hmm, that is strange. I won't be doing a review of this model, because the context size is way too small for me in its current state (but it holds potential). It's also been giving me some of the best gens I've seen yet and it's 5 months old. 8 which is under more active development, and has added many major I have decided to remove old samplers and only keep basic presets, I want people to play around and find what works best for them. And use the NAI context template. Of course, you can use a regular role-playing game, but I heard Best practice is to use the 'NovelAI' context template (find it under the Advanced formatting panel, the big "A" icon). Some examples I heard: - Keep the character short, add information in lorebooks. And that's it! You're set! **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. This can be applied to In the User Settings tab, under Theme Settings, there's a drop-down menu called Chat Style. Characters answer me incoherent things, sometimes they don't finish their sentence (no matter how much I increase the response tokens) and on more than one occasion their AI breaks and they answer me with their instructions. Also, make sure to use Dynamic Temperature and Min P 0. Lmao. Addressed common complaints about the behavior of the context length slider: - fixed min context value 0 => 512, - removed 8k min limit for unlocked context, - adjusted slider steps for both unlock states, - increased debounce duration for type-in values 500ms => 2sec. Yo, thank you for this post. Recently (two days ago), I started using Mixtral 8x7B (so not the 'Instruct') because I was looking for a model with a large context (tired of being limited to 4k or 16k) for uncensored roleplay, and I found that one. json files in the repo that you can directly import into SillyTavern for both context and instruct template. I've been lucky and my chats with characters are almost coherent. 6+ Templates (Included): Mistral, ChatML, LLAMA3, > Match CONTEXT TEMPLATE & INSTRUCT TEMPLATE (by names - while loading them up in Silly Tavern UI) templates may be used by creators **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. Context (tokens): 32768 (same as max_seq_len in ooba) Add BOS Token: OFF. When using these presets you must set Example Messages Behavior: Never include examples otherwise they will be Corrected Context Template and Instruct Mode settings for SillyTavern. That's as far as my ambitions go for the time being, maybe until some documentation on the settings is made available. Preset management for Context templates and Instruct templates OpenRouter prompt cost calculations Support for Markdown tables Renamed Live2D extension to TalkingHead Proxy passwords hidden by default More NovelAI settings Chat Lazy Loading AI21 API support Per-chat CFG support HotKey: Escape key to close panels and popups. It should look something like this. If you do use this with Gemini Pro, Simple Proxy for Tavern context template seems to work well for me, with instruct mode turned off. If I have the context set to 16k, and that gets filled up Yeah, this happens quickly because Corrected Context Template and Instruct Mode settings for SillyTavern. Hidden in there is the Document View, which makes ST's interface less chat-y and more book-y. Good luck. Like everyone else here, I'm also experimenting with MythoMax 13B. Every fine-tune stands on a base model, which is usually specified by its creator in a manner of "We're presenting the first fine-tune of Mistral Small" or "Nemo on steroids" or "LLAMA 3 with our special sauce" etc. json file with one of the default SillyTavern templates. This is a subreddit dedicated to discussing Claude, an AI assistant created by Anthropic to be helpful, harmless, and honest. but not many people use the summarize tab, as i said in the post, the best summary is the one you write yourself but i use it as a base, and other users use other methods such as smart context and vector storage which i have never actually used so i can not help there, also some people prefer to put the summary in the card description, which should be the same as putting it in the Trying this for the first time and it is very impressive. - I got it, I was managed to load those setting to SillyTavern in Adavnced Formating, Context Template, Instruct Mode but some of them just giving information like "Alpaca tested" - so I'm assuming this may be related to context template like "Alpaca-Roleplay". Context template: Pygmalion Kobold preset Universal-Light with load Koboldcpp order (Set context size 8k) You may also use my old preset linked at bottom of OP, there's also Mirostat parameters there although dynamic temperature is always better if it's supported. Tweaking your system prompt is sometimes magical to the point it feels like an entirely FWIW Noromaid Mixtral may still be worth a shot even if you can't fit everything in VRAM. I know NovelAi is great, and I A place to discuss the SillyTavern fork of TavernAI. Setting up the prompts to decouple back-and-forth ST is designed for might take more work, but it should be definitely possible. Your best bet to try to mitigate this somewhat is by using the summarization extension. This is a more complicated installation. There is no one way to do that. For 13B, on its huggingface page, there are downloads for both a context and instruct template. I also have my max response length and target length set to 2000 tokens so that the agents have plenty of room to work. Import the preset, I switched the Context Template to Novel Ai, Tokenizer switched to Nerdstash v2 Kayra. It makes no sense to me. the long context is sent under the user role (most of the (Chat Completion users - feel free to skip that) A big rework of Instruct Mode just hit the staging branch (not yet a stable release). Tags & Folders: added the ability to show It in the drop down menu. Under Context Template I'm not able to see any preset for Vicuna. The more context you have, the better. ROPE could work to extend context but the trade-off in quality is likely not worth it on a model that is I've been using Noromaid-v0. Edit these settings in the "Advanced Formatting" panel. Hermans AshhLimaRP SillyTavern template seems to solve a brevity problem this model otherwise has when using the I would like to know what is the best setting in your opinion, to be honest, Mancer-mythomax gives me too much control and I don't even know what am I doing, most of my responses are too short, the story telling is rushing, and character's dialogues are My recommended settings to replace the "simple-proxy-for-tavern" in SillyTavern's latest release: SillyTavern Recommended Proxy Replacement Settings 🆕 UPDATED 2023-08-30! UPDATES: 2023-08-30: SillyTavern 1. If you're using openAI model, then 4k context size is totally normal lol. I'm using both with the Cohere API and the non-plus version is objectively better, like it's not even close. You may get better results using something different. My settings are pretty much default. You can have a great model and the wrong templates and it will output nothing but garbage. I'm currently running the default model and settings for summarization of Silly Tavern. when you look at system prompts in most datasets it's something like 2-3 lines setting up the nature and expected behavior of "the assistant". 12 votes, 15 comments. You will likely want to change the system prompt after selecting your instruct format. true. Still working out the details, but for now: Here is some suggested settings (from this post about the update) for making use of the new Roleplay preset, which has been working pretty well for me so far. I would start off by importing those into SillyTavern, then you can copy paste it to specific I have the noromaid v0. In your main prompt, add an instruction telling the LLM to handle OOC messages. I'm sure there's something going on that's causing this for me, but CMDR+ gets repetitive real quick, but so much so that at some point, it will not change a single letter in swipes or regeneration, no matter how much I crank up the sliders. 7 beforehand). I mean - sentence trimming is ON exactly to prevent the messages from getting broken mid-sentence. Instruct Mode: Enabled. It's not like pyg that needs to nerf the context size so you won't get oom. 120b+12k~16k context is the best experience, I'd recommend miqu-midnight (it's not very smart or creative, but the responses feel more "human", and hf's comment mentions that it's ranked at the top of the EQ bench) goliath is always a good choice, it's just the spelling mistakes and context that make it not the first choice anymore A place to discuss the SillyTavern fork of TavernAI. 4-bit KV cache in exllama fits ~60K context on 24gb VRAM. I'm using recommended settings in HuggingFace page for context template and instruct template. Yesterday I bought a subscription to use NovelAI on Sillytavern and it is giving me a lot of problems. That said, i'm not into non-con so I don't know if it starts throwing off refusals if you're into freaky stuff. 3 context and instruct templates. Added instruct/context templates for the Phi model, and variants of Llama 3 and ChatML with message names inclusion. The only thing I really adjust frequently is the context. The "Advanced search" option now sorts the search results by relevance. Hello everyone. For whatever reason, I've always found the GGUF versions to be too slow unless we're talking like a 7B or 11B model. System Sequence Prefix: SYSTEM: (with a space after the colon) A place to discuss the SillyTavern fork of TavernAI. NAI is an entirely different kind of AI and the 2nd toppy-m-7b - Download Hermans AshhLimaRP SillyTavern templates, then edit it with the quality jailbreak. You can pick a Context Template, which auto-selects a corresponding Instruct Preset if there is one. Pandora's templates are going to look different to the ones in this repo. However, even after searching around for answers, I couldn't find the optimal settings for temperature, etc. I also have "\n{{user}} " in my stop sequences without the * since I have a few cards that don't italicize actions. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. For example, there is a model MythoMax. 10. Care to share "Text Completion presets", "Context Template" and "Instruct Mode" for this model? How to best write character cards for it? ChatML (OpenAI's Chat Markup Language-inspired) context template and instruct prompt format, used by e. I updated my recommended proxy replacement settings accordingly (see above link). Honestly, it's not just the settings. MistralTrix-4x9B-ERP-GGUF is an interesting one too. Advanced Formatting Context Template: Default. Ensure that you are using the correct Context Template and Instruct Mode Formatting for the model you're using (Advanced Formatting Menu). However, if you have adjusted the context template as you can see in the embedded Reddit screenshot, you bypass this issue because the note is now in-between ### Instruction End and ### Response: . For "Context Template" and "Instruct Mode" I suggest Alpaca Roleplay. i haven't used mistral but claude 3 is a moderately more lenient than open ai's product, characters as long as their in a relationship can have sexual relations like say make love for example works and it be written well and fights also can have blood and implements and a degree of depth of bloody conflicts if written loosley and carefully but more leneint than you would think. Min-P Higher values chop off more probabilities. migtissera/Synthia-7B-v1. Context template is "Default", Instruct mode is disabled - I think - and set to roleplay. it's probably the context template but idk your backend or setup A place to discuss the SillyTavern fork of TavernAI. Fimbulvetr 11b v2 uses either alpaca or vicuna format. However, you won't get there on consumer hardware. After posting about the new SillyTavern release and it's newly included, model-agnostic Roleplay instruct mode preset, there was a discussion about if every model should be prompted accordingly to the prompt format established SillyTavern (Version): 1. Waiting for fine-tunes, which will rope it up successfully to at least 32k. (Though to be fair, the Wizard-Kun-Lake one got weird after 8k tokens) All the 8b models are going to be based off of Llama 3, and have 8k context windows. Both are 11b models, which means the Q5KM quantisation can vit into 12Gb of vRAM with 8k context. I'm specifically using Fimbulvetr variants mostly. Do i have to do something else or am i doing anything wrong? heres the prompt im using for both text and chat completion. SillyTavern adds an abstraction layer with a lot of text to your prompts, so you are getting a less direct sense of how your input affects the response. So far, most of their models (which are released often) are 32k context, which is very important for complex roleplay where context from characters, dialogue, lore books, etc fills up token space quickly and uncensored. Or find a template of an existing character you like and edit the settings. 9. What does this mean? Hopefully, SillyTavern will now be able to send more correctly formatted prompts to models that require strict template following (such as Mixtral and ChatML), as well as include some improvements to "system" role message Preset management for Context templates and Instruct templates OpenRouter prompt cost calculations Support for Markdown tables Renamed Live2D extension to TalkingHead Proxy passwords hidden by default More NovelAI settings Chat Lazy Loading AI21 API support Per-chat CFG support HotKey: Escape key to close panels and popups. When you're done, there's a lot of settings. g. Here there are some screeshots of my settings. Not being able to edit the actual data structure sent to the model is extremely limiting. I thought 32k context should've lasted longer than just 200 posts. I really want to test sillytaven for dnd like adventures and misc things, but im not sure if anyhting work with my setup on windows 10. But some models are being posted without specific information at all. Change context to desired context length. If you are already, try alternates. The creator suggests the universal light preset. Language models don't have any persistent memory, we give them a context that represents character information and chat history to receive the reply based on that. Am I missing something? Many Thanks All of GGUF models works best in KoboldCPP or you can get another versions at links (just look in top of description of model when follow links-there a list of versions most of times). if you get weird responses or broken formatting/regex, play with the sampler settings. 2 model for my A place to discuss the SillyTavern fork of TavernAI. I know the limit is somewhere around 8000 but I don't want to take any stupid risks. Presets: Vicuna 1. 1 Koboldcpp (Context set to 32k) So far, I've been using just one character to chat with, to test the max number of responses before context line shows up, and last night about 200 posts in it showed it's ugly face. 3 Finally, there's Libra-32B which comes from Envoid/Libra-32B but is actually a more universally useful Alpaca-like preset just like the Roleplay and simple-proxy-for I run at 4k context locally. 12. 0bpw-h6-EXL2 and it's pretty great at staying in character and it's not hallucinating often, as long as context stays inside the context length it's pretty consistent. Then for TextGen Preset, I'm using 6144 contexts size, Temperature 1, Min P 0. RP Stew V2 has 200k of natural context and worked perfectly fine in my tests even on the one as high as 65k. This is the main way to format your character card for text completion and instruct models. I had been struggling to get a context window over 3500k. The template I would strongly suggest using the Mistral templates (either the new ones in the staging branch of SillyTavern or some of the ones provided in this thread) since they give It works with context better - good job. I can't help much. Heyo, need sugestion. Bonus: If you only use the Alpaca instruction template, you can manually write things like ### Instruction: into your author's note by hand, in order to append example **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. 05, Rep Pen Range 2048, then the other sliders are at their disabled values, with do sample, skip special tokens, and temperature last boxes checked. I didn't like how a lot of the 32k context models would start to break down slightly right around that limit. I recently discovered Google Gemini offers API access to their basic model for free, and I've been trying it out. I've done the standard and followed all the suggestions on the official website of ST, so now I'm asking anyone here if they have any tips regarding settings, presets, context templates, or character card additions. For "AI Response configuration" I suggest universal light preset. Context size — I'm only interested in models with at least 32k of context or higher. As my story grows, you'll notice ST doesn't send back replies anymore so I increase the context a bit. 2. Derive templates option must be enabled in the Advanced Formatting menu. # Story string. If not though, you can download the template from the sillytavern github at this link for context, and this link for instruct presets, and put them in their respective folders in your sillytavern/public folder. but not many people use the summarize tab, as the best summary is the one you write yourself, this is because the summary is not perfect, and sometimes it adds things that did not happen, but I use it as a base, that i can then change as i want, other users use other methods such as smart context and vector storage which i have never actually used so i can not help there, also some Obviously you need version 1. Now I'm messing with LLaMA2-13B-Psyfighter2 and I'm on a 7500k context, when it was trained on 4k. Some Text Completion sources provide an ability to automatically choose templates recommended by the model author. 1. Beginners: Read further. For 20B, I use the default context template and the Alpaca instruct template. Screenshot: https://ibb. default, chatml, alpaca roleplay, rpstew (from my current main model, merged rp stew and rpstew 2. My only RP experience this far has been with Replika. For non commercial use it's completely free. Thanks for making these models! And, sure, my settings are all part of SillyTavern: Deterministic generation preset, Default context template, Rogue Rose or Roleplay instruct preset (with their default system prompts). Right now this is my KoboldCPP launch instructions. So I’ve decided to spin up the Sao10K/L3-8B-Stheno-v3. He's too good that it doesn't get interesting. My main question is in regards to the AI response configuration tab (Temperature, Top K, etc. Oh great idea with the "slow paced" style. Every week new settings are added to sillytavern and koboldcpp and it's too much too keep up with. SillyTavern includes a list of pre-made conversion rules for different models, but you may customize them however you like. My "Hehe" Method?Run Smart Context, make sure bot stays in their writing format, and notes. I tend to test at native context length because artificially extending that has rarely worked well for me. 5 minimum and 3 maximum, doesn't really seem to matter too much) with min_p at 0. co/F6KKT15 For example for the Miqu model the Vicuna prompt is recommended. No model has ever done such a smooth narration. Works great out to 60K context too, I feel like it just keeps getting smarter and more nuanced with the larger context. Pandora from MistralAI has opened a PR for SillyTavern to add corrected templates that properly accommodate all of I have installed the latest SillyTavern 1. 3. djwft xwmqkjv fzpod mrvsozz rgg gdrcb laelu met cchola umrz