Pattern Resilience Intermediate
Response Regeneration
A model gives a different response each time, so people regenerate on purpose. If your app keeps each earlier attempt as a variant that users can switch back to, they can compare responses without losing the previous one.
The problem
A user gets a response that’s almost right, with a good structure but the wrong tone. In a normal web app, there’d be nothing to do, because the same request returns the same response. A model gives a different response when you ask again, so it’s worth asking for another one.
The pattern
With regeneration, your app re-runs the same prompt and stores the new result next to the previous one, instead of replacing it. Each assistant turn stores a list of variants and the index of the one that’s on screen. Regenerating adds a new variant to the end of the list and makes it the one on screen, and a small ”‹ 2 / 3 ›” control lets the user page back and forth between attempts, so they never lose the previous response.
Problem
Some apps don’t have a regenerate option at all, so the user has to retype or rephrase the prompt to try again. Others have a regenerate button that overwrites the previous response, so if the new one is worse, the good-enough version they had is gone. The user might end up regenerating again and again, hoping the next one is better, with no way to get back to the one they liked.
Often nothing went wrong with the first response at all. The user just wants to compare two responses and keep the better one, and they can’t do that if the second one replaces the first.
Solution
Under the hood, regeneration goes through the same generation steps as the original response, so the Streaming Response, Stop Generation, and Retry patterns all apply. The only difference is where the result is stored. A Message Thread already identifies each turn by its id, so adding a variants array to the assistant message is a small change.
Regenerating reuses the exact same prompt. Editing a prompt and resubmitting it is a different action, because the prompt changes, so everything after it is out of date. The thread is cut off at that point and generation starts again from there, and the old branch is only kept if your app supports branching. If your API allows it, you can also nudge the model toward a different response when the user regenerates, by raising the sampling temperature or adding an explicit “try a different approach” instruction, so the new variant doesn’t come back almost the same as the last one.
Example
In these examples, all the attempts for a turn are stored in one assistant message, so the rest of the thread doesn’t change when the user regenerates.
Modeling Variants
An assistant message stores every attempt and the index of the one that’s on screen.
// An assistant turn with switchable attempts
// { id, role: 'assistant', variants: [{ id, content, status }], activeVariant: 0 }
function messageReducer(message, action) {
switch (action.type) {
case 'regenerate': // append a fresh, streaming variant and switch to it
return {
...message,
variants: [...message.variants, { id: crypto.randomUUID(), content: '', status: 'streaming' }],
activeVariant: message.variants.length,
};
case 'token': // target the streaming variant by id, so paging away mid-stream is safe
return {
...message,
variants: message.variants.map(v =>
v.id === action.variantId ? { ...v, content: v.content + action.chunk } : v
),
};
case 'switch': // page between attempts without losing any
return { ...message, activeVariant: action.index };
default:
return message;
}
}
Regenerate and Switch Between Attempts
function AssistantTurn({ message, onRegenerate, onSwitch }) {
const { variants, activeVariant } = message;
const active = variants[activeVariant];
const many = variants.length > 1;
return (
<article className="assistant-turn">
<div className="content">{active.content}</div>
<footer className="actions">
{many && (
<div className="variant-pager" role="group" aria-label="Response versions">
<button onClick={() => onSwitch(activeVariant - 1)} disabled={activeVariant === 0}>‹</button>
<span>{activeVariant + 1} / {variants.length}</span>
<button onClick={() => onSwitch(activeVariant + 1)} disabled={activeVariant === variants.length - 1}>›</button>
</div>
)}
<button onClick={onRegenerate} disabled={active.status === 'streaming'}>
Regenerate
</button>
</footer>
</article>
);
} <template>
<article class="assistant-turn">
<div class="content">{{ active.content }}</div>
<footer class="actions">
<div v-if="variants.length > 1" class="variant-pager" role="group" aria-label="Response versions">
<button @click="onSwitch(message.activeVariant - 1)" :disabled="message.activeVariant === 0">‹</button>
<span>{{ message.activeVariant + 1 }} / {{ variants.length }}</span>
<button @click="onSwitch(message.activeVariant + 1)" :disabled="message.activeVariant === variants.length - 1">›</button>
</div>
<button @click="onRegenerate" :disabled="active.status === 'streaming'">Regenerate</button>
</footer>
</article>
</template>
<script setup>
import { computed } from 'vue';
const props = defineProps({ message: Object });
const emit = defineEmits(['regenerate', 'switch']);
const onRegenerate = () => emit('regenerate');
const onSwitch = (index) => emit('switch', index);
const variants = computed(() => props.message.variants);
const active = computed(() => variants.value[props.message.activeVariant]);
</script> <script>
let { message, onRegenerate, onSwitch } = $props();
let variants = $derived(message.variants);
let active = $derived(variants[message.activeVariant]);
</script>
<article class="assistant-turn">
<div class="content">{active.content}</div>
<footer class="actions">
{#if variants.length > 1}
<div class="variant-pager" role="group" aria-label="Response versions">
<button onclick={() => onSwitch(message.activeVariant - 1)} disabled={message.activeVariant === 0}>‹</button>
<span>{message.activeVariant + 1} / {variants.length}</span>
<button onclick={() => onSwitch(message.activeVariant + 1)} disabled={message.activeVariant === variants.length - 1}>›</button>
</div>
{/if}
<button onclick={onRegenerate} disabled={active.status === 'streaming'}>Regenerate</button>
</footer>
</article> class AssistantTurn extends HTMLElement {
set message(m) {
const active = m.variants[m.activeVariant];
const many = m.variants.length > 1;
this.innerHTML = `
<div class="content"></div>
<footer class="actions">
${many ? `<span>${m.activeVariant + 1} / ${m.variants.length}</span>` : ''}
<button class="regen">Regenerate</button>
</footer>`;
// Model output goes in as text, not HTML
this.querySelector('.content').textContent = active.content;
this.querySelector('.regen').onclick = () =>
this.dispatchEvent(new CustomEvent('regenerate'));
}
}
customElements.define('assistant-turn', AssistantTurn); Nudging the Model to Differ
If each regeneration comes back with almost the same text, raising the sampling temperature or adding an explicit instruction can make the new variant different.
function regenerate(messages, previousAttempts) {
return model.chat({
messages,
// Raise the temperature a little with each attempt, so the responses vary more.
// Check your provider's allowed range: some cap temperature at 1.
temperature: Math.min(0.7 + previousAttempts * 0.1, 1),
});
}
Edit-and-Resubmit Truncates the Thread
Changing an earlier prompt makes everything generated after it out of date, so the thread is cut off at that turn and generation starts again from there.
function editAndResubmit(messages, editedId, newText) {
const index = messages.findIndex(m => m.id === editedId);
// Everything after the edited prompt is now stale
const kept = messages.slice(0, index);
return [...kept, { ...messages[index], content: newText }];
// …then start a new generation from this truncated history
}
Benefits
- Users can keep asking for a better response without retyping the prompt.
- Because the app stores each attempt as a variant, regenerating doesn’t throw anything away, and the previous response is still a click away.
- The pager lets users compare versions and pick the one they prefer, which is why they asked the model the same thing twice in the first place.
- Regeneration reuses the streaming and stop code you already have, so most of the work is changing how the state is structured.
- With edit-and-resubmit, users can fix their original prompt instead of arguing with the model in follow-up messages.
Tradeoffs
- Each variant you keep is a full response stored in memory, and in storage too if you save conversations, so memory usage and storage use can add up in long conversations.
- The pager adds UI and state that have to stay in sync while a new attempt is still streaming.
- If you don’t nudge the model, regeneration can return almost identical responses, and the feature might feel broken to users.
- Edit-and-resubmit has to delete the turns that came after the edited prompt, so users lose them unless you build full conversation branching.
- Once a turn has several variants, you have to decide which one to save or export as the response, and the right choice isn’t always clear.
Summary
A model responds differently each time, so regenerating is a normal part of using it. Each regeneration is stored as a new variant next to the previous attempts, and a pager lets users compare them and choose one. The generation itself reuses your streaming and stop code, and a nudge to the sampling temperature or the prompt helps the variants come out different. Editing an earlier prompt is the one case where something gets thrown away, because the thread is deliberately cut off at that point.
See also
- Stop Generation — Resilience
- Message Thread — Components
- Streaming Response — Resilience
- Retry — Resilience
- Prompt Input — Components
- Temperature parameter - OpenAI API reference
- Regenerating responses with useChat - AI SDK
- Structuring Reducers - Redux