Skip to main content
Saved

Pattern Resilience Intermediate

Stop Generation

A stop button aborts the fetch with an AbortController but keeps whatever text has already streamed in. The server needs to stop generating too, so you’re not billed for tokens the user will never read.

Examples
React, Vue, Svelte and native web
Read
4 min

The problem

The model got the question wrong and is now generating six paragraphs about the wrong thing. The user can watch it going wrong as the text streams in, but there’s no stop button, so they either wait for it to finish or refresh the page and lose the whole conversation.

The pattern

As a model response starts generating, you create an AbortController and pass its signal to fetch, then keep the controller while the response streams in. If the user clicks the stop button, your code calls controller.abort(), and whichever call is still in progress, the fetch or the next reader.read(), rejects with an AbortError. That ends the read loop, and your catch block should treat the AbortError as a normal stop, separately from real errors.

Problem

Even when there’s a stop button, it often only looks like it works. Clicking it might stop new text from appearing and re-enable the prompt input, but the request is still open, and the server is still generating tokens that you’re billed for.

A stop button is more useful on a model response than on most things in a UI. The response is slow, so the user has time to react, and because the model gives a different response each time, it often goes somewhere the user doesn’t want. You’re also billed for each token, so the tokens generated after the user has given up on a response still cost you money.

Solution

The tokens that already arrived should stay when the user stops a response. Your code can mark the message as stopped and leave its text in place, so the user keeps what they’ve read so far, and can use Response Regeneration if they want to try again. While the response streams, the stop button takes the place of the Prompt Input’s send button, and the send button comes back when the generation ends, whether it finished, was stopped, or failed.

Aborting the fetch stops the browser from reading the response, but on many setups the server keeps generating unless your code tells it to stop. Your server needs to detect when the connection closes, for example with res.on('close') in Express, and abort the upstream model call too. Without that, the user stops seeing new tokens, but you’re still paying for them.

Example

Each framework version below keeps the controller for the current response and has a stop function that aborts it.

Wiring the Abort

function useGeneration() {
  const [text, setText] = useState('');
  const [status, setStatus] = useState('idle'); // idle | streaming | complete | stopped | error
  const controllerRef = useRef(null);

  const start = async (prompt) => {
    const controller = new AbortController();
    controllerRef.current = controller;
    setText('');
    setStatus('streaming');

    try {
      const res = await fetch('/api/chat', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ prompt }),
        signal: controller.signal,
      });
      const reader = res.body.getReader();
      const decoder = new TextDecoder();

      while (true) {
        const { value, done } = await reader.read();
        if (done) break;
        setText(prev => prev + decoder.decode(value, { stream: true }));
      }
      setStatus('complete');
    } catch (err) {
      // Abort is an expected outcome, not an error
      setStatus(err.name === 'AbortError' ? 'stopped' : 'error');
    }
  };

  const stop = () => controllerRef.current?.abort();

  return { text, status, start, stop };
}

The Stop Control

The button only appears while the response is streaming, and its label tells screen reader users what it does.

function GenerationControls({ status, onStop }) {
  if (status !== 'streaming') return null;
  return (
    <button type="button" className="stop" onClick={onStop} aria-label="Stop generating">
      <StopIcon aria-hidden="true" /> Stop generating
    </button>
  );
}

Keeping the Partial Response

Stopping a response shouldn’t clear the message. The message is marked as stopped and keeps its text, and there’s a button to regenerate it.

function AssistantMessage({ text, status, onRegenerate }) {
  return (
    <article data-status={status}>
      <div className="content">{text}</div>
      {status === 'stopped' && (
        <footer className="stopped-note">
          <span>Stopped.</span>
          <button onClick={onRegenerate}>Regenerate</button>
        </footer>
      )}
    </article>
  );
}

Stopping the Work on the Server

Aborting the fetch closes the connection. The server can detect that and cancel the upstream model call, so the model stops generating too.

// Express handler proxying a streaming model API
app.post('/api/chat', async (req, res) => {
  const upstream = new AbortController();

  // When the browser aborts, the connection closes before the response
  // finishes, so stop generating too
  res.on('close', () => {
    if (!res.writableFinished) upstream.abort();
  });

  try {
    const modelStream = await model.chat({
      messages: [{ role: 'user', content: req.body.prompt }],
      signal: upstream.signal,
    });
    for await (const chunk of modelStream) {
      res.write(chunk);
    }
    res.end();
  } catch (err) {
    if (err.name !== 'AbortError') throw err;
  }
});

Benefits

  • Users can stop a response that’s going the wrong way, instead of waiting for it to finish.
  • If the server also cancels the model call, generation stops, so you don’t pay for tokens the user never reads.
  • The partial text stays when the user stops, so they keep what was useful and can regenerate the rest.
  • It uses the same reader and signal as Streaming Response, which you already have, so it takes very little code to add.

Tradeoffs

  • Aborting the fetch doesn’t stop generation on the server, and it’s easy to forget the connection-close handling, because the UI looks exactly the same without it.
  • AbortError arrives as a rejected promise, so if your code doesn’t handle it separately, a normal stop shows up as an error.
  • There’s a race condition if the user clicks stop just as the last token arrives, so you need to decide whether that counts as stopped or complete.
  • A partial response that’s cut off mid-sentence can be misleading, so users need to be able to see clearly that it was stopped.
  • These examples assume the send button is disabled while a response is streaming. If your app lets a new generation start before the last one ends, the old stream keeps adding its text to the new response, so start should abort the previous controller first and ignore any late updates from the old request.

Summary

Streaming model responses often go somewhere the user doesn’t want, and you pay for each token, so users should be able to stop them. You keep an AbortController while the response streams, call abort() when the user clicks stop, handle the AbortError as a normal stopped outcome, and keep the partial text. The server also has to cancel the upstream call when the connection closes, otherwise the button only stops new tokens from appearing, and you’re still paying for them.

See also

Next in Resilience Streaming Response A streaming response shows a language model's reply token by token as it arrives, so users don't have to wait for the whole reply to finish.

A monthly email
from Den Odell.

What I'm thinking about frontend patterns, plus news when the site changes.

No spam. Unsubscribe anytime.