Streaming
Sending a model's answer as it is produced, token by token, instead of waiting for the whole reply. A reader starts reading sooner, and a bad answer can be stopped mid-way.
- See also: Token
- See also: LLM
- Used in: AI Chatbot
- Used in: LLMs
Sending a model's answer as it is produced, token by token, instead of waiting for the whole reply. A reader starts reading sooner, and a bad answer can be stopped mid-way.