mARTin-B78
ce3ae79132
Confirmed in production: a streaming attribution/character-sheets request held _attribution_llm_lock for 11+ minutes, well past its configured timeout, silently rejecting every subsequent passage as "Attribution engine busy" and falling back to the non-streaming view - looking exactly like live-thinking had stopped working, when actually one earlier request never finished. Root cause: requests' timeout= on a stream=True call only covers the connect + first byte, not gaps between later body reads. If the LLM backend goes silent mid-stream (connection left open, no more chunks), the blocked socket recv() inside iter_lines() can hang indefinitely. Since that's a native blocking call, not a Python-level yield point, neither an in-loop wall-clock check nor GeneratorExit from a disconnected client can interrupt it - both only take effect at the next bytecode boundary, which never arrives while blocked in the C extension. Added _watchdog_close: a daemon thread that force-closes the upstream connection if the wrapped block hasn't finished within the configured timeout. Closing the socket from another thread makes the blocked recv() raise, unblocking the generator so its normal except/finally cleanup (including releasing the lock) actually runs. Verified against the live LLM backend post-restart: stream completes normally with real token-by-token deltas, and the lock is confirmed free immediately after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2 lines
7 B
Plaintext
2 lines
7 B
Plaintext
1.14.2
|