[BUG] Tool Calls Not Applied Until End of Generation (Causes Timeout with Large Responses)

[billybasass]

Problem (one or two sentences)

When running Qwen3 Coder Next in LM Studio connected to Roo Code (VS Code extension), tool calls are not executed incrementally during generation. Instead, they appear to be applied only after the model finishes generating its entire response.

If the model produces a large response, this causes Roo Code to time out before the tool calls are applied.

This is not a long prompt processing issue — the problem occurs during response generation.

Context (who is affected and when)

Here’s a clean, professional GitHub issue you can copy and paste into the Roo Code repo:

---

Bug Report: Tool Calls Not Applied Until End of Generation (Causes Timeout with Large Responses)

Summary

When running Qwen3 Coder Next in LM Studio connected to Roo Code , tool calls are not executed incrementally during generation. Instead, they appear to be applied only after the model finishes generating its entire response.

If the model produces a large response, this causes Roo Code to time out before the tool calls are applied.

This is not a long prompt processing issue — the problem occurs during response generation.

---

Environment

• Model: Qwen3 Coder Next
• Backend: LM Studio
• Client: Roo Code (VS Code extension)
• Connection: LM Studio local server → Roo Code
• Hardware: Strix Halo 128gb
• OS: Windows

---

Important Clarification

This is not caused by:

• Slow prompt processing
• Initial inference delay
• Hardware limitations

The issue happens specifically while the model is generating a long response.

---

Additional Notes

It appears Roo Code may be:

• Buffering the entire response before parsing tool calls, or
• Waiting for a full completion event before executing tools

If tool calls were processed incrementally from the stream, this timeout issue would likely not occur.

---

Potential Area to Investigate

• Streaming tool call parsing
• OpenAI-compatible streaming implementation
• Function/tool call handling in streaming mode

---

If logs or additional diagnostics are needed, I can provide them.

Reproduction steps

Reproduction Steps

1. Run LM Studio with Qwen3 Coder Next.

2. Connect Roo Code to LM Studio.

3. Trigger a coding task that produces:

   • A long generation
   • Multiple tool calls

4. Observe that tool calls are not applied until the model finishes generating.

5. Large outputs result in timeout before tool execution.

Expected result

Expected Behavior * Tool calls should be detected and executed as soon as they are generated (streaming tool execution). * Roo Code should not wait for the full model completion before applying tool calls. * Large responses should not cause timeouts if tool calls are already available in the stream.

Actual result

Actual Behavior * Model begins generating a response normally. * Tool calls appear in the output stream. * Roo Code does not apply the tool calls immediately. * It waits until the model completes generation. * For large responses, this results in a timeout before tool execution occurs.

Variations tried (optional)

No response

App Version

3.47.3

API Provider (optional)

None

Model Used (optional)

Qwen3 Coder Next

Roo Code Task Links (optional)

No response

Relevant logs or errors (optional)

[roomote]

I've analyzed this issue and the relevant code paths. Here's what I found:

Architecture Analysis

The LM Studio provider (src/api/providers/lm-studio.ts) already supports incremental streaming of tool call arguments. During streaming:

1. Tool call argument chunks are received and emitted as tool_call_partial events
2. NativeToolCallParser converts these into tool_call_start / tool_call_delta / tool_call_end events
3. presentAssistantMessage tracks partial tool_use blocks in real-time and displays them in the UI

However, tool execution only begins when a tool call is finalized (marked as non-partial), which happens when:

• finish_reason: "tool_calls" is received from the server, OR
• The stream ends and finalizeRawChunks() cleans up remaining tool calls

This is inherent to the OpenAI-compatible streaming protocol -- the model must signal completion (finish_reason) before the client can know all tool call arguments are complete. So tool calls are displayed incrementally during streaming, but executed only after the model finishes generating. This is the expected behavior for all OpenAI-compatible providers.

The Timeout Issue

The default apiRequestTimeout is 600 seconds (10 minutes). This timeout is set on the OpenAI client instance and covers the entire request lifecycle, including the full streaming duration. For a large local model like Qwen3 Coder Next running on consumer hardware, generation can easily exceed this limit, causing the request to be aborted before the model finishes.

Proposed Implementation Plan

1. Disable the default timeout for LM Studio -- In src/api/providers/lm-studio.ts, change the OpenAI client construction to use no timeout by default (passing undefined) instead of getApiRequestTimeout(). Local models have unpredictable generation times and should not be constrained by a cloud-oriented timeout. Users can still configure apiRequestTimeout explicitly if needed.

2. Document the apiRequestTimeout setting -- Ensure it's clear in the extension settings that users of local providers (LM Studio, Ollama) may need to increase or disable (set to 0) this timeout for large/slow models.

Clarifying Questions

To confirm my analysis, could you provide:

• The specific error or message you see when the timeout occurs? (e.g., is it a "Request timed out" error, or does the UI just appear frozen?)
• Roughly how long it takes your model to finish generating before the timeout hits?
• Have you tried adjusting the roo-cline.apiRequestTimeout setting in VS Code settings?

Please mention @roomote in a comment saying that you approve this plan, and I'll proceed with the implementation.