Don't Block HTTP Requests Waiting for LLMs

The danger of synchronous LLM requests

I often see junior developers making direct HTTP requests to OpenAI or Anthropic endpoints inside synchronous web controllers. They then wonder why their application crashes when an API takes fifteen seconds to respond. Users will not wait that long. This slow response ruins the user experience. It also blocks system processes that should handle other traffic.

To solve this, your controller must immediately return a 202 Accepted response and offload the API call to a Laravel Queue worker. This pattern frees up your web worker in milliseconds. A background job then takes over the heavy lifting of calling the API. This isolates the work from the user's web request lifecycle.

public function store(Request $request)
{
    ProcessAiRequest::dispatch($request->user(), $request->input('prompt'));

    return response()->json(['status' => 'Processing task.'], 202);
}

Isolating network timeouts

Outgoing HTTP connections sometimes do not respect queue job timeouts. For this reason, you must specify connection and request timeouts directly on your HTTP client. You can also configure exception throttling middleware to handle the inevitable HTTP client exceptions gracefully.

Updating the client asynchronously

When the background job finishes its call to the external LLM, it saves the generated content directly to your database. At that point, you can broadcast a completed event using Laravel Echo or Pusher to update your frontend user interface in real time. If real-time broadcasting is too complex for your current setup, you can set up simple frontend polling against a database status endpoint. This keeps the application responsive. It also keeps your application completely decoupled from upstream network latency.