webui: Agentic Loop + MCP Client with support for Tools, Resources and Prompts (#18655)

This commit is contained in:
Aleksander Grygier
2026-03-06 10:00:39 +01:00
committed by GitHub
parent 2850bc6a13
commit f6235a41ef
147 changed files with 15285 additions and 366 deletions
+59 -5
View File
@@ -2,8 +2,10 @@
sequenceDiagram
participant UI as 🧩 ChatForm / ChatMessage
participant chatStore as 🗄️ chatStore
participant agenticStore as 🗄️ agenticStore
participant convStore as 🗄️ conversationsStore
participant settingsStore as 🗄️ settingsStore
participant mcpStore as 🗄️ mcpStore
participant ChatSvc as ⚙️ ChatService
participant DbSvc as ⚙️ DatabaseService
participant API as 🌐 /v1/chat/completions
@@ -25,6 +27,9 @@ sequenceDiagram
Note over convStore: → see conversations-flow.mmd
end
chatStore->>mcpStore: consumeResourceAttachmentsAsExtras()
Note right of mcpStore: Converts pending MCP resource<br/>attachments into message extras
chatStore->>chatStore: addMessage("user", content, extras)
chatStore->>DbSvc: createMessageBranch(userMsg, parentId)
chatStore->>convStore: addMessageToActive(userMsg)
@@ -38,7 +43,7 @@ sequenceDiagram
deactivate chatStore
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,API: 🌊 STREAMING
Note over UI,API: 🌊 STREAMING (with agentic flow detection)
%% ═══════════════════════════════════════════════════════════════════════════
activate chatStore
@@ -52,10 +57,17 @@ sequenceDiagram
chatStore->>chatStore: getApiOptions()
Note right of chatStore: Merge from settingsStore.config:<br/>temperature, max_tokens, top_p, etc.
chatStore->>ChatSvc: sendMessage(messages, options, signal)
alt agenticConfig.enabled && mcpStore has connected servers
chatStore->>agenticStore: runAgenticFlow(convId, messages, assistantMsg, options, signal)
Note over agenticStore: Multi-turn agentic loop:<br/>1. Call ChatService.sendMessage()<br/>2. If response has tool_calls → execute via mcpStore<br/>3. Append tool results as messages<br/>4. Loop until no more tool_calls or maxTurns<br/>→ see agentic flow details below
agenticStore-->>chatStore: final response with timings
else standard (non-agentic) flow
chatStore->>ChatSvc: sendMessage(messages, options, signal)
end
activate ChatSvc
ChatSvc->>ChatSvc: convertMessageToChatData(messages)
ChatSvc->>ChatSvc: convertDbMessageToApiChatMessageData(messages)
Note right of ChatSvc: DatabaseMessage[] → ApiChatMessageData[]<br/>Process attachments (images, PDFs, audio)
ChatSvc->>API: POST /v1/chat/completions
@@ -63,7 +75,7 @@ sequenceDiagram
loop SSE chunks
API-->>ChatSvc: data: {"choices":[{"delta":{...}}]}
ChatSvc->>ChatSvc: parseSSEChunk(line)
ChatSvc->>ChatSvc: handleStreamResponse(response)
alt content chunk
ChatSvc-->>chatStore: onChunk(content)
@@ -154,12 +166,15 @@ sequenceDiagram
Note over UI,API: ✏️ EDIT USER MESSAGE
%% ═══════════════════════════════════════════════════════════════════════════
UI->>chatStore: editUserMessagePreserveResponses(msgId, newContent)
UI->>chatStore: editMessageWithBranching(msgId, newContent, extras)
activate chatStore
chatStore->>chatStore: Get parent of target message
chatStore->>DbSvc: createMessageBranch(editedMsg, parentId)
chatStore->>convStore: refreshActiveMessages()
Note right of chatStore: Creates new branch, original preserved
chatStore->>chatStore: createAssistantMessage(editedMsg.id)
chatStore->>chatStore: streamChatCompletion(...)
Note right of chatStore: Automatically regenerates response
deactivate chatStore
%% ═══════════════════════════════════════════════════════════════════════════
@@ -171,4 +186,43 @@ sequenceDiagram
Note right of chatStore: errorDialogState = {type: 'timeout'|'server', message}
chatStore->>convStore: removeMessageAtIndex(failedMsgIdx)
chatStore->>DbSvc: deleteMessage(failedMsgId)
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,API: 🤖 AGENTIC LOOP (when agenticConfig.enabled)
%% ═══════════════════════════════════════════════════════════════════════════
Note over agenticStore: agenticStore.runAgenticFlow(convId, messages, assistantMsg, options, signal)
activate agenticStore
agenticStore->>agenticStore: getSession(convId) or create new
agenticStore->>agenticStore: updateSession(turn: 0, running: true)
loop executeAgenticLoop (until no tool_calls or maxTurns)
agenticStore->>agenticStore: turn++
agenticStore->>ChatSvc: sendMessage(messages, options, signal)
ChatSvc->>API: POST /v1/chat/completions
API-->>ChatSvc: response with potential tool_calls
ChatSvc-->>agenticStore: onComplete(content, reasoning, timings, toolCalls)
alt response has tool_calls
agenticStore->>agenticStore: normalizeToolCalls(toolCalls)
loop for each tool_call
agenticStore->>agenticStore: updateSession(streamingToolCall)
agenticStore->>mcpStore: executeTool(mcpCall, signal)
mcpStore-->>agenticStore: tool result
agenticStore->>agenticStore: extractBase64Attachments(result)
agenticStore->>agenticStore: emitToolCallResult(convId, ...)
agenticStore->>convStore: addMessageToActive(toolResultMsg)
agenticStore->>DbSvc: createMessageBranch(toolResultMsg)
end
agenticStore->>agenticStore: Create new assistantMsg for next turn
Note right of agenticStore: Continue loop with updated messages
else no tool_calls (final response)
agenticStore->>agenticStore: buildFinalTimings(allTurns)
Note right of agenticStore: Break loop, return final response
end
end
agenticStore->>agenticStore: updateSession(running: false)
agenticStore-->>chatStore: final content, timings, model
deactivate agenticStore
```
@@ -6,7 +6,7 @@ sequenceDiagram
participant DbSvc as ⚙️ DatabaseService
participant IDB as 💾 IndexedDB
Note over convStore: State:<br/>conversations: DatabaseConversation[]<br/>activeConversation: DatabaseConversation | null<br/>activeMessages: DatabaseMessage[]<br/>isInitialized: boolean<br/>usedModalities: $derived({vision, audio})
Note over convStore: State:<br/>conversations: DatabaseConversation[]<br/>activeConversation: DatabaseConversation | null<br/>activeMessages: DatabaseMessage[]<br/>isInitialized: boolean<br/>pendingMcpServerOverrides: Map&lt;string, McpServerOverride&gt;
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,IDB: 🚀 INITIALIZATION
@@ -37,6 +37,13 @@ sequenceDiagram
convStore->>convStore: conversations.unshift(conversation)
convStore->>convStore: activeConversation = $state(conversation)
convStore->>convStore: activeMessages = $state([])
alt pendingMcpServerOverrides has entries
loop each pending override
convStore->>DbSvc: Store MCP server override for new conversation
end
convStore->>convStore: clearPendingMcpServerOverrides()
end
deactivate convStore
%% ═══════════════════════════════════════════════════════════════════════════
@@ -58,8 +65,7 @@ sequenceDiagram
Note right of convStore: Filter to show only current branch path
convStore->>convStore: activeMessages = $state(filtered)
convStore->>chatStore: syncLoadingStateForChat(convId)
Note right of chatStore: Sync isLoading/currentResponse if streaming
Note right of convStore: Route (+page.svelte) then calls:<br/>chatStore.syncLoadingStateForChat(convId)
deactivate convStore
%% ═══════════════════════════════════════════════════════════════════════════
@@ -121,16 +127,36 @@ sequenceDiagram
end
deactivate convStore
UI->>convStore: deleteAll()
activate convStore
convStore->>DbSvc: Delete all conversations and messages
convStore->>convStore: conversations = []
convStore->>convStore: clearActiveConversation()
deactivate convStore
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,IDB: 📊 MODALITY TRACKING
Note over UI,IDB: MCP SERVER PER-CHAT OVERRIDES
%% ═══════════════════════════════════════════════════════════════════════════
Note over convStore: usedModalities = $derived.by(() => {<br/> calculateModalitiesFromMessages(activeMessages)<br/>})
Note over convStore: Conversations can override which MCP servers are enabled.
Note over convStore: Uses pendingMcpServerOverrides before conversation<br/>is created, then persists to conversation metadata.
Note over convStore: Scans activeMessages for attachments:<br/>- IMAGE → vision: true<br/>- PDF (processedAsImages) → vision: true<br/>- AUDIO → audio: true
UI->>convStore: setMcpServerOverride(convId, serverName, override)
Note right of convStore: override = {enabled: boolean}
UI->>convStore: getModalitiesUpToMessage(msgId)
Note right of convStore: Used for regeneration validation<br/>Only checks messages BEFORE target
UI->>convStore: toggleMcpServerForChat(convId, serverName, enabled)
activate convStore
convStore->>convStore: setMcpServerOverride(convId, serverName, {enabled})
deactivate convStore
UI->>convStore: isMcpServerEnabledForChat(convId, serverName)
Note right of convStore: Check override → fall back to global MCP config
UI->>convStore: getAllMcpServerOverrides(convId)
Note right of convStore: Returns all overrides for a conversation
UI->>convStore: removeMcpServerOverride(convId, serverName)
UI->>convStore: getMcpServerOverride(convId, serverName)
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,IDB: 📤 EXPORT / 📥 IMPORT
@@ -148,8 +174,10 @@ sequenceDiagram
UI->>convStore: importConversations(file)
activate convStore
convStore->>convStore: Parse JSON file
convStore->>convStore: importConversationsData(parsed)
convStore->>DbSvc: importConversations(parsed)
DbSvc->>IDB: Bulk INSERT conversations + messages
Note right of DbSvc: Skips duplicate conversations<br/>(checks existing by ID)
DbSvc->>IDB: INSERT conversations + messages (skip existing)
convStore->>convStore: loadConversations()
deactivate convStore
```
+25 -6
View File
@@ -66,6 +66,14 @@ sequenceDiagram
DbSvc-->>Store: rootMessageId
deactivate DbSvc
Store->>DbSvc: createSystemMessage(convId, content, parentId)
activate DbSvc
DbSvc->>DbSvc: Create message {role: "system", parent: parentId}
DbSvc->>Dexie: db.messages.add(systemMsg)
Dexie->>IDB: INSERT
DbSvc-->>Store: DatabaseMessage
deactivate DbSvc
Store->>DbSvc: createMessageBranch(message, parentId)
activate DbSvc
DbSvc->>DbSvc: Generate UUID for new message
@@ -116,6 +124,13 @@ sequenceDiagram
end
DbSvc->>Dexie: db.messages.delete(msgId)
Dexie->>IDB: DELETE target message
alt target message has a parent
DbSvc->>Dexie: db.messages.get(parentId)
DbSvc->>DbSvc: parent.children.filter(id !== msgId)
DbSvc->>Dexie: db.messages.update(parentId, {children})
Note right of DbSvc: Remove deleted message from parent's children[]
end
deactivate DbSvc
%% ═══════════════════════════════════════════════════════════════════════════
@@ -125,12 +140,16 @@ sequenceDiagram
Store->>DbSvc: importConversations(data)
activate DbSvc
loop each conversation in data
DbSvc->>DbSvc: Generate new UUIDs (avoid conflicts)
DbSvc->>Dexie: db.conversations.add(conversation)
Dexie->>IDB: INSERT conversation
loop each message
DbSvc->>Dexie: db.messages.add(message)
Dexie->>IDB: INSERT message
DbSvc->>Dexie: db.conversations.get(conv.id)
alt conversation already exists
Note right of DbSvc: Skip duplicate (keep existing)
else conversation is new
DbSvc->>Dexie: db.conversations.add(conversation)
Dexie->>IDB: INSERT conversation
loop each message
DbSvc->>Dexie: db.messages.add(message)
Dexie->>IDB: INSERT message
end
end
end
deactivate DbSvc
+226
View File
@@ -0,0 +1,226 @@
```mermaid
sequenceDiagram
participant UI as 🧩 McpServersSettings / ChatForm
participant chatStore as 🗄️ chatStore
participant mcpStore as 🗄️ mcpStore
participant mcpResStore as 🗄️ mcpResourceStore
participant convStore as 🗄️ conversationsStore
participant MCPSvc as ⚙️ MCPService
participant LS as 💾 LocalStorage
participant ExtMCP as 🔌 External MCP Server
Note over mcpStore: State:<br/>isInitializing, error<br/>toolCount, connectedServers<br/>healthChecks (Map)<br/>connections (Map)<br/>toolsIndex (Map)<br/>serverConfigs (Map)
Note over mcpResStore: State:<br/>serverResources (Map)<br/>cachedResources (Map)<br/>subscriptions (Map)<br/>attachments[]
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,ExtMCP: 🚀 INITIALIZATION (App Startup)
%% ═══════════════════════════════════════════════════════════════════════════
UI->>mcpStore: ensureInitialized()
activate mcpStore
mcpStore->>LS: get(MCP_SERVERS_LOCALSTORAGE_KEY)
LS-->>mcpStore: MCPServerSettingsEntry[]
mcpStore->>mcpStore: parseServerSettings(servers)
Note right of mcpStore: Filter enabled servers<br/>Build MCPServerConfig objects<br/>Per-chat overrides checked via convStore
loop For each enabled server
mcpStore->>mcpStore: runHealthCheck(serverId)
mcpStore->>mcpStore: updateHealthCheck(id, CONNECTING)
mcpStore->>MCPSvc: connect(serverName, config, clientInfo, capabilities, onPhase)
activate MCPSvc
MCPSvc->>MCPSvc: createTransport(config)
Note right of MCPSvc: WebSocket / StreamableHTTP / SSE<br/>with optional CORS proxy
MCPSvc->>ExtMCP: Transport handshake
ExtMCP-->>MCPSvc: Connection established
MCPSvc->>ExtMCP: Initialize request
Note right of ExtMCP: Exchange capabilities<br/>Server info, protocol version
ExtMCP-->>MCPSvc: InitializeResult (serverInfo, capabilities)
MCPSvc->>ExtMCP: listTools()
ExtMCP-->>MCPSvc: Tool[]
MCPSvc-->>mcpStore: MCPConnection
deactivate MCPSvc
mcpStore->>mcpStore: connections.set(serverName, connection)
mcpStore->>mcpStore: indexTools(connection.tools, serverName)
Note right of mcpStore: toolsIndex.set(toolName, serverName)<br/>Handle name conflicts with prefixes
mcpStore->>mcpStore: updateHealthCheck(id, SUCCESS)
mcpStore->>mcpStore: _connectedServers.push(serverName)
alt Server supports resources
mcpStore->>MCPSvc: listAllResources(connection)
MCPSvc->>ExtMCP: listResources()
ExtMCP-->>MCPSvc: MCPResource[]
MCPSvc-->>mcpStore: resources
mcpStore->>MCPSvc: listAllResourceTemplates(connection)
MCPSvc->>ExtMCP: listResourceTemplates()
ExtMCP-->>MCPSvc: MCPResourceTemplate[]
MCPSvc-->>mcpStore: templates
mcpStore->>mcpResStore: setServerResources(serverName, resources, templates)
end
end
mcpStore->>mcpStore: _isInitializing = false
deactivate mcpStore
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,ExtMCP: 🔧 TOOL EXECUTION (Chat with Tools)
%% ═══════════════════════════════════════════════════════════════════════════
UI->>mcpStore: executeTool(mcpCall: MCPToolCall, signal?)
activate mcpStore
mcpStore->>mcpStore: toolsIndex.get(mcpCall.function.name)
Note right of mcpStore: Resolve serverName from toolsIndex<br/>MCPToolCall = {id, type, function: {name, arguments}}
mcpStore->>mcpStore: acquireConnection()
Note right of mcpStore: activeFlowCount++<br/>Prevent shutdown during execution
mcpStore->>mcpStore: connection = connections.get(serverName)
mcpStore->>MCPSvc: callTool(connection, {name, arguments}, signal)
activate MCPSvc
MCPSvc->>MCPSvc: throwIfAborted(signal)
MCPSvc->>ExtMCP: callTool(name, arguments)
alt Tool execution success
ExtMCP-->>MCPSvc: ToolCallResult (content, isError)
MCPSvc->>MCPSvc: formatToolResult(result)
Note right of MCPSvc: Handle text, image (base64),<br/>embedded resource content
MCPSvc-->>mcpStore: ToolExecutionResult
else Tool execution error
ExtMCP-->>MCPSvc: Error
MCPSvc-->>mcpStore: throw Error
else Aborted
MCPSvc-->>mcpStore: throw AbortError
end
deactivate MCPSvc
mcpStore->>mcpStore: releaseConnection()
Note right of mcpStore: activeFlowCount--
mcpStore-->>UI: ToolExecutionResult
deactivate mcpStore
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,ExtMCP: RESOURCE ATTACHMENT CONSUMPTION
%% ═══════════════════════════════════════════════════════════════════════════
chatStore->>mcpStore: consumeResourceAttachmentsAsExtras()
activate mcpStore
mcpStore->>mcpResStore: getAttachments()
mcpResStore-->>mcpStore: MCPResourceAttachment[]
mcpStore->>mcpStore: Convert attachments to message extras
mcpStore->>mcpResStore: clearAttachments()
mcpStore-->>chatStore: MessageExtra[] (for user message)
deactivate mcpStore
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,ExtMCP: 📝 PROMPT OPERATIONS
%% ═══════════════════════════════════════════════════════════════════════════
UI->>mcpStore: getAllPrompts()
activate mcpStore
loop For each connected server with prompts capability
mcpStore->>MCPSvc: listPrompts(connection)
MCPSvc->>ExtMCP: listPrompts()
ExtMCP-->>MCPSvc: Prompt[]
MCPSvc-->>mcpStore: prompts
end
mcpStore-->>UI: MCPPromptInfo[] (with serverName)
deactivate mcpStore
UI->>mcpStore: getPrompt(serverName, promptName, args?)
activate mcpStore
mcpStore->>MCPSvc: getPrompt(connection, name, args)
MCPSvc->>ExtMCP: getPrompt({name, arguments})
ExtMCP-->>MCPSvc: GetPromptResult (messages)
MCPSvc-->>mcpStore: GetPromptResult
mcpStore-->>UI: GetPromptResult
deactivate mcpStore
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,ExtMCP: 📁 RESOURCE OPERATIONS
%% ═══════════════════════════════════════════════════════════════════════════
UI->>mcpResStore: addAttachment(resourceInfo)
activate mcpResStore
mcpResStore->>mcpResStore: Create MCPResourceAttachment (loading: true)
mcpResStore-->>UI: attachment
UI->>mcpStore: readResource(serverName, uri)
activate mcpStore
mcpStore->>MCPSvc: readResource(connection, uri)
MCPSvc->>ExtMCP: readResource({uri})
ExtMCP-->>MCPSvc: MCPReadResourceResult (contents)
MCPSvc-->>mcpStore: contents
mcpStore-->>UI: MCPResourceContent[]
deactivate mcpStore
UI->>mcpResStore: updateAttachmentContent(attachmentId, content)
mcpResStore->>mcpResStore: cacheResourceContent(resource, content)
deactivate mcpResStore
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,ExtMCP: 🔄 AUTO-RECONNECTION
%% ═══════════════════════════════════════════════════════════════════════════
Note over mcpStore: On WebSocket close or connection error:
mcpStore->>mcpStore: autoReconnect(serverName, attempt)
activate mcpStore
mcpStore->>mcpStore: Calculate backoff delay
Note right of mcpStore: delay = min(30s, 1s * 2^attempt)
mcpStore->>mcpStore: Wait for delay
mcpStore->>mcpStore: reconnectServer(serverName)
alt Reconnection success
mcpStore->>mcpStore: updateHealthCheck(id, SUCCESS)
else Max attempts reached
mcpStore->>mcpStore: updateHealthCheck(id, ERROR)
end
deactivate mcpStore
%% ═══════════════════════════════════════════════════════════════════════════
Note over UI,ExtMCP: 🛑 SHUTDOWN
%% ═══════════════════════════════════════════════════════════════════════════
UI->>mcpStore: shutdown()
activate mcpStore
mcpStore->>mcpStore: Wait for activeFlowCount == 0
loop For each connection
mcpStore->>MCPSvc: disconnect(connection)
MCPSvc->>MCPSvc: transport.onclose = undefined
MCPSvc->>ExtMCP: close()
end
mcpStore->>mcpStore: connections.clear()
mcpStore->>mcpStore: toolsIndex.clear()
mcpStore->>mcpStore: _connectedServers = []
mcpStore->>mcpResStore: clear()
deactivate mcpStore
```