luneski/mqtt-explorer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add observability for LLM topic context inclusion (#1038)

Browse Source 
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
Co-authored-by: Thomas Nordquist <thomasnordquist@users.noreply.github.com>
This commit is contained in:
Copilot
2026-01-30 20:53:29 +01:00
committed by GitHub
parent 080a773dbd
commit ed8a7f559e
194 changed files with 35234 additions and 4085 deletions
Download Patch File Download Diff File Expand all files Collapse all files

111
ENV_VARS_EXAMPLE.md
View File

@@ -22,13 +22,13 @@ export LLM_API_KEY=sk-proj-xxxxxxxxxxxxxxxxxxxx
```bash
# Configure token limit for neighboring topics context
export LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=100 # Default: 100 tokens
export LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=500 # Default: 500 tokens (increased for better device detection)
# Example: Increase token limit for more context
# Example: Increase token limit for large home automation setups
export LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=1000
# Example: Decrease token limit to reduce API costs (may reduce proposal quality)
export LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=200
# Example: Decrease token limit to reduce API costs
export LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=50
```
## Complete Example for Server Deployment
@@ -45,7 +45,7 @@ export MQTT_AUTO_CONNECT_PORT=1883
# LLM Configuration
export LLM_PROVIDER=gemini
export GEMINI_API_KEY=AIzaxxxxxxxxxxxxxxxxxxxx
export LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=100
export LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=500
# Start the server
node dist/src/server.js
@@ -63,7 +63,7 @@ RUN yarn install && yarn build:server
# Environment variables can be set at runtime
ENV LLM_PROVIDER=openai
ENV LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=100
ENV LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=500
EXPOSE 3000
CMD ["node", "dist/src/server.js"]
@@ -74,7 +74,7 @@ CMD ["node", "dist/src/server.js"]
docker run -d \
-e OPENAI_API_KEY=sk-proj-xxxxxxxxxxxxxxxxxxxx \
-e LLM_PROVIDER=openai \
-e LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=100 \
-e LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=500 \
-e MQTT_AUTO_CONNECT_HOST=mqtt.example.com \
-p 3000:3000 \
mqtt-explorer
@@ -82,63 +82,64 @@ docker run -d \
## Context Generation with Token Limits
The `LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT` controls how many tokens are allocated for neighboring topics in the context. Here's what happens:
The `LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT` controls how many tokens are allocated for neighboring topics in the context. The default has been increased from 100 to 500 tokens to provide better device relationship detection and enable multi-device automation proposals.
### With Default 100 Tokens
### With Default 500 Tokens (Recommended)
```
Topic Path: sensors/living_room/temperature
Value: 22.5
Topic Path: home/living_room/light
Value: {"state":"ON","brightness":75,"color_temp":350}
Status: Retained
Related Topics (5 shown):
humidity: 65
pressure: 1013.25
air_quality: {"pm25":12,"pm10":8,"co2":450,"voc":120}
motion: false
light_level: 450
Related Topics (18 shown):
home/living_room: {"scene":"evening"}
home/living_room/thermostat: {"temperature":22.5,"target":23,"mode":"heat"}
home/living_room/motion: true
home/living_room/humidity: 65
home/living_room/light/set: (command topic)
home/living_room/light/availability: "online"
home/living_room/light/config: {"transition":0.5,"fade":true}
home/living_room/blinds: {"position":75,"state":"open"}
home/living_room/blinds/set: (command topic)
home/living_room/tv: {"power":"ON","input":"HDMI1"}
home/kitchen/light: {"state":"ON","brightness":100}
home/bedroom/light: {"state":"OFF"}
home/living_room/light/brightness/set: (command topic)
home/living_room/light/color/set: (command topic)
home/living_room/motion_sensor/battery: 95
home/living_room/motion_sensor/last_motion: "2026-01-27T19:45:30Z"
Message Count: 1
Subtopics: 0
Message Count: 42
Subtopics: 5
```
### With 50 Tokens (Reduced)
**Benefits:** AI can see full room context (multiple devices), detect controllable devices (set topics), understand device hierarchies (parent/child relationships), and propose coordinated multi-device actions like "turn off all living room devices" or "set evening scene".
### With 200 Tokens (Reduced - May Miss Context)
```
Topic Path: sensors/living_room/temperature
Value: 22.5
Status: Retained
Related Topics (3 shown):
humidity: 65
pressure: 1013.25
air_quality: {"pm25":12,"pm10":8...
Message Count: 1
Subtopics: 0
```
### With 200 Tokens (Increased)
```
Topic Path: sensors/living_room/temperature
Value: 22.5
Topic Path: home/living_room/light
Value: {"state":"ON","brightness":75,"color_temp":350}
Status: Retained
Related Topics (8 shown):
humidity: 65
pressure: 1013.25
air_quality: {"pm25":12,"pm10":8,"co2":450,"voc":120}
motion: false
light_level: 450
battery: 85
signal_strength: -45
last_seen: 2026-01-26T23:45:00Z
home/living_room: {"scene":"evening"}
home/living_room/thermostat: {"temperature":22.5,"target":23}
home/living_room/motion: true
home/living_room/light/set: (command topic)
home/living_room/blinds: {"position":75}
home/kitchen/light: {"state":"ON","brightness":100}
Message Count: 1
Subtopics: 0
Message Count: 42
Subtopics: 5
```
**Limitations:** Less context = fewer multi-device proposals, may miss related devices in same room.
### With 1000 Tokens (Maximum - For Large Setups)
Use for extensive home automation with many rooms and devices. Provides comprehensive context including grandchildren, cousins, and extended device relationships.
## Priority Order
The AI Assistant checks configuration in this order:
@@ -182,12 +183,18 @@ node -e "console.log(process.env.OPENAI_API_KEY)"
### Token Limit Too Low
If you're seeing truncated context:
If you're seeing truncated context or poor multi-device proposals:
```bash
# Increase the token limit
export LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=200
# Increase the token limit (recommended: 500-1000 for home automation)
export LLM_NEIGHBORING_TOPICS_TOKEN_LIMIT=1000
```
**Note:** The default was increased from 100 to 500 tokens to better support:
- Multi-device detection and relationships
- Hierarchical topic structures (parent → children → grandchildren)
- Room-level automation proposals
- Complex home automation scenarios
### Want to Use UI Configuration
Simply don't set the environment variables - the UI configuration will be used instead.