mqtt-explorer/.github/copilot-instructions.md

copilot-instructions.md - Agent Long-Term Memory

META-INSTRUCTIONS (IMMUTABLE)

1. Long-term memory: If you learn something during a session that would save time in future sessions, add it to .github/copilot-instructions.md
2. Paper knowledge must go: If something is no longer true, update or remove it immediately
3. Evaluate after every session: Consider whether the instructions need updates based on what you learned
4. Concise and useful: All information must be actionable, current, and concise

Test Commands

Unit tests:

• yarn test - All unit tests (app + backend)
• yarn test:app - Frontend tests only
• yarn test:backend - Backend tests only

Integration tests:

• yarn test:ui - Browser tests (requires yarn build first)
• yarn test:demo-video - UI recording (requires Xvfb, mosquitto, tmux, ffmpeg)
• yarn test:mcp - Model Context Protocol tests
• yarn test:all - All tests (unit + demo-video)
• ./scripts/runBrowserTests.sh - Browser mode UI tests (requires mosquitto service)
• ./scripts/uiTests.sh - Demo video tests with Electron (requires Xvfb, mosquitto)

CI jobs: test, ui-tests, demo-video, test-browser, browser-ui-tests

Important:

• Browser UI tests require MQTT broker. In CI, GitHub Actions health checks ensure the mosquitto service is ready before tests run.
• Demo video tests use the same test scenarios as browser tests - if browser tests pass, demo video tests should pass too (they use identical selectors in src/spec/scenarios/)

Debugging Demo Video / UI Tests

When demo video tests fail in CI but you need to debug locally:

Prerequisites:

# Install required packages
sudo apt-get install xvfb mosquitto tmux ffmpeg

# Ensure mosquitto is stopped (script will start its own instance)
sudo systemctl stop mosquitto

Steps to debug:

1. Build the project: yarn build
2. Run the demo video tests: ELECTRON_DISABLE_SANDBOX=1 ./scripts/uiTests.sh
3. If tests fail, check the error messages for:
   • Missing data-test or data-test-type attributes
   • Elements not visible (hidden, outside viewport, or covered by overlays)
   • Click interception (tooltips, dialogs blocking clicks)
   • XPath selector issues (check the data-test value format)

Common issues:

• "locator not visible": Element exists but is hidden or outside viewport
• "locator.click intercepted": Another element (tooltip, dialog) is covering the click target
• Empty data-test attribute: For simple numeric values, ensure you're using the topic name, not props.literal.path (which is empty for non-JSON values)
• "Process failed to launch": Electron can't start - ensure DISPLAY is set and Xvfb is running

Environment-specific notes:

• Demo video tests use Electron with Xvfb (virtual display)
• Browser tests use Chromium without Electron (easier to debug locally)
• CI environment has proper Electron setup - if local tests are flaky, trust CI results
• Both test types use the same scenario files in src/spec/scenarios/

Material-UI Tooltip considerations:

• Tooltips wrap their children and create overlay divs
• Test attributes (data-test-type, data-test) must be on the inner clickable element that's passed as a child to the Tooltip
• Mouse event handlers (onMouseEnter, onMouseLeave, ref) go on outer wrapper
• onClick handler and test attributes go on the span inside the Tooltip
• The clickable child inside the Tooltip is what Playwright should target

Example:

// ❌ WRONG - attributes on outer wrapper, Tooltip wraps and hides them
<Tooltip title="...">
  <span onClick={...} data-test-type="Button" data-test="example">
    <Icon />
  </span>
</Tooltip>

// ✅ CORRECT - attributes on the actual clickable child inside Tooltip
<span ref={ref} onMouseEnter={...} onMouseLeave={...}>
  <Tooltip title="...">
    <span onClick={...} data-test-type="Button" data-test="example">
      <Icon />
    </span>
  </Tooltip>
</span>

data-test value format:

• ShowChart: Use last segment of topic path (e.g., "heater" for "kitchen/coffee_maker/heater")
• ChartSettings: Use full topic + dotPath (e.g., "kitchen/coffee_maker/heater-" with trailing dash when dotPath is empty)
• Test XPaths use contains(@data-test, "substring") so partial matches work

Running Browser Tests Locally

Prerequisites:

# Install mosquitto (if not already installed)
sudo apt-get install mosquitto

# Start mosquitto service
sudo systemctl start mosquitto
sudo systemctl status mosquitto

Run browser tests:

# Build server and run tests
yarn build:server
./scripts/runBrowserTests.sh

The script automatically:

• Starts a local mosquitto broker
• Builds the TypeScript code
• Starts the browser mode server on port 3000
• Runs Playwright tests in browser mode
• Cleans up processes on exit

Test environment variables:

• MQTT_EXPLORER_USERNAME - Browser auth username (default: test)
• MQTT_EXPLORER_PASSWORD - Browser auth password (default: test123)
• PORT - Server port (default: 3000)
• TESTS_MQTT_BROKER_HOST - MQTT broker host (default: 127.0.0.1)
• TESTS_MQTT_BROKER_PORT - MQTT broker port (default: 1883)
• USE_MOBILE_VIEWPORT - Enable mobile viewport (default: false)

Common test failures after UI changes:

• Update test selectors in src/spec/ui-tests.spec.ts if UI structure changes
• Use data-testid attributes for stable test selectors
• Avoid using role + name selectors for dynamic content (use direct testid selectors instead)

Browser Mode

Prerequisites: Node.js ≥24, Yarn, Mosquitto broker (for testing)

Development (hot reload):

export MQTT_EXPLORER_USERNAME=admin MQTT_EXPLORER_PASSWORD=yourpass
yarn dev:server
# Backend: http://localhost:3000, Frontend: http://localhost:8080 (use this one)

Production:

yarn build:server
export MQTT_EXPLORER_USERNAME=admin MQTT_EXPLORER_PASSWORD=yourpass
yarn start:server  # http://localhost:3000

Build artifacts: dist/src/server.js, app/build/*.js, app/build/index.html

Debugging Browser Mode

DevTools checks:

• Console: JavaScript errors, CSP errors (security headers), WebSocket issues
• Network: Static asset loading, WebSocket status, API auth

Common issues:

• Blank page/CSP errors: Add 'unsafe-eval' to scriptSrc in src/server.ts helmet config
• Auth loop: WebSocket auth failing - check Network → WS → Messages and server logs
• Theme errors: Verify both ThemeProvider and LegacyThemeProvider in app/src/index.tsx

Expected warnings (non-fatal):

• React 18 + Material-UI v5 type warnings
• IpcRendererEventBus errors (no Electron IPC in browser mode)
• MUI locale, componentWillReceiveProps, ACE editor warnings

WebSocket debugging:

node dist/src/server.js 2>&1 | tee server.log
# Check DevTools → Network → WS → Messages for handshake
# CORS: check ALLOWED_ORIGINS env var

Security notes:

• unsafe-eval in CSP required for webpack (security tradeoff)
• Never hardcode credentials (use env vars)
• Production: use HTTPS with reverse proxy
• Rate limit: 5 auth attempts/15min/IP
• File upload limit: 16MB

Key files:

• src/server.ts - Express server, security middleware
• app/webpack.browser.config.mjs - Browser webpack config
• app/src/browserEventBus.ts - Socket.io client
• app/src/components/BrowserAuthWrapper.tsx - Auth dialog
• app/src/index.tsx - React entry, theme providers

Styling Conventions

When modifying or creating UI components, follow the styling patterns documented in STYLING.md.

Key points for AI agents:

• Use Material-UI (MUI) v7 components with withStyles HOC for styling
• Access theme colors via theme.palette.*, spacing via theme.spacing(), typography via theme.typography.*
• Support both light and dark modes with theme-conditional styling
• Import Material-UI colors: import { blueGrey, amber, green, red } from '@mui/material/colors'

Mobile Testing Workflow

Prerequisites for mobile testing:

# Install Playwright browsers
npx playwright install --with-deps chromium

# Configure mosquitto to allow anonymous connections (for local testing)
echo "listener 1883
allow_anonymous true" | sudo tee /etc/mosquitto/conf.d/allow-anonymous.conf
sudo systemctl restart mosquitto

Interactive testing with mobile viewport:

# Set up environment
export MQTT_EXPLORER_SKIP_AUTH=true
export MQTT_AUTO_CONNECT_HOST=127.0.0.1

# Build and start server
yarn build:server
node dist/src/server.js

# In another terminal, run Playwright test with mobile viewport
# Create test script with viewport: { width: 412, height: 914 }
# Always INSPECT the rendered output, don't rely on assumptions

Key lesson: Mobile tree visibility issues often stem from:

1. CSS flex/absolute positioning conflicts
2. Missing Redux state updates (connection not propagated to frontend)
3. MQTT broker authentication (mosquitto requires allow_anonymous true for testing)
4. Timing issues (frontend subscribing to events after backend emits them)

Server-side auto-connect (for testing):

export MQTT_AUTO_CONNECT_HOST=127.0.0.1
export MQTT_AUTO_CONNECT_PORT=1883  # optional
export MQTT_AUTO_CONNECT_PROTOCOL=mqtt  # optional