308b748d0ed5bb09700559d7e5e5b190d48a3918
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Copilot
|
2c147a92ad |
Add Docker build for browser mode with optimized 3-stage build, multi-platform support, comprehensive UI testing, one-click deployment, enterprise SSO integration, and biweekly CI pipeline (#934)
## Docker Build for Browser Solution This PR creates a Docker build for the browser solution (MQTT Explorer server mode). ### Completed: - [x] Create a production Dockerfile for the browser solution (`Dockerfile.browser`) - **NEW**: 3-stage build for maximum optimization - **NEW**: Clean production dependency installation with `yarn --production` - **NEW**: Only compiled dist/ folder copied (no source code) - Alpine Linux base with Node.js 24 - Non-root user (UID 1001) for security - Health check endpoint with proper error handling - Proper signal handling with dumb-init - Production dependencies automatically filtered by yarn - [x] Apply Docker best practices (multi-stage build, minimal image, non-root user, .dockerignore) - Created comprehensive .dockerignore - **FIXED**: Removed events from .dockerignore (needed for build) - **NEW**: Optimized for smaller layers with combined RUN commands - **NEW**: Removed development dependencies from final image - Used alpine base image - [x] Create GitHub Actions workflow for building, publishing, and testing the Docker image - Builds for linux/amd64, linux/arm64, linux/arm/v7 - **FIXED**: Added tsconfig.json and events/** to workflow trigger paths - **FIXED**: Attestation now uses correct digest from build step output - **NEW**: Mosquitto MQTT broker service for integration testing - **NEW**: MQTT broker configurable via MQTT_BROKER_HOST and MQTT_BROKER_PORT environment variables - **NEW**: Full UI test suite runs against containerized application - Tests container startup, health check, HTTP response, data persistence - **NEW**: Image size reporting in workflow summary - Tests verify application works with MQTT broker - Publishes to GitHub Container Registry (ghcr.io/thomasnordquist/mqtt-explorer) - Includes build attestation for supply chain security - [x] Configure workflow to run on push and every two weeks via cron schedule - Runs on 1st and 15th of each month at 2:00 AM UTC - Also runs on push to master/beta/release branches when relevant files change - Manual trigger via workflow_dispatch - [x] Add comprehensive test suite - Basic smoke tests: startup, health check, HTTP response, data persistence - **NEW**: Full UI test suite (`test:browser`) runs against Docker container - **NEW**: Tests connect to configurable MQTT broker (default localhost:1883) - Tests execute with Mosquitto MQTT broker available for backend integration - Same comprehensive tests validate both Electron and browser modes - [x] Test organization and naming - **NEW**: Renamed `test:ui` to `test:electron` for Electron-specific tests - **NEW**: Added `test:browser` script for browser mode tests (runs same UI test suite) - **NEW**: Kept `test:ui` as backward-compatible alias - **NEW**: Renamed `ui-tests` workflow job to `electron-tests` for clarity - [x] Update documentation with Docker usage instructions - Created DOCKER.md with comprehensive Docker documentation - Updated README.md with Docker quick start - **UPDATED**: CI_CD.md now lists all 10 test steps accurately - **NEW**: Added one-click deployment options section - **NEW**: Added authentication modes documentation - [x] **NEW**: One-click deployment solutions - **NEW**: Created docker-compose.yml for easy deployment - **NEW**: Added Play with Docker (PWD) badge for instant browser-based demo - **NEW**: Added DigitalOcean App Platform deployment badge - **NEW**: Added Koyeb deployment badge - **NEW**: Comprehensive deployment options documentation in DOCKER.md - **NEW**: "Try It Now" section in README.md and DOCKER.md with PWD badge - [x] **NEW**: Enterprise authentication integration - **NEW**: Added `MQTT_EXPLORER_SKIP_AUTH` environment variable - **NEW**: Allows disabling built-in authentication for proxy-based auth (OAuth2 Proxy, Authelia, enterprise SSO) - **NEW**: Socket.IO emits `auth-status` event on connection with authentication state - **NEW**: Frontend receives auth status via Socket.IO and skips login dialog when disabled - **NEW**: Logout button hidden when authentication is disabled - **NEW**: Created AuthContext for managing authentication state across components - **NEW**: Comprehensive security warnings in documentation about using skip auth only behind trusted authentication proxies - **NEW**: Updated docker-compose.yml with commented example for proxy authentication - [x] Code review and security scan passed - Fixed health check to handle connection errors properly - Corrected cron schedule comment - No security vulnerabilities found - Fixed image tag naming consistency - Simplified dependency management - **FIXED**: Workflow trigger paths now include all build-affecting files - **FIXED**: Attestation digest reference corrected - **FIXED**: events directory included in Docker build context - **FIXED**: Mosquitto service properly configured for integration testing - **FIXED**: MQTT broker connection now configurable for flexible testing environments - **IMPROVED**: Auth status now communicated via Socket.IO for better real-time synchronization - [x] Rename image to ghcr.io/thomasnordquist/mqtt-explorer (removed -browser suffix) - [x] Add multi-platform support for Raspberry Pi - linux/arm64 (Raspberry Pi 3/4/5) - linux/arm/v7 (Raspberry Pi 2/3) - [x] Upgrade to Node.js 24 (matching project requirements) - [x] **NEW**: Optimize Docker image for minimal size - Only production dependencies (no devDependencies) - No backend source code (only compiled JavaScript) - Removed build tools and dev dependencies - Combined layers for smaller image - **Image size reported in workflow summary** - [x] **NEW**: Fix webpack build configuration - **Enable minification** for production builds (was disabled) - **Update Material-UI references** from @material-ui to @mui - Fix vendor chunking to include @mui and @emotion packages - Reduces bundle size and fixes missing component issues ### Docker Image Features: - **Base**: Alpine Linux with Node.js 24 - **Size**: Reported automatically in workflow summary - **Platforms**: amd64, arm64, arm/v7 (Raspberry Pi support) - **Security**: Non-root user, minimal attack surface - **Reliability**: Health checks, graceful shutdown - **Persistence**: Data volume at `/app/data` - **Registry**: ghcr.io/thomasnordquist/mqtt-explorer - **Runtime deps**: Only production dependencies (automatically filtered) - **Frontend**: Minified webpack bundles with proper vendor splitting - **Testing**: Full UI test suite with configurable MQTT broker integration - **One-Click Deploy**: Play with Docker, DigitalOcean, Koyeb - **Enterprise Ready**: Optional authentication bypass for proxy-based SSO ### Available Tags: - `latest` - Latest stable from master - `master`, `beta`, `release` - Latest from each branch - `<branch>-<sha>` - Specific commits ### Authentication Options: 1. **Standard Mode** (default): Built-in username/password authentication - Set credentials via `MQTT_EXPLORER_USERNAME` and `MQTT_EXPLORER_PASSWORD` environment variables 2. **Skip Authentication Mode**: Set `MQTT_EXPLORER_SKIP_AUTH=true` for proxy-based auth - Use only behind trusted authentication proxies (OAuth2 Proxy, Authelia, enterprise SSO) - Socket.IO automatically informs frontend about auth status on connection - Frontend skips login dialog and logout button is hidden when authentication is disabled - ⚠️ **Security Warning**: Only use in environments with external authentication protection ### One-Click Deployment: Try MQTT Explorer instantly without installation: - **Play with Docker**: Free browser-based demo (click badge in README.md or DOCKER.md) - **DigitalOcean**: Deploy to managed platform starting at $5/month - **Koyeb**: Deploy to global edge network with free tier ### Security Summary: - CodeQL scan passed with no vulnerabilities - Docker image runs as non-root user (UID 1001) - Multi-stage build reduces attack surface - Health check includes proper error handling - Minimal runtime dependencies reduce vulnerability exposure - Full UI test suite validates application functionality - Build attestation with correct digest reference - MQTT broker integration tested with configurable connection via environment variables - Optional authentication bypass for enterprise SSO integration (with comprehensive security warnings) - Auth status communicated via Socket.IO for real-time synchronization <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>create a docker build for the browser solution</issue_title> > <issue_description>Create a docker build for amd64, that ships with a minimal image including nodes. Apply best practices and create a test workflow that builds it , publishes it and tests the built image. Build it every two weeks</issue_description> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> <!-- START COPILOT CODING AGENT SUFFIX --> - Fixes thomasnordquist/MQTT-Explorer#933 <!-- START COPILOT CODING AGENT TIPS --> --- 💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more [Copilot coding agent tips](https://gh.io/copilot-coding-agent-tips) in the docs. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com> Co-authored-by: Thomas Nordquist <thomasnordquist@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Copilot
|
26ed0aadd2 |
Upgrade to Node.js 24, update dependencies, migrate configs to ES modules, replace ts-node with tsx, upgrade React/Material-UI, and update GitHub Actions (#940)
## Update Dependencies, Node.js 24, and Migrate to ES Modules This PR updates the project dependencies, Node.js version, workflows, and migrates from CommonJS to ES modules. ### Checklist #### Phase 1: Assessment and Configuration - [x] Assess current project structure and dependencies - [x] Update Node.js version requirements (>=24 for builds, >=20 for runtime) - [x] Update GitHub workflow files to use Node 24 - [x] Update Dockerfile to use Node 24 #### Phase 2: TypeScript Configuration for ES Modules - [x] Update root tsconfig.json to use ES2020 target with CommonJS modules - [x] Update backend/tsconfig.json to use ES2020 target with CommonJS modules - [x] Update app/tsconfig.json to use ES2020 target with ESNext modules #### Phase 3: Update Dependencies - [x] Update root package.json dependencies to latest compatible versions - [x] Update app/package.json dependencies to latest compatible versions - [x] Update backend/package.json dependencies to latest compatible versions - [x] Run yarn install to update lockfile #### Phase 4: Convert CommonJS to ES Modules - [x] Convert webpack config files to ES modules (.js → .mjs) - [x] Convert prettier.config.js to ES modules - [x] Update TypeScript files with CommonJS require() to use ES imports - [x] Fix breaking changes from dependency API updates #### Phase 5: Replace ts-node with tsx - [x] Replace ts-node with tsx in all package.json scripts - [x] Update root package.json to use tsx for prepare-release and package scripts - [x] Update backend package.json to use tsx with mocha - [x] Update app package.json to use tsx with mocha - [x] Update script shebangs to use tsx - [x] Add tsx to devDependencies, remove ts-node #### Phase 6: Upgrade React and Material-UI - [x] Upgrade React from 16.14.0 to 18.3.1 - [x] Upgrade React-DOM from 16.14.0 to 18.3.1 - [x] Migrate from @material-ui (v4) to @mui/material (v5) - [x] Add @emotion/react and @emotion/styled (required for MUI v5) - [x] Update all import paths from @material-ui/* to @mui/* - [x] Update theme creation from createMuiTheme to createTheme - [x] Update palette.type to palette.mode - [x] Update ReactDOM.render to ReactDOM.createRoot (React 18) - [x] Update ThemeProvider import to use @mui/material/styles - [x] Add @mui/styles for withStyles compatibility - [x] Separate Theme and withStyles imports correctly - [x] Replace fade with alpha in theme styles - [x] Replace ExpansionPanel with Accordion - [x] Fix all component imports from wrong modules - [x] Replace withTheme HOC with useTheme hook - [x] Replace theme.palette.text.hint with theme.palette.text.secondary - [x] Update all Redux reducers for Redux v5 compatibility #### Phase 7: Fix All TypeScript Errors - [x] Fix Dialog disableBackdropClick removal (use onClose handler) - [x] Fix Button classes.label removal (use sx prop) - [x] Fix Select onChange signature (MUI v5 API change) - [x] Fix Snackbar onClose signature (MUI v5 API change) - [x] Fix ClickAwayListener onClickAway signature (MUI v5 API change) - [x] Fix ReactResizeDetector (migrate to useResizeDetector hook) - [x] Fix Redux connect + withStyles type compatibility (use type assertions) - [x] Fix all connected component prop type errors - [x] Add children prop to ErrorBoundary - [x] Add parameter types to callbacks #### Phase 8: Build and Test - [x] Run yarn build - ✅ **SUCCESSFUL with 0 errors, 1 minor warning** - [x] Run yarn test - ✅ **All 27 tests passing (5 app + 22 backend)** #### Phase 9: Update All GitHub Actions - [x] Update Node.js to 24 in copilot-setup.yml workflow - [x] Update Node.js to 24 in update-website.yml workflow - [x] Update docker/build-push-action from v5 to v6 - [x] Replace deprecated tibdex/github-app-token@v2 with actions/create-github-app-token@v1 - [x] All other actions already at latest versions (v4 for GitHub actions, v3 for Docker actions) #### Phase 10: Final Validation - [x] All TypeScript compilation errors fixed - [x] All tests passing - [x] Build completes successfully - [x] Clarified Node.js engine requirements per use case - [x] All GitHub Actions updated to latest versions ### Node.js Version Requirements This project has different Node.js requirements depending on the use case: #### Building the Electron App (Root package.json) - **Required:** Node.js >= 24 - **Why:** Build tools like @electron/notarize and semantic-release require Node.js 24+ - **Affected files:** `/package.json` #### Running the Backend/Server (Backend package.json) - **Required:** Node.js >= 20 - **Why:** The MQTT server runtime is compatible with Node.js 20+ - **Affected files:** `/backend/package.json` #### Frontend App (App package.json) - **Required:** Node.js >= 20 - **Why:** React and webpack tools are compatible with Node.js 20+ - **Affected files:** `/app/package.json` ### Summary of All Changes **Major Dependency Updates:** - TypeScript: 4.5.5 → 5.9.3 - Node.js: >=24 for builds, >=20 for runtime - React: 16.14.0 → 18.3.1 - React-DOM: 16.14.0 → 18.3.1 - Redux: 4.2.1 → 5.0.1 - @material-ui/core → @mui/material 5.18.0 - @material-ui/icons → @mui/icons-material 5.18.0 - mqtt: 4.3.6 → 5.14.1 - axios: 0.28.0 → 1.13.2 - redux-thunk: 2.3.0 → 3.1.0 - electron-builder: 24.13.3 → 26.0.12 - @electron/notarize: 3.1.1 (latest) - semantic-release: 25.0.2 (latest) - react-resize-detector: migrated to useResizeDetector hook - 50+ other dependencies **GitHub Actions Updated:** - ✅ actions/checkout@v4 (latest) - ✅ actions/setup-node@v4 (latest) - Now uses Node 24 in all workflows - ✅ actions/cache@v4 (latest) - ✅ actions/upload-artifact@v4 (latest) - ✅ docker/build-push-action: v5 → **v6** (latest) - ✅ docker/login-action@v3 (latest) - ✅ docker/setup-buildx-action@v3 (latest) - ✅ cycjimmy/semantic-release-action@v4 (latest) - ✅ tibdex/github-app-token@v2 → **actions/create-github-app-token@v1** (official replacement for deprecated action) - ✅ stefanzweifel/git-auto-commit-action@v5 (latest) - ✅ hkusu/s3-upload-action@v2 (latest) **All TypeScript Errors Fixed:** - Removed disableBackdropClick from Dialog (deprecated in MUI v5) - Removed Button classes.label (deprecated in MUI v5) - Updated all event handler signatures to match MUI v5 APIs - Migrated ReactResizeDetector component to useResizeDetector hook - Added type assertions for Redux connect + withStyles compatibility - Fixed all connected component prop passing - Added missing children props to components **Final Result:** ✅ **0 TypeScript errors** ✅ **27/27 tests passing** ✅ **Build successful** ✅ **Node 24 for builds, Node 20+ for runtime** ✅ **All GitHub Actions updated to latest versions** ⚠️ 1 minor warning (source map parsing in ace-builds dependency) <!-- START COPILOT ORIGINAL PROMPT --> <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>update dependencies</issue_title> > <issue_description>- update to nodejs 24 > - update npm dependencies > - update workflows > - change from commons to esmodules</issue_description> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details> <!-- START COPILOT CODING AGENT SUFFIX --> - Fixes thomasnordquist/MQTT-Explorer#939 <!-- START COPILOT CODING AGENT TIPS --> --- 💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey). --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com> Co-authored-by: Thomas Nordquist <thomasnordquist@users.noreply.github.com> |
||
|
Copilot
|
c55c3a8245 |
Fix UI tests: correct expandTopic parameter order and CI workflow (#936)
Fixes TypeScript compilation errors in UI tests and resolves CI workflow
configuration issue.
## Changes Made
### 1. Fixed expandTopic parameter order in ui-tests.spec.ts
- Corrected 5 function calls from `expandTopic(page, 'path')` to
`expandTopic('path', page)`
- Function signature: `expandTopic(path: string, browser: Page)`
- Aligns with existing usage in all scenario files (showNumericPlot.ts,
publishTopic.ts, etc.)
### 2. Fixed CI workflow configuration
- Updated `.github/workflows/tests.yml` to checkout PR code instead of
base branch
- Added `ref: ${{ github.event.pull_request.head.sha }}` to all 4
checkout actions
- The `pull_request_target` event defaults to checking out the base
branch; this fix ensures CI tests the PR's code
## Root Cause
The CI workflow was testing the base branch (master) which still had the
wrong parameter order, while the PR had the correct fix. This caused CI
to report TypeScript errors even though the PR code was correct.
## Testing
- ✅ TypeScript compilation passes locally (`tsc` and `yarn build`)
- ✅ Parameter order matches function signature and codebase conventions
- ✅ CI workflow now correctly tests PR code
- ✅ All 4 CI jobs (test, ui-tests, demo-video, test-browser) will use
corrected code
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
>
> ----
>
> *This section details on the original issue you should resolve*
>
> <issue_title>Fix tests</issue_title>
> <issue_description>- fix backend tests
> - fix UI tests</issue_description>
>
> ## Comments on the Issue (you are @copilot in this section)
>
> <comments>
> </comments>
>
</details>
<!-- START COPILOT CODING AGENT SUFFIX -->
- Fixes thomasnordquist/MQTT-Explorer#935
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/thomasnordquist/MQTT-Explorer/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
|
||
|
Copilot
|
92aa2c9fa8 | Fix UI test timeouts, TypeScript compilation, dependency compatibility, and backend tests with isolated test suite using per-test mocking (#930) | ||
|
Copilot
|
91df6de4d4 | Add browser support with Socket.io transport, authentication, performance-optimized IPC, and CI/CD (#925) | ||
|
Copilot
|
8285627c5f | Implement comprehensive UI test suite with meaningful assertions and best practices (#921) | ||
|
Björn Dalfors
|
efc9fb9736 | dont use pull_request_target as it opens the repo for pwnage.. | ||
|
Björn Dalfors
|
3229ef5643 |
Chore/fix workflow sha (#807)
* checkout merge commit of PR, not base branch head https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request_target * use link to test result, gif exceeds github allowed content length |
||
|
Thomas Nordquist
|
ee783f15c0 | chore: move website update to separate release action |