Add S3 object expiration and PR comment posting for demo videos (#970)

Demo videos from PR builds now expire after 90 days and are posted directly to PR threads. Uses object tagging with S3 lifecycle policies for automatic cleanup. ## Changes **Workflow** (`.github/workflows/tests.yml`) - Replace `hkusu/s3-upload-action@v2` with `ramonpaolo/action-upload-s3@main` for tagging support - Tag uploaded objects: `expiration=90days`, `Source=github-actions`, `Type=pr-demo-video` - Generate unique filenames: `pr-{number}-{timestamp}.gif` - Post PR comment with embedded video using `actions/github-script@v7` **Documentation** (`CI_CD.md`) - S3 lifecycle policy configuration (filters on `expiration=90days` tag) - IAM permission requirements: `s3:PutObject`, `s3:PutObjectTagging` ## S3 Lifecycle Setup Required ```json { "Rules": [{ "ID": "ExpirePRDemoVideosAfter90Days", "Status": "Enabled", "Filter": {"Tag": {"Key": "expiration", "Value": "90days"}}, "Expiration": {"Days": 90} }] } ``` Apply with: `aws s3api put-bucket-lifecycle-configuration --bucket <bucket> --lifecycle-configuration file://policy.json` ## Notes - gh-pages `video.mp4` unaffected (served from GitHub Pages, not S3) - Existing S3 objects without tags remain unchanged > [!WARNING] > > <details> > <summary>Firewall rules blocked me from connecting to one or more addresses (expand for details)</summary> > > #### I tried to connect to the following addresses, but was blocked by firewall rules: > > - `https://api.github.com/repos/ramonpaolo/action-upload-s3/tags` > - Triggering command: `/usr/bin/curl curl -s REDACTED` (http block) > > If you need me to access, download, or install something from one of these locations, you can either: > > - Configure [Actions setup steps](https://gh.io/copilot/actions-setup-steps) to set up my environment, which run before the firewall is enabled > - Add the appropriate URLs or hosts to the custom allowlist in this repository's [Copilot coding agent settings](https://github.com/thomasnordquist/MQTT-Explorer/settings/copilot/coding_agent) (admins only) > > </details>   <details> <summary>Original prompt</summary> > when a demo-video is generated from a pr, add an expiration of 90 days to the S3 file and post the video as image to the pr thread. </details>  --- ✨ Let Copilot coding agent [set things up for you](https://github.com/thomasnordquist/MQTT-Explorer/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
2025-12-23 10:57:25 +01:00
parent bb1e52feae
commit 7437c796a0
2 changed files with 120 additions and 14 deletions
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -67,20 +67,52 @@ jobs:
        run: yarn ui-test
      - name: Post-processing
        run: ./scripts/prepareVideo.sh
-      - uses: hkusu/s3-upload-action@v2
-        id: upload # specify some ID for use in subsequent steps
+      - name: Generate unique filename
+        id: filename
+        run: |
+          TIMESTAMP=$(date +%Y%m%d-%H%M%S)
+          FILENAME="pr-${{ github.event.pull_request.number }}-${TIMESTAMP}.gif"
+          echo "filename=${FILENAME}" >> $GITHUB_OUTPUT
+      - name: Upload to S3 with expiration tag
+        id: upload
+        uses: ramonpaolo/action-upload-s3@main
        with:
-          aws-access-key-id: ${{ vars.AWS_KEY_ID }}
-          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
-          aws-region: 'eu-central-1'
-          aws-bucket: ${{ vars.AWS_BUCKET }}
-          file-path: './ui-test.gif'
-          content-type: image/gif
-          output-file-url: 'true'
+          AWS_BUCKET_NAME: ${{ vars.AWS_BUCKET }}
+          AWS_REGION: 'eu-central-1'
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          AWS_ACCESS_KEY_ID: ${{ vars.AWS_KEY_ID }}
+          local_path_upload: './ui-test.gif'
+          bucket_path_upload: '/'
+          name_to_save_on_s3: ${{ steps.filename.outputs.filename }}
+          tags: 'expiration=90days&Source=github-actions&Type=pr-demo-video'
+      - name: Generate file URL
+        id: fileurl
+        env:
+          AWS_BUCKET: ${{ vars.AWS_BUCKET }}
+          FILENAME: ${{ steps.filename.outputs.filename }}
+        run: |
+          FILE_URL="https://${AWS_BUCKET}.s3.eu-central-1.amazonaws.com/${FILENAME}"
+          echo "file-url=${FILE_URL}" >> $GITHUB_OUTPUT
+          echo "Uploaded to: ${FILE_URL}"
      - name: Show URL
-        run: echo '${{ steps.upload.outputs.file-url }}'
+        run: echo '${{ steps.fileurl.outputs.file-url }}'
        id: artifact-upload-step
-      - run: echo '<picture><img src="${{ steps.upload.outputs.file-url }}"></picture>' >> $GITHUB_STEP_SUMMARY
+      - run: echo '<picture><img src="${{ steps.fileurl.outputs.file-url }}"></picture>' \
+          >> $GITHUB_STEP_SUMMARY
+      - name: Post video to PR
+        uses: actions/github-script@v7
+        env:
+          VIDEO_URL: ${{ steps.fileurl.outputs.file-url }}
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          script: |
+            const videoUrl = process.env.VIDEO_URL;
+            github.rest.issues.createComment({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: context.issue.number,
+              body: `## 🎬 Demo Video Generated\n\n![Demo Video](${videoUrl})\n\n_This video will expire in 90 days._`
+            });

  test-browser:
    runs-on: ubuntu-latest
--- a/CI_CD.md
+++ b/CI_CD.md
@@ -70,10 +70,15 @@ Tests the traditional Electron desktop application:
  2. Build the Electron application
  3. Run unit tests (app + backend)
  4. Run UI tests with video recording
-  5. Upload test video to S3
-  6. Display test results in GitHub summary
+  5. Upload test video to S3 with 90-day expiration tag
+  6. Post demo video to PR as comment
+  7. Display test results in GitHub summary

-**Artifacts**: UI test video (GIF format) uploaded to S3
+**Artifacts**: 
+- UI test video (GIF format) uploaded to S3
+- Video is tagged with `expiration=90days` for automatic lifecycle deletion
+- Video is posted to the PR thread as an embedded image
+- Videos expire after 90 days via S3 lifecycle policy

 ##### 2. `test-browser` - Browser Mode Tests

@@ -204,6 +209,75 @@ The repository includes a devcontainer configuration that automatically sets up:

 See [.devcontainer/README.md](.devcontainer/README.md) for details.

+## S3 Configuration for Demo Videos
+
+### Required S3 Lifecycle Policy
+
+Demo videos uploaded from PRs are tagged with `expiration=90days` and require an S3 lifecycle policy to automatically delete them after 90 days.
+
+**Important**: The `video.mp4` file in the gh-pages branch is NOT tagged and will NOT expire.
+
+#### Setting up the Lifecycle Policy
+
+1. Create a file named `s3-lifecycle-pr-videos.json`:
+
+```json
+{
+  "Rules": [
+    {
+      "ID": "ExpirePRDemoVideosAfter90Days",
+      "Status": "Enabled",
+      "Filter": {
+        "Tag": {
+          "Key": "expiration",
+          "Value": "90days"
+        }
+      },
+      "Expiration": {
+        "Days": 90
+      }
+    }
+  ]
+}
+```
+
+2. Apply the policy to your S3 bucket:
+
+```bash
+aws s3api put-bucket-lifecycle-configuration \
+  --bucket YOUR_BUCKET_NAME \
+  --lifecycle-configuration file://s3-lifecycle-pr-videos.json
+```
+
+3. Verify the policy:
+
+```bash
+aws s3api get-bucket-lifecycle-configuration --bucket YOUR_BUCKET_NAME
+```
+
+#### How It Works
+
+- **PR demo videos**: Uploaded with filename pattern `pr-{number}-{timestamp}.gif` and tagged with:
+  - `expiration=90days` - Used by lifecycle policy for automatic deletion
+  - `Source=github-actions` - Identifies source of upload
+  - `Type=pr-demo-video` - Categorizes the object type
+- **S3 lifecycle rule**: Automatically deletes objects tagged with `expiration=90days` after 90 days
+- **Upload mechanism**: Uses `ramonpaolo/action-upload-s3@main` GitHub Action with object tagging support
+- **gh-pages video**: `video.mp4` in gh-pages branch is served from GitHub Pages, not S3, so it persists indefinitely
+
+#### Required AWS Credentials
+
+The workflow requires the following secrets/variables:
+- `vars.AWS_KEY_ID` - AWS access key ID (requires `s3:PutObject` and `s3:PutObjectTagging` permissions)
+- `secrets.AWS_SECRET_ACCESS_KEY` - AWS secret access key
+- `vars.AWS_BUCKET` - S3 bucket name
+- AWS region: `eu-central-1` (hardcoded in workflow)
+
+The S3 bucket must have:
+- Public read access enabled for uploaded objects
+- Object tagging enabled
+- Lifecycle policy configured as described above
+
 ## Troubleshooting

 ### Browser Tests Failing