From 7437c796a0d0e46b39fa01f118b64efa021d4adc Mon Sep 17 00:00:00 2001 From: Copilot <198982749+Copilot@users.noreply.github.com> Date: Tue, 23 Dec 2025 10:57:25 +0100 Subject: [PATCH] Add S3 object expiration and PR comment posting for demo videos (#970) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Demo videos from PR builds now expire after 90 days and are posted directly to PR threads. Uses object tagging with S3 lifecycle policies for automatic cleanup. ## Changes **Workflow** (`.github/workflows/tests.yml`) - Replace `hkusu/s3-upload-action@v2` with `ramonpaolo/action-upload-s3@main` for tagging support - Tag uploaded objects: `expiration=90days`, `Source=github-actions`, `Type=pr-demo-video` - Generate unique filenames: `pr-{number}-{timestamp}.gif` - Post PR comment with embedded video using `actions/github-script@v7` **Documentation** (`CI_CD.md`) - S3 lifecycle policy configuration (filters on `expiration=90days` tag) - IAM permission requirements: `s3:PutObject`, `s3:PutObjectTagging` ## S3 Lifecycle Setup Required ```json { "Rules": [{ "ID": "ExpirePRDemoVideosAfter90Days", "Status": "Enabled", "Filter": {"Tag": {"Key": "expiration", "Value": "90days"}}, "Expiration": {"Days": 90} }] } ``` Apply with: `aws s3api put-bucket-lifecycle-configuration --bucket --lifecycle-configuration file://policy.json` ## Notes - gh-pages `video.mp4` unaffected (served from GitHub Pages, not S3) - Existing S3 objects without tags remain unchanged > [!WARNING] > >
> Firewall rules blocked me from connecting to one or more addresses (expand for details) > > #### I tried to connect to the following addresses, but was blocked by firewall rules: > > - `https://api.github.com/repos/ramonpaolo/action-upload-s3/tags` > - Triggering command: `/usr/bin/curl curl -s REDACTED` (http block) > > If you need me to access, download, or install something from one of these locations, you can either: > > - Configure [Actions setup steps](https://gh.io/copilot/actions-setup-steps) to set up my environment, which run before the firewall is enabled > - Add the appropriate URLs or hosts to the custom allowlist in this repository's [Copilot coding agent settings](https://github.com/thomasnordquist/MQTT-Explorer/settings/copilot/coding_agent) (admins only) > >
Original prompt > when a demo-video is generated from a pr, add an expiration of 90 days to the S3 file and post the video as image to the pr thread.
--- ✨ Let Copilot coding agent [set things up for you](https://github.com/thomasnordquist/MQTT-Explorer/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com> --- .github/workflows/tests.yml | 54 ++++++++++++++++++++----- CI_CD.md | 80 +++++++++++++++++++++++++++++++++++-- 2 files changed, 120 insertions(+), 14 deletions(-) diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 24c017f..ff0b181 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -67,20 +67,52 @@ jobs: run: yarn ui-test - name: Post-processing run: ./scripts/prepareVideo.sh - - uses: hkusu/s3-upload-action@v2 - id: upload # specify some ID for use in subsequent steps + - name: Generate unique filename + id: filename + run: | + TIMESTAMP=$(date +%Y%m%d-%H%M%S) + FILENAME="pr-${{ github.event.pull_request.number }}-${TIMESTAMP}.gif" + echo "filename=${FILENAME}" >> $GITHUB_OUTPUT + - name: Upload to S3 with expiration tag + id: upload + uses: ramonpaolo/action-upload-s3@main with: - aws-access-key-id: ${{ vars.AWS_KEY_ID }} - aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} - aws-region: 'eu-central-1' - aws-bucket: ${{ vars.AWS_BUCKET }} - file-path: './ui-test.gif' - content-type: image/gif - output-file-url: 'true' + AWS_BUCKET_NAME: ${{ vars.AWS_BUCKET }} + AWS_REGION: 'eu-central-1' + AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + AWS_ACCESS_KEY_ID: ${{ vars.AWS_KEY_ID }} + local_path_upload: './ui-test.gif' + bucket_path_upload: '/' + name_to_save_on_s3: ${{ steps.filename.outputs.filename }} + tags: 'expiration=90days&Source=github-actions&Type=pr-demo-video' + - name: Generate file URL + id: fileurl + env: + AWS_BUCKET: ${{ vars.AWS_BUCKET }} + FILENAME: ${{ steps.filename.outputs.filename }} + run: | + FILE_URL="https://${AWS_BUCKET}.s3.eu-central-1.amazonaws.com/${FILENAME}" + echo "file-url=${FILE_URL}" >> $GITHUB_OUTPUT + echo "Uploaded to: ${FILE_URL}" - name: Show URL - run: echo '${{ steps.upload.outputs.file-url }}' + run: echo '${{ steps.fileurl.outputs.file-url }}' id: artifact-upload-step - - run: echo '' >> $GITHUB_STEP_SUMMARY + - run: echo '' \ + >> $GITHUB_STEP_SUMMARY + - name: Post video to PR + uses: actions/github-script@v7 + env: + VIDEO_URL: ${{ steps.fileurl.outputs.file-url }} + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const videoUrl = process.env.VIDEO_URL; + github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number, + body: `## 🎬 Demo Video Generated\n\n![Demo Video](${videoUrl})\n\n_This video will expire in 90 days._` + }); test-browser: runs-on: ubuntu-latest diff --git a/CI_CD.md b/CI_CD.md index ad170d2..59c65a8 100644 --- a/CI_CD.md +++ b/CI_CD.md @@ -70,10 +70,15 @@ Tests the traditional Electron desktop application: 2. Build the Electron application 3. Run unit tests (app + backend) 4. Run UI tests with video recording - 5. Upload test video to S3 - 6. Display test results in GitHub summary + 5. Upload test video to S3 with 90-day expiration tag + 6. Post demo video to PR as comment + 7. Display test results in GitHub summary -**Artifacts**: UI test video (GIF format) uploaded to S3 +**Artifacts**: +- UI test video (GIF format) uploaded to S3 +- Video is tagged with `expiration=90days` for automatic lifecycle deletion +- Video is posted to the PR thread as an embedded image +- Videos expire after 90 days via S3 lifecycle policy ##### 2. `test-browser` - Browser Mode Tests @@ -204,6 +209,75 @@ The repository includes a devcontainer configuration that automatically sets up: See [.devcontainer/README.md](.devcontainer/README.md) for details. +## S3 Configuration for Demo Videos + +### Required S3 Lifecycle Policy + +Demo videos uploaded from PRs are tagged with `expiration=90days` and require an S3 lifecycle policy to automatically delete them after 90 days. + +**Important**: The `video.mp4` file in the gh-pages branch is NOT tagged and will NOT expire. + +#### Setting up the Lifecycle Policy + +1. Create a file named `s3-lifecycle-pr-videos.json`: + +```json +{ + "Rules": [ + { + "ID": "ExpirePRDemoVideosAfter90Days", + "Status": "Enabled", + "Filter": { + "Tag": { + "Key": "expiration", + "Value": "90days" + } + }, + "Expiration": { + "Days": 90 + } + } + ] +} +``` + +2. Apply the policy to your S3 bucket: + +```bash +aws s3api put-bucket-lifecycle-configuration \ + --bucket YOUR_BUCKET_NAME \ + --lifecycle-configuration file://s3-lifecycle-pr-videos.json +``` + +3. Verify the policy: + +```bash +aws s3api get-bucket-lifecycle-configuration --bucket YOUR_BUCKET_NAME +``` + +#### How It Works + +- **PR demo videos**: Uploaded with filename pattern `pr-{number}-{timestamp}.gif` and tagged with: + - `expiration=90days` - Used by lifecycle policy for automatic deletion + - `Source=github-actions` - Identifies source of upload + - `Type=pr-demo-video` - Categorizes the object type +- **S3 lifecycle rule**: Automatically deletes objects tagged with `expiration=90days` after 90 days +- **Upload mechanism**: Uses `ramonpaolo/action-upload-s3@main` GitHub Action with object tagging support +- **gh-pages video**: `video.mp4` in gh-pages branch is served from GitHub Pages, not S3, so it persists indefinitely + +#### Required AWS Credentials + +The workflow requires the following secrets/variables: +- `vars.AWS_KEY_ID` - AWS access key ID (requires `s3:PutObject` and `s3:PutObjectTagging` permissions) +- `secrets.AWS_SECRET_ACCESS_KEY` - AWS secret access key +- `vars.AWS_BUCKET` - S3 bucket name +- AWS region: `eu-central-1` (hardcoded in workflow) + +The S3 bucket must have: +- Public read access enabled for uploaded objects +- Object tagging enabled +- Lifecycle policy configured as described above + ## Troubleshooting ### Browser Tests Failing