Add S3 object expiration and PR comment posting for demo videos (#970)
Demo videos from PR builds now expire after 90 days and are posted
directly to PR threads. Uses object tagging with S3 lifecycle policies
for automatic cleanup.
## Changes
**Workflow** (`.github/workflows/tests.yml`)
- Replace `hkusu/s3-upload-action@v2` with
`ramonpaolo/action-upload-s3@main` for tagging support
- Tag uploaded objects: `expiration=90days`, `Source=github-actions`,
`Type=pr-demo-video`
- Generate unique filenames: `pr-{number}-{timestamp}.gif`
- Post PR comment with embedded video using `actions/github-script@v7`
**Documentation** (`CI_CD.md`)
- S3 lifecycle policy configuration (filters on `expiration=90days` tag)
- IAM permission requirements: `s3:PutObject`, `s3:PutObjectTagging`
## S3 Lifecycle Setup Required
```json
{
"Rules": [{
"ID": "ExpirePRDemoVideosAfter90Days",
"Status": "Enabled",
"Filter": {"Tag": {"Key": "expiration", "Value": "90days"}},
"Expiration": {"Days": 90}
}]
}
```
Apply with: `aws s3api put-bucket-lifecycle-configuration --bucket
<bucket> --lifecycle-configuration file://policy.json`
## Notes
- gh-pages `video.mp4` unaffected (served from GitHub Pages, not S3)
- Existing S3 objects without tags remain unchanged
> [!WARNING]
>
> <details>
> <summary>Firewall rules blocked me from connecting to one or more
addresses (expand for details)</summary>
>
> #### I tried to connect to the following addresses, but was blocked by
firewall rules:
>
> - `https://api.github.com/repos/ramonpaolo/action-upload-s3/tags`
> - Triggering command: `/usr/bin/curl curl -s REDACTED` (http block)
>
> If you need me to access, download, or install something from one of
these locations, you can either:
>
> - Configure [Actions setup
steps](https://gh.io/copilot/actions-setup-steps) to set up my
environment, which run before the firewall is enabled
> - Add the appropriate URLs or hosts to the custom allowlist in this
repository's [Copilot coding agent
settings](https://github.com/thomasnordquist/MQTT-Explorer/settings/copilot/coding_agent)
(admins only)
>
> </details>
<!-- START COPILOT CODING AGENT SUFFIX -->
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
> when a demo-video is generated from a pr, add an expiration of 90 days
to the S3 file and post the video as image to the pr thread.
</details>
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/thomasnordquist/MQTT-Explorer/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
This commit is contained in:
54
.github/workflows/tests.yml
vendored
54
.github/workflows/tests.yml
vendored
@@ -67,20 +67,52 @@ jobs:
|
|||||||
run: yarn ui-test
|
run: yarn ui-test
|
||||||
- name: Post-processing
|
- name: Post-processing
|
||||||
run: ./scripts/prepareVideo.sh
|
run: ./scripts/prepareVideo.sh
|
||||||
- uses: hkusu/s3-upload-action@v2
|
- name: Generate unique filename
|
||||||
id: upload # specify some ID for use in subsequent steps
|
id: filename
|
||||||
|
run: |
|
||||||
|
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
|
||||||
|
FILENAME="pr-${{ github.event.pull_request.number }}-${TIMESTAMP}.gif"
|
||||||
|
echo "filename=${FILENAME}" >> $GITHUB_OUTPUT
|
||||||
|
- name: Upload to S3 with expiration tag
|
||||||
|
id: upload
|
||||||
|
uses: ramonpaolo/action-upload-s3@main
|
||||||
with:
|
with:
|
||||||
aws-access-key-id: ${{ vars.AWS_KEY_ID }}
|
AWS_BUCKET_NAME: ${{ vars.AWS_BUCKET }}
|
||||||
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
AWS_REGION: 'eu-central-1'
|
||||||
aws-region: 'eu-central-1'
|
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
||||||
aws-bucket: ${{ vars.AWS_BUCKET }}
|
AWS_ACCESS_KEY_ID: ${{ vars.AWS_KEY_ID }}
|
||||||
file-path: './ui-test.gif'
|
local_path_upload: './ui-test.gif'
|
||||||
content-type: image/gif
|
bucket_path_upload: '/'
|
||||||
output-file-url: 'true'
|
name_to_save_on_s3: ${{ steps.filename.outputs.filename }}
|
||||||
|
tags: 'expiration=90days&Source=github-actions&Type=pr-demo-video'
|
||||||
|
- name: Generate file URL
|
||||||
|
id: fileurl
|
||||||
|
env:
|
||||||
|
AWS_BUCKET: ${{ vars.AWS_BUCKET }}
|
||||||
|
FILENAME: ${{ steps.filename.outputs.filename }}
|
||||||
|
run: |
|
||||||
|
FILE_URL="https://${AWS_BUCKET}.s3.eu-central-1.amazonaws.com/${FILENAME}"
|
||||||
|
echo "file-url=${FILE_URL}" >> $GITHUB_OUTPUT
|
||||||
|
echo "Uploaded to: ${FILE_URL}"
|
||||||
- name: Show URL
|
- name: Show URL
|
||||||
run: echo '${{ steps.upload.outputs.file-url }}'
|
run: echo '${{ steps.fileurl.outputs.file-url }}'
|
||||||
id: artifact-upload-step
|
id: artifact-upload-step
|
||||||
- run: echo '<picture><img src="${{ steps.upload.outputs.file-url }}"></picture>' >> $GITHUB_STEP_SUMMARY
|
- run: echo '<picture><img src="${{ steps.fileurl.outputs.file-url }}"></picture>' \
|
||||||
|
>> $GITHUB_STEP_SUMMARY
|
||||||
|
- name: Post video to PR
|
||||||
|
uses: actions/github-script@v7
|
||||||
|
env:
|
||||||
|
VIDEO_URL: ${{ steps.fileurl.outputs.file-url }}
|
||||||
|
with:
|
||||||
|
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
script: |
|
||||||
|
const videoUrl = process.env.VIDEO_URL;
|
||||||
|
github.rest.issues.createComment({
|
||||||
|
owner: context.repo.owner,
|
||||||
|
repo: context.repo.repo,
|
||||||
|
issue_number: context.issue.number,
|
||||||
|
body: `## 🎬 Demo Video Generated\n\n\n\n_This video will expire in 90 days._`
|
||||||
|
});
|
||||||
|
|
||||||
test-browser:
|
test-browser:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
|
|||||||
80
CI_CD.md
80
CI_CD.md
@@ -70,10 +70,15 @@ Tests the traditional Electron desktop application:
|
|||||||
2. Build the Electron application
|
2. Build the Electron application
|
||||||
3. Run unit tests (app + backend)
|
3. Run unit tests (app + backend)
|
||||||
4. Run UI tests with video recording
|
4. Run UI tests with video recording
|
||||||
5. Upload test video to S3
|
5. Upload test video to S3 with 90-day expiration tag
|
||||||
6. Display test results in GitHub summary
|
6. Post demo video to PR as comment
|
||||||
|
7. Display test results in GitHub summary
|
||||||
|
|
||||||
**Artifacts**: UI test video (GIF format) uploaded to S3
|
**Artifacts**:
|
||||||
|
- UI test video (GIF format) uploaded to S3
|
||||||
|
- Video is tagged with `expiration=90days` for automatic lifecycle deletion
|
||||||
|
- Video is posted to the PR thread as an embedded image
|
||||||
|
- Videos expire after 90 days via S3 lifecycle policy
|
||||||
|
|
||||||
##### 2. `test-browser` - Browser Mode Tests
|
##### 2. `test-browser` - Browser Mode Tests
|
||||||
|
|
||||||
@@ -204,6 +209,75 @@ The repository includes a devcontainer configuration that automatically sets up:
|
|||||||
|
|
||||||
See [.devcontainer/README.md](.devcontainer/README.md) for details.
|
See [.devcontainer/README.md](.devcontainer/README.md) for details.
|
||||||
|
|
||||||
|
## S3 Configuration for Demo Videos
|
||||||
|
|
||||||
|
### Required S3 Lifecycle Policy
|
||||||
|
|
||||||
|
Demo videos uploaded from PRs are tagged with `expiration=90days` and require an S3 lifecycle policy to automatically delete them after 90 days.
|
||||||
|
|
||||||
|
**Important**: The `video.mp4` file in the gh-pages branch is NOT tagged and will NOT expire.
|
||||||
|
|
||||||
|
#### Setting up the Lifecycle Policy
|
||||||
|
|
||||||
|
1. Create a file named `s3-lifecycle-pr-videos.json`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"Rules": [
|
||||||
|
{
|
||||||
|
"ID": "ExpirePRDemoVideosAfter90Days",
|
||||||
|
"Status": "Enabled",
|
||||||
|
"Filter": {
|
||||||
|
"Tag": {
|
||||||
|
"Key": "expiration",
|
||||||
|
"Value": "90days"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"Expiration": {
|
||||||
|
"Days": 90
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Apply the policy to your S3 bucket:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
aws s3api put-bucket-lifecycle-configuration \
|
||||||
|
--bucket YOUR_BUCKET_NAME \
|
||||||
|
--lifecycle-configuration file://s3-lifecycle-pr-videos.json
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Verify the policy:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
aws s3api get-bucket-lifecycle-configuration --bucket YOUR_BUCKET_NAME
|
||||||
|
```
|
||||||
|
|
||||||
|
#### How It Works
|
||||||
|
|
||||||
|
- **PR demo videos**: Uploaded with filename pattern `pr-{number}-{timestamp}.gif` and tagged with:
|
||||||
|
- `expiration=90days` - Used by lifecycle policy for automatic deletion
|
||||||
|
- `Source=github-actions` - Identifies source of upload
|
||||||
|
- `Type=pr-demo-video` - Categorizes the object type
|
||||||
|
- **S3 lifecycle rule**: Automatically deletes objects tagged with `expiration=90days` after 90 days
|
||||||
|
- **Upload mechanism**: Uses `ramonpaolo/action-upload-s3@main` GitHub Action with object tagging support
|
||||||
|
- **gh-pages video**: `video.mp4` in gh-pages branch is served from GitHub Pages, not S3, so it persists indefinitely
|
||||||
|
|
||||||
|
#### Required AWS Credentials
|
||||||
|
|
||||||
|
The workflow requires the following secrets/variables:
|
||||||
|
- `vars.AWS_KEY_ID` - AWS access key ID (requires `s3:PutObject` and `s3:PutObjectTagging` permissions)
|
||||||
|
- `secrets.AWS_SECRET_ACCESS_KEY` - AWS secret access key
|
||||||
|
- `vars.AWS_BUCKET` - S3 bucket name
|
||||||
|
- AWS region: `eu-central-1` (hardcoded in workflow)
|
||||||
|
|
||||||
|
The S3 bucket must have:
|
||||||
|
- Public read access enabled for uploaded objects
|
||||||
|
- Object tagging enabled
|
||||||
|
- Lifecycle policy configured as described above
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
### Browser Tests Failing
|
### Browser Tests Failing
|
||||||
|
|||||||
Reference in New Issue
Block a user