ci: group Dependabot example updates and verify only what changed #283

Workflow file for this run

.github/workflows/examples.yaml at 0316c21

	name: Verify Examples

	on:
	pull_request:
	branches:
	- main
	paths:
	- "examples/**"
	# This workflow's own machinery. The matrix file and the selector decide what
	# gets verified at all, so a pull request that changes only those would
	# otherwise not run here — and a malformed matrix or a selector regression
	# would land unverified and surface later on an unrelated examples pull
	# request. Including them makes the workflow verify its own changes.
	- ".github/workflows/examples.yaml"
	- ".github/scripts/select-examples.sh"
	- ".github/scripts/verify-http.sh"
	- ".github/scripts/check-example-config.sh"
	- ".github/example-matrix.json"
	- ".github/dependabot.yml"
	push:
	branches:
	- main
	# Every input select-examples.sh treats as shared, not just src/. layer/bootstrap in
	# particular is the code path all eight zip examples boot — build-layer copies it
	# into the artifact each of them injects — so leaving it out meant a regression there
	# was verified by nothing at all.
	paths:
	- "src/**"
	- "layer/**"
	- "Cargo.toml"
	- "Cargo.lock"
	# Those same paths are deliberately NOT pull request triggers: adding them would run
	# all eighteen matrix entries on every source pull request. Adapter changes are
	# verified against the examples on push to main (above), and the selector's shared-path
	# rule still applies to a pull request that touches both source and examples.
	workflow_dispatch:

	permissions:
	contents: read

	env:
	CARGO_TERM_COLOR: always

	jobs:
	# Narrows the test matrices to the examples a pull request actually touches, so a
	# dependency bump in one example does not rebuild and boot all eighteen. Any change
	# to shared code (src/, layer/, this workflow) still verifies everything. See
	# .github/scripts/select-examples.sh.
	select:
	runs-on: ubuntu-24.04
	outputs:
	image: ${{ steps.filter.outputs.image }}
	zip: ${{ steps.filter.outputs.zip }}
	stream: ${{ steps.filter.outputs.stream }}
	steps:
	- uses: actions/checkout@v4
	with:
	# Needed to diff against the base commit rather than a shallow clone.
	fetch-depth: 0

	- name: Select the examples to verify
	id: filter
	env:
	BASE_SHA: ${{ github.event.pull_request.base.sha }}
	run: .github/scripts/select-examples.sh

	validate:
	runs-on: ubuntu-24.04
	steps:
	- uses: actions/checkout@v4

	- uses: actions/setup-python@v4
	with:
	python-version: "3.13"

	- uses: aws-actions/setup-sam@v2
	with:
	use-installer: true
	token: ${{ secrets.GITHUB_TOKEN }}

	# The two hand-maintained lists describing the examples drift silently otherwise:
	# an unclaimed manifest reverts to one Dependabot pull request per advisory with a
	# commit header Commit Lint rejects, and that is invisible until the pull requests
	# appear weeks later.
	- name: Check the example config lists against the tree
	run: \|
	pip install pyyaml --quiet
	.github/scripts/check-example-config.sh

	- name: Validate all SAM templates
	run: \|
	failed=0
	for template in $(find examples -maxdepth 2 -name "template.yaml" \| sort); do
	dir=$(dirname "$template")
	echo "Validating $dir..."
	if ! sam validate --template "$template" --lint 2>&1; then
	echo "FAIL: $dir"
	failed=1
	fi
	done
	if [ "$failed" -eq 1 ]; then
	echo "Some templates failed validation"
	exit 1
	fi

	# Only when something will use it. The artifact exists for the test jobs, so building
	# it when `select` chose nothing is several minutes of runner time for nobody.
	build-layer:
	needs: [select]
	if: >-
	needs.select.outputs.image != '[]' \|\|
	needs.select.outputs.zip != '[]' \|\|
	needs.select.outputs.stream != '[]'
	runs-on: ubuntu-24.04
	steps:
	- uses: actions/checkout@v4

	- name: Install stable toolchain
	run: rustup target add x86_64-unknown-linux-musl

	- name: Install cargo lambda
	run: pip3 install cargo-lambda

	- name: Configure Rust cache
	uses: Swatinem/rust-cache@v2

	- name: Build x86_64 layer
	run: \|
	cargo lambda build --release --extension --target x86_64-unknown-linux-musl
	mkdir -p layer-x86_64
	cp layer/bootstrap layer-x86_64/
	cp target/lambda/extensions/lambda-adapter layer-x86_64/

	- uses: actions/upload-artifact@v4
	with:
	name: layer-x86_64
	path: layer-x86_64/

	# Docker-based examples verified with sam local start-api. Each matrix entry
	# asserts an exact status (and optional body substring) via verify-http.sh,
	# not just liveness.
	# Excluded: nginx, flask, aspnet-mvc (web app hardcodes port 8080
	# which conflicts with SAM's Lambda Runtime Interface Emulator).
	test-image:
	needs: [select, build-layer]
	if: needs.select.outputs.image != '[]'
	runs-on: ubuntu-24.04
	strategy:
	fail-fast: false
	matrix:
	example: ${{ fromJSON(needs.select.outputs.image) }}
	steps:
	- uses: actions/checkout@v4

	- uses: actions/setup-python@v4
	with:
	python-version: "3.13"

	- uses: aws-actions/setup-sam@v2
	with:
	use-installer: true
	token: ${{ secrets.GITHUB_TOKEN }}

	- uses: actions/download-artifact@v4
	with:
	name: layer-x86_64
	path: /tmp/layer-x86_64

	- name: Build local adapter image
	run: \|
	# Build the local adapter under every tag the example Dockerfiles
	# reference, so each example's `docker build` (COPY --from=...:<tag>)
	# uses the locally compiled binary instead of pulling the released
	# image. Deriving the tags from the Dockerfiles keeps CI in lockstep
	# with them (no hardcoded version to forget at release time).
	chmod +x /tmp/layer-x86_64/lambda-adapter
	tags=$(grep -rhoE 'aws-lambda-adapter:[0-9A-Za-z._-]+' examples \| sed 's/.*://' \| sort -u)
	echo "Building local adapter for tags: $tags"
	for tag in $tags; do
	printf 'FROM scratch\nCOPY --chmod=755 lambda-adapter /lambda-adapter\n' \| \
	docker build -t public.ecr.aws/awsguru/aws-lambda-adapter:$tag -f- /tmp/layer-x86_64
	done

	- name: Build
	working-directory: examples/${{ matrix.example.name }}
	run: \|
	# Retry to tolerate transient registry rate limits (toomanyrequests: Rate
	# exceeded). public.ecr.aws throttles unauthenticated pulls at ~1 req/s per
	# source IP, and the matrix runs many jobs from shared runner IPs, so bursts
	# collide. Exponential backoff with random jitter desynchronizes retries
	# across jobs to avoid a thundering herd re-colliding on the same boundary.
	max_attempts=5
	attempt=1
	while true; do
	if sam build; then exit 0; fi
	if [ "$attempt" -ge "$max_attempts" ]; then
	echo "sam build failed after ${max_attempts} attempts"
	exit 1
	fi
	delay=$(( 10 * 2 ** (attempt - 1) + RANDOM % 16 )) # ~10/20/40/80s + 0-15s jitter
	echo "sam build failed (attempt ${attempt}/${max_attempts}), retrying in ${delay}s..."
	sleep "$delay"
	attempt=$(( attempt + 1 ))
	done

	- name: Start local API and verify
	working-directory: examples/${{ matrix.example.name }}
	run: \|
	# Set PORT=8000 to avoid conflict with SAM's RIE on port 8080.
	# Start and verify in the same step so the backgrounded sam local
	# stays attached to this shell while requests are served (splitting
	# it across steps races the container warm-up and yields 500s).
	echo '{"Parameters":{"PORT":"8000"}}' > /tmp/env.json
	sam local start-api --port 3000 --warm-containers EAGER --container-env-vars /tmp/env.json &
	echo "SAM_PID=$!" >> $GITHUB_ENV
	"$GITHUB_WORKSPACE/.github/scripts/verify-http.sh" \
	http://127.0.0.1:3000 \
	"${{ matrix.example.path }}" \
	200 \
	"${{ matrix.example.expect_body }}"

	- name: Stop local API
	if: always()
	run: kill $SAM_PID 2>/dev/null \|\| true

	# Zip-based examples verified with local layer. Each matrix entry asserts an
	# exact status (and optional body substring) via verify-http.sh.
	# `port` is the app's listening port, passed through to the adapter via the
	# PORT parameter; it must differ from SAM's RIE port (8080).
	# Excluded: aspnet-*-zip (hardcode port 8080),
	# bun/nginx (need third-party layers), arm64 examples (javalin, rust-*),
	# nextjs-zip (Makefile produces a zip artifact incompatible with sam local).
	test-zip:
	needs: [select, build-layer]
	if: needs.select.outputs.zip != '[]'
	runs-on: ubuntu-24.04
	strategy:
	fail-fast: false
	matrix:
	example: ${{ fromJSON(needs.select.outputs.zip) }}
	steps:
	- uses: actions/checkout@v4

	- uses: actions/setup-python@v4
	with:
	python-version: "3.13"

	- uses: aws-actions/setup-sam@v2
	with:
	use-installer: true
	token: ${{ secrets.GITHUB_TOKEN }}

	- uses: denoland/setup-deno@v2
	if: matrix.example.name == 'deno-zip'
	with:
	deno-version: v2.x

	- uses: actions/download-artifact@v4
	with:
	name: layer-x86_64
	path: /tmp/layer-x86_64

	- name: Build
	working-directory: examples/${{ matrix.example.name }}
	run: \|
	# Retry to tolerate transient registry rate limits (toomanyrequests: Rate
	# exceeded). public.ecr.aws throttles unauthenticated pulls at ~1 req/s per
	# source IP, and the matrix runs many jobs from shared runner IPs, so bursts
	# collide. Exponential backoff with random jitter desynchronizes retries
	# across jobs to avoid a thundering herd re-colliding on the same boundary.
	max_attempts=5
	attempt=1
	while true; do
	if sam build; then exit 0; fi
	if [ "$attempt" -ge "$max_attempts" ]; then
	echo "sam build failed after ${max_attempts} attempts"
	exit 1
	fi
	delay=$(( 10 * 2 ** (attempt - 1) + RANDOM % 16 )) # ~10/20/40/80s + 0-15s jitter
	echo "sam build failed (attempt ${attempt}/${max_attempts}), retrying in ${delay}s..."
	sleep "$delay"
	attempt=$(( attempt + 1 ))
	done

	- name: Inject local layer into build output
	working-directory: examples/${{ matrix.example.name }}
	run: \|
	# Prepare local layer directory with correct structure
	LAYER_DIR=".aws-sam/build/LocalAdapterLayer"
	mkdir -p "$LAYER_DIR/extensions"
	cp /tmp/layer-x86_64/bootstrap "$LAYER_DIR/"
	cp /tmp/layer-x86_64/lambda-adapter "$LAYER_DIR/extensions/"
	chmod +x "$LAYER_DIR/bootstrap" "$LAYER_DIR/extensions/lambda-adapter"
	# Add local layer resource and replace remote layer ARN refs
	pip install pyyaml -q
	python3 << 'PYEOF'
	import yaml
	with open(".aws-sam/build/template.yaml") as f:
	t = yaml.safe_load(f)
	t["Resources"]["LocalAdapterLayer"] = {
	"Type": "AWS::Lambda::LayerVersion",
	"Properties": {"Content": "LocalAdapterLayer", "LayerName": "local-adapter"},
	}
	for r in t.get("Resources", {}).values():
	layers = r.get("Properties", {}).get("Layers", [])
	for i, l in enumerate(layers):
	if "LambdaAdapterLayerX86" in str(l):
	layers[i] = {"Ref": "LocalAdapterLayer"}
	with open(".aws-sam/build/template.yaml", "w") as f:
	yaml.dump(t, f)
	PYEOF

	- name: Start local API and verify
	working-directory: examples/${{ matrix.example.name }}
	run: \|
	# Pass the app's listening port via PORT (must differ from RIE's 8080).
	# Start and verify in the same step so the backgrounded sam local
	# stays attached to this shell while requests are served (splitting
	# it across steps races the container warm-up and yields 500s).
	echo '{"Parameters":{"PORT":"${{ matrix.example.port }}"}}' > /tmp/env.json
	sam local start-api --port 3000 --warm-containers EAGER --container-env-vars /tmp/env.json &
	echo "SAM_PID=$!" >> $GITHUB_ENV
	"$GITHUB_WORKSPACE/.github/scripts/verify-http.sh" \
	http://127.0.0.1:3000 \
	"${{ matrix.example.path }}" \
	200 \
	"${{ matrix.example.expect_body }}"

	- name: Stop local API
	if: always()
	run: kill $SAM_PID 2>/dev/null \|\| true

	# Response-streaming examples verified by building and running the real app,
	# then asserting an exact status/body with verify-http.sh. These use a Lambda
	# Function URL with RESPONSE_STREAM and have no API Gateway events, so
	# sam local start-api (used by test-image/test-zip) cannot drive them.
	#
	# `kind` selects how the app is launched:
	# image - build the app image (its Dockerfile COPYs the local adapter
	# image built below) and run it with PORT=8000.
	# zip-fasthtml - run `python app/main.py` (FastHTML's serve() binds 8000).
	# AWS_REGION is set so boto3 / AnthropicBedrock clients construct without real
	# creds; the verified routes only render UI and never call Bedrock.
	test-stream:
	needs: [select, build-layer]
	if: needs.select.outputs.stream != '[]'
	runs-on: ubuntu-24.04
	strategy:
	fail-fast: false
	matrix:
	example: ${{ fromJSON(needs.select.outputs.stream) }}
	steps:
	- uses: actions/checkout@v4

	- uses: actions/setup-python@v4
	with:
	python-version: "3.12"

	- uses: actions/download-artifact@v4
	with:
	name: layer-x86_64
	path: /tmp/layer-x86_64

	- name: Build local adapter image
	run: \|
	# Build the local adapter under every tag the example Dockerfiles
	# reference, so each example's `docker build` (COPY --from=...:<tag>)
	# uses the locally compiled binary instead of pulling the released
	# image. Deriving the tags from the Dockerfiles keeps CI in lockstep
	# with them (no hardcoded version to forget at release time).
	chmod +x /tmp/layer-x86_64/lambda-adapter
	tags=$(grep -rhoE 'aws-lambda-adapter:[0-9A-Za-z._-]+' examples \| sed 's/.*://' \| sort -u)
	echo "Building local adapter for tags: $tags"
	for tag in $tags; do
	printf 'FROM scratch\nCOPY --chmod=755 lambda-adapter /lambda-adapter\n' \| \
	docker build -t public.ecr.aws/awsguru/aws-lambda-adapter:$tag -f- /tmp/layer-x86_64
	done

	- name: Build and start app
	working-directory: examples/${{ matrix.example.name }}
	env:
	KIND: ${{ matrix.example.kind }}
	run: \|
	case "$KIND" in
	image)
	docker build -t stream-app app
	docker run -d --name stream-app \
	-e PORT=8000 -e AWS_REGION=us-east-1 -e AWS_DEFAULT_REGION=us-east-1 \
	-p "8000:8000" stream-app
	;;
	zip-fasthtml)
	pip install -r app/requirements.txt
	PORT=8000 nohup python app/main.py > /tmp/app.log 2>&1 &
	echo "APP_PID=$!" >> $GITHUB_ENV
	;;
	*)
	echo "Unknown kind: $KIND"; exit 1 ;;
	esac

	- name: Verify
	run: \|
	"$GITHUB_WORKSPACE/.github/scripts/verify-http.sh" \
	http://127.0.0.1:8000 \
	"${{ matrix.example.path }}" \
	200 \
	"${{ matrix.example.expect_body }}" \
	\|\| { echo "--- app logs ---"; docker logs stream-app 2>/dev/null \|\| cat /tmp/app.log 2>/dev/null \|\| true; exit 1; }

	- name: Stop app
	if: always()
	run: \|
	docker rm -f stream-app 2>/dev/null \|\| true
	kill $APP_PID 2>/dev/null \|\| true

	# One aggregate result for the whole workflow, so a reviewer has a single thing to look
	# at before merging: the matrix jobs' names embed their parameters
	# ("test-zip (deno-zip, /, success, 8000)") and vanish entirely when `select` filters an
	# example out, so there is no stable per-example name to read.
	#
	# `skipped` counts as a pass — that is what a filtered-out matrix means. `failure`
	# and `cancelled` do not.
	#
	# Note if you ever make this a required check in main's ruleset: this workflow is
	# path-filtered, so it never runs on a pull request that touches none of those paths and
	# the check would never report there, blocking that pull request indefinitely. Making it
	# required needs a companion job reporting the same name on the complementary paths.
	examples-verified:
	if: always()
	needs: [select, validate, build-layer, test-image, test-zip, test-stream]
	runs-on: ubuntu-24.04
	steps:
	- name: Check the verification results
	env:
	RESULTS: ${{ join(needs.*.result, ' ') }}
	IMAGE: ${{ needs.select.outputs.image }}
	ZIP: ${{ needs.select.outputs.zip }}
	STREAM: ${{ needs.select.outputs.stream }}
	run: \|
	echo "job results: $RESULTS"
	for result in $RESULTS; do
	case "$result" in
	success \| skipped) ;;
	*)
	echo "A verification job reported '$result'."
	exit 1
	;;
	esac
	done

	# Green is not the same claim in both cases. Most examples have no matrix entry,
	# so their pull requests filter every matrix to empty, skip the test jobs, and
	# arrive here green having built and booted nothing. Say which it was, because a
	# reviewer reading this one check has nothing else to go on — the per-example job
	# names vanish when the matrix is filtered, which is why this job exists.
	verified=$(jq -rn \
	--argjson image "${IMAGE:-[]}" --argjson zip "${ZIP:-[]}" --argjson stream "${STREAM:-[]}" \
	'[$image, $zip, $stream] \| flatten \| map(.name) \| unique \| join(", ")')
	if [[ -n "$verified" ]]; then
	echo "Built and booted: $verified"
	echo "Verified: $verified" >>"$GITHUB_STEP_SUMMARY"
	else
	echo "No example was built or booted; this run verified templates only."
	echo "No example was built or booted — template validation only." \
	>>"$GITHUB_STEP_SUMMARY"
	fi

Navigation Menu

Uh oh!

ci: group Dependabot example updates and verify only what changed #283

Workflow file

ci: group Dependabot example updates and verify only what changed #283

Uh oh!

Workflow file for this run