Skip to content

feat(scale-set): add scale set orchestration in Terraform - #5299

Open
edersonbrilhante wants to merge 18 commits into
mainfrom
feat-scale-set-terraform
Open

edersonbrilhante wants to merge 18 commits into
mainfrom
feat-scale-set-terraform

Conversation

@edersonbrilhante

@edersonbrilhante edersonbrilhante commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Description

Adds Terraform support for experimental GitHub Actions runner scale-set orchestration in the multi-runner v2 interface.

This PR includes:

  • A standalone modules/orchestration-providers/scale-set module that deploys one ECS Fargate controller service per resolved controller group, including the ECS cluster/service/task definition, private networking, security groups, CloudWatch logging, health checks, deployment rollback, and hardened task settings.
  • Plan-known grouping by compute-provider type (the default), runner configuration, or explicit custom membership. Grouping keeps each GitHub scale set and message session independent while packing multiple reconcilers into one controller task.
  • Versioned non-secret reconciler configuration delivery through SSM Parameter Store and the controller manifest. GitHub App credentials remain SSM parameter references, with task and compute-role permissions restricted to the configured resources.
  • Validation for GitHub scope and scale-set name ownership, grouping coverage, plan-time for_each inputs, provider contracts, configuration and task-definition size limits, reserved environment variables, wildcard IAM actions, and AWS inline-policy quotas.
  • A compute-provider capability boundary that carries provider-owned runtime configuration, environment variables, and IAM statements. The EC2 adapter supplies scale-set launch, tagging, termination, JIT-configuration storage, AMI, and related permissions with ownership conditions.
  • Multi-runner and runner-config wiring, the examples/multi-runner-scale-set example, MiniStack fixtures, and CI updates for the new Terraform path.

The module adopts scale sets that exist in GitHub by name; it does not create or delete GitHub scale-set resources. The TypeScript controller that consumes this Terraform contract is implemented in #5300.

Test Plan

  • Added focused Terraform tests for the scale-set module, computed-input fixtures, grouping, ownership validation, configuration delivery, IAM policy construction, and quota checks.
  • Added the multi-runner scale-set example and MiniStack variables/runner coverage.
  • Terraform formatting, validation/TFLint, and the repository CI workflows cover the updated modules and example paths.

Related Issues

@edersonbrilhante
edersonbrilhante requested review from a team as code owners August 26, 2026 10:44
@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@edersonbrilhante
edersonbrilhante marked this pull request as draft August 26, 2026 13:59
@edersonbrilhante
edersonbrilhante force-pushed the experimental-multi-runner-config-v2-20260805 branch 2 times, most recently from 5eafe5c to 2b6bb21 Compare September 2, 2026 20:18
@edersonbrilhante
edersonbrilhante changed the base branch from experimental-multi-runner-config-v2-20260805 to refactor-runner-config-module September 3, 2026 19:59
@edersonbrilhante
edersonbrilhante force-pushed the refactor-runner-config-module branch 2 times, most recently from c979cbb to 4d9e31c Compare September 3, 2026 20:03
@edersonbrilhante
edersonbrilhante changed the base branch from refactor-runner-config-module to experimental-multi-runner-config-v2-20260805 September 3, 2026 20:05
@edersonbrilhante
edersonbrilhante changed the base branch from experimental-multi-runner-config-v2-20260805 to fix-multi-runner-v2-routing September 3, 2026 20:21
@edersonbrilhante edersonbrilhante changed the title feat(scale-set): add ECS orchestration feat(scale-set): add scale set orchestration in Terraform Sep 3, 2026
@edersonbrilhante
edersonbrilhante force-pushed the feat-scale-set-terraform branch 2 times, most recently from 5ab14e1 to 4375247 Compare September 3, 2026 21:13
@edersonbrilhante
edersonbrilhante force-pushed the fix-multi-runner-v2-routing branch from 00c76ce to 01c4a78 Compare September 3, 2026 21:19
@edersonbrilhante
edersonbrilhante force-pushed the fix-multi-runner-v2-routing branch from 01c4a78 to 5b2fbf4 Compare September 3, 2026 21:40
@edersonbrilhante
edersonbrilhante force-pushed the fix-multi-runner-v2-routing branch from 5b2fbf4 to afc760b Compare September 3, 2026 21:52
@edersonbrilhante
edersonbrilhante changed the base branch from fix-multi-runner-v2-routing to microvm-multi-runner-terraform September 3, 2026 22:03
@edersonbrilhante
edersonbrilhante changed the base branch from microvm-multi-runner-terraform to microvm-multi-runner-integration-split September 3, 2026 22:04
@edersonbrilhante
edersonbrilhante changed the base branch from microvm-multi-runner-integration-split to fix-multi-runner-v2-routing September 3, 2026 22:04
@edersonbrilhante
edersonbrilhante force-pushed the fix-multi-runner-v2-routing branch from afc760b to e17ae16 Compare September 4, 2026 17:59
@edersonbrilhante
edersonbrilhante changed the base branch from fix-multi-runner-v2-routing to remove-v2-legacy-input-requirements September 4, 2026 18:08
@edersonbrilhante
edersonbrilhante changed the base branch from remove-v2-legacy-input-requirements to add-multi-runner-v2-example September 4, 2026 18:10
@edersonbrilhante
edersonbrilhante changed the base branch from add-multi-runner-v2-example to main September 4, 2026 18:12
@edersonbrilhante
edersonbrilhante changed the base branch from main to add-multi-runner-v2-example September 4, 2026 18:12
@edersonbrilhante
edersonbrilhante force-pushed the add-multi-runner-v2-example branch 7 times, most recently from 4ea1b14 to 69cdafe Compare September 10, 2026 09:09
@edersonbrilhante
edersonbrilhante force-pushed the add-multi-runner-v2-example branch from 8879acc to 23fe569 Compare September 10, 2026 12:22

@guicaulada guicaulada left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on this! The overall approach looks good to me, but we should address the enterprise scope configuration and IAM role-name limits before merging. I’ve added details inline.

Comment thread modules/orchestration-providers/scale-set/locals.tf
Comment thread modules/orchestration-providers/scale-set/locals.tf Outdated
edersonbrilhante and others added 15 commits September 17, 2026 20:20
## Description

Wire scale-set orchestration through the provider-aware Terraform
composition.

- Resolve global and per-lane scale-set settings into the canonical
runner configuration.
- Route scale-set lanes through `runner-config` and the scale-set
orchestration provider.
- Extend the EC2 provider contract with the scale-set runtime
configuration, IAM fragments, and outputs consumed by the controller.
- Gate webhook resources by the selected orchestration provider.
- Add scale-set validation and Terraform coverage across the affected
modules.

This PR provides the Terraform wiring used by the scale-set example and
the later MiniStack integration PRs.

## Test Plan

- Terraform formatting, validation/TFLint, and merge-conflict checks
passed through the repository hooks.
- Scale-set routing and configuration-resolution tests passed.
- Multi-runner and runner-config scale-set Terraform tests passed.
- Native provider tests remain subject to the local macOS arm64
plugin-handshake limitation.

## Related Issues

Depends on #5299.

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
## Description

Restore the multi-runner scale-set orchestration example removed by
revert PR #5403, replacing merged PR #5378.

The example provides the Terraform configuration, provider locks,
outputs, and documentation needed to deploy an ECS scale-set controller
with the EC2 runner compute provider. It is intentionally limited to the
example and its generated/provider metadata; the reusable MiniStack
fixture support and ECS/MockServer lifecycle smoke test are provided by
the follow-up PRs.

## Test Plan

- Terraform formatting passed through the repository hooks.
- Merge-conflict checks passed.
- Parent-branch Terraform checks passed after the idle-configuration
correction.
- No live AWS deployment was performed for this example-only PR; CI
validation remains the authoritative deployment check.

## Related Issues

Depends on #5350. Replaces #5378.

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants