add end to end tests

[jesseturner21]

Description

This pr establishes the base for end to end tests.

1. added etoe test folder for tests that call aws resources
2. added workflow to call to run end to end test in github actions. (the team can discuss when/how this workflow should be called eg. every day or before releases)
3. removed deploy and invoke integ test, to have clear division of responsibilities for integ and etoe tests. Integ tests run locally and etoe deploy and make calls to aws.

The first end to end test this adds is create -> deploy -> invoke. I plan to add more coverage (eg. frameworks, memory, in a follow up)

• Bug fix
• New feature
• Breaking change
• Documentation update
• Other (please describe): test

Testing

How have you tested the change?

• I ran npm run test:all
• I ran npm run typecheck
• I ran npm run lint
• If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

Checklist

• I have read the CONTRIBUTING document
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

[github-actions]

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	28.26%	1807 / 6393
🔵	Statements	27.59%	1875 / 6794
🔵	Functions	27.36%	356 / 1301
🔵	Branches	24.52%	918 / 3743

Generated in workflow #372 for commit 0f18d30 by the Vitest Coverage Report Action

[aidandaly24]

in general looks good. the JSON.parse thing is just a nit that could be improved in the places that it is used around the update

[notgitika]

left some comments. overall this is a great start towards end to end tests!!

few things overall we might need to think as follow ups when you head towards a. more complete e2e test suite:

• mechanism for cleaning up resources if tests crash before you run teardown
• adding when are these e2e tests triggered. imo it should be every pre-release.

[aidandaly24]

/strands can you review this PR. We don't want to make any changes just want to review and make sure the changes are in line with our code base. Please take a look and leave relevant comments. Don't take actions on the comments.

[github-actions]

Code Review Summary

I've completed a comprehensive review of this PR. Overall, this is solid foundational work for E2E testing with good patterns. Here are the key findings organized by priority:

🔴 High Priority Issues

1. Naming Consistency
The PR mixes e2e and etoe naming throughout:

• Directory: etoe-tests/, Test description: e2e: create →...
• Workflow name: "E2E Tests", File: etoe-tests.yml
• Secret: ETOE_AWS_ROLE_ARN, Script: test:etoe

Recommendation: Standardize on e2e (industry standard) across all locations.

2. Missing Documentation

• docs/TESTING.md needs E2E section (purpose, differences from integ, prerequisites, when they run)
• Missing e2e-tests/README.md (should follow integ-tests/README.md pattern)

3. Removed Test Coverage
The PR removes 205 lines from integ-tests/deploy.test.ts and integ-tests/invoke-agent.test.ts. Can you confirm what scenarios (if any) are not covered by the new E2E test? This helps reviewers assess the trade-off.

🟡 Medium Priority Issues

4. JSON Parsing (Lines 44, 82, 106)
Use parseJsonOutput from test-utils instead of raw JSON.parse():

• Better ANSI handling
• Clear error messages
• Consistent with codebase patterns

5. Test Execution Order (Lines 68-110)
Tests share state and depend on order but Vitest doesn't guarantee this. Consider describe.sequential() or restructure as single test.

6. Git Config Inconsistency (workflow line 24-26)
Uses [email protected] but build-and-test.yml uses [email protected]. Should be consistent.

🟢 Low Priority / Nits

7. Future Work Comment (Line 52)
Add comment explaining aws-targets.json manual write bypasses add target command temporarily.

8. Error Logging Enhancement (Lines 75-77, 99-101)
Consider logging the command that failed, not just stdout/stderr.

9. Workflow Improvements

• Consider description field and cost warning
• Document why single Node version vs matrix
• Align workflow name with file name

10. Vitest Config (Line 57-58)
Add comment explaining why 10-minute timeouts (AWS operations).

✅ What's Done Well

• ✅ Proper OIDC authentication (no static credentials)
• ✅ Good use of existing test utilities
• ✅ Appropriate cleanup with guards
• ✅ Clear separation between test types
• ✅ Reasonable timeouts for cloud operations

Next Steps

The high-priority items (naming, documentation, coverage validation) should be addressed. The medium-priority items improve code quality and align with existing patterns. Low-priority items are nice-to-haves.

This is the right approach for E2E testing - great foundation! 🚀