Skip to content

ci: add Codecov#3237

Merged
sh0ji merged 17 commits intomainfrom
evan/codecov
Jan 26, 2026
Merged

ci: add Codecov#3237
sh0ji merged 17 commits intomainfrom
evan/codecov

Conversation

@sh0ji
Copy link
Member

@sh0ji sh0ji commented Jan 21, 2026

This adds Codecov to Gamut, including an initial codecov.yml config, coverage reporting, and their new test analytics so we can try it out. Note that it only runs for PRs so we can test it. It should result in a report at https://app.codecov.io/gh/Codecademy/gamut/. If it works, I'll add it for the full repo.

Here are the Codecov results for this branch:

I also made a few optimizations to the test workflow:

And I refactored the Jest configs a bit in b184152:

  • All coverage and junit reports now just go into ./coverage. These reports were previously writing to /tmp in CI, which felt unnecessary, since ./coverage is gitignored and anything written to the filesystem during a GitHub Actions run goes away after the runner finishes the job, so there's no real need to write to memory. Correct me if I'm misunderstanding something about how nx caches things, though.
  • I moved everything from the jest.preset.js file into the jest.config.base.ts except the @nx/jest/preset stuff. I initially just got rid of the preset entirely, but it seemed like the preset: 'path/to/jest-preset.js is doing something more than just merging objects, so I left it as-is.
    • I also tried rerunning nx g @nx/jest:configuration --project=gamut to see what the generator created, and it did something similar (packages/gamut/jest.config.ts had a preset: field pointing to the root preset).

PR Checklist

  • Related to designs:
  • Related to JIRA ticket: GMT-1547
  • I have run this code to verify it works
  • This PR includes unit tests for the code change
  • This PR includes testing instructions tests for the code change
  • The alpha package of this PR is passing end-to-end tests in all relevant Codecademy repositories

@sh0ji sh0ji requested a review from a team as a code owner January 21, 2026 18:05
@sh0ji sh0ji requested review from LinKCoding and dreamwasp January 21, 2026 18:05
@sh0ji sh0ji changed the title Add CodeCov CI: Add CodeCov Jan 21, 2026
@sh0ji sh0ji changed the title CI: Add CodeCov ci: Add CodeCov Jan 21, 2026
@nx-cloud
Copy link

nx-cloud bot commented Jan 21, 2026

View your CI Pipeline Execution ↗ for commit e002eac


☁️ Nx Cloud last updated this comment at 2026-01-26 21:10:57 UTC

@sh0ji sh0ji marked this pull request as draft January 21, 2026 18:12
sh0ji added 6 commits January 21, 2026 13:45
- we're just using the default coverageReporters: https://jestjs.io/docs/configuration#coveragereporters-arraystring--string-options
- the base config is using the preset, so we only need to declare reporters in one place
- 'summary' is an invalid reporter
- we want to always include the github-actions reporter in CI
these are different from the reporters
@codecov
Copy link

codecov bot commented Jan 21, 2026

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@sh0ji sh0ji changed the title ci: Add CodeCov ci: add Codecov Jan 21, 2026
@sh0ji sh0ji marked this pull request as ready for review January 26, 2026 18:34
@sh0ji sh0ji requested review from LinKCoding and dreamwasp January 26, 2026 18:34
@sh0ji
Copy link
Member Author

sh0ji commented Jan 26, 2026

heads up that while trying to get the test analytics working, I discovered that the junit reports weren't ever being generated/uploaded, so I took some time to fix that and refactored the Jest configs a bit.

You can verify that the reports weren't being uploading by looking at the test-results artifacts in this PR and others:

  • Cass' open InfoTip test-results.zip. if you download the test-results.zip and extract it, you'll just see the clover.xml & coverage-final.json, but not junit XML report, which should be named gamut-report.xml.
  • This branch's test-results.zip. I reorganized it so it's ${packageName}/{coverage files + junit.xml}. Codecov can parse this to discover everything it needs.

This should also theoretically enable GitHub's coverage annotations in PR diffs, but we'll have to wait until merging to verify that.

Copy link
Contributor

@aresnik11 aresnik11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! looks good and makes sense to me!

run: git fetch origin main:main
if: github.event_name == 'pull_request'
- name: Ensure workflow is associated with a pull request
uses: ./.github/actions/validate-pr-context
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think the whole validate-pr-context thing is a remnant of circleCI where we couldn't choose triggers other than push, so we had to make sure that the PR was associated before making comments and stuff. I don't think it is relevant/needed with this workflow. Could clean that up later but in the meantime this is a little confusing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, that explains it! I had a version where I removed these steps but reverted it because it felt too unrelated to the PR.

Comment on lines 36 to +38
- name: Skip build from automated commit
uses: ./.github/actions/skip-automated-commits
if: github.event_name == 'pull_request'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this also doesn't really matter anymore, it's ok if we run tests after the bot commits changes after publishing.

Copy link
Member

@jakemhiller jakemhiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! a couple non-blocking comments

@sh0ji sh0ji enabled auto-merge (squash) January 26, 2026 21:09
@sh0ji sh0ji merged commit 6ae8f8d into main Jan 26, 2026
15 checks passed
@sh0ji sh0ji deleted the evan/codecov branch January 26, 2026 21:12
@github-actions
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants