chore: add chainsaw e2e tests support

[eddycharly]

Checklist:

• Have you added an explanation of what your changes do and why you'd like them to be included?
• Have you updated or added documentation for the change, as applicable?
• Have you tested your changes on all related environments with successful results, as applicable?
  • Have you added automated tests?

Type of Changes:

• New feature
• Bug fix
• Documentation
• Testing enhancement
• Other

What is the current behavior (link to any open issues here)?

Tests are run with kuttl

What is the new behavior (if this is a feature change)?

• Breaking change (fix or feature that would cause existing functionality to change)

Tests are run with both chainsaw and kuttl

Other Information:

Chainsaw is similar to kuttl but a lot more flexible and actively maintained.

If you need more infos 👇

• GitHub: https://github.com/kyverno/chainsaw
• Docs: https://kyverno.github.io/chainsaw
• Slack: https://kubernetes.slack.com/archives/C067LUFL43U

[eddycharly]

cc @jmckulk ;)

[jmckulk]

Thanks for doing this! We have a story in our backlog to check out chainsaw and have added your pr to it!

Were you able to use the chainsaw migrate command? I know when I tried before it didn't like our scripts.

[eddycharly]

Yup, it was all done with chainsaw migrate kuttl tests ....

Only a few commands didn't pass (i fixed a bunch of bugs in v0.1.3 and a couple of other fixes will come in v0.1.4).

BTW would you mind approving the workflows to check how it works ?

[jmckulk]

The generate-kuttl Make target in the workflow is used to fill out these variables with certain variables that might be different between tests. We would need to update/duplicate that target for chainsaw or figure out a different way to use variables.

@eddycharly Does chainsaw have a concept of variables or variable expansion?

[eddycharly]

v0.1.3 introduced support to pass arbitrary values to tests and v0.1.4 will come with a solution to inject values in the resource manifests so basically no need for a generate step anymore :)