chore: adding vap testing#618
chore: adding vap testing#618JaydipGabani wants to merge 10 commits intoopen-policy-agent:masterfrom
Conversation
JaydipGabani
force-pushed
the
vap-test
branch
2 times, most recently
from
7dd57a7 to
0a8532a
Compare
| @@ -36,9 +36,9 @@ deploy: | |||
| ifeq ($(POLICY_ENGINE), rego) | |||
There was a problem hiding this comment.
Can you add a comment here that only the first engine in the template gets used for evaluation UNLESS enableK8sNativeValidation=false which ensures the subsequent non-K8sNativeValidation engine gets used?
| ifneq ($(GATEKEEPER_VERSION), 3.15.1) | ||
| helm install -n gatekeeper-system gatekeeper gatekeeper/gatekeeper --create-namespace --version $(GATEKEEPER_VERSION) --set enableK8sNativeValidation=true | ||
| endif | ||
| else ifeq ($(POLICY_ENGINE), vap) |
There was a problem hiding this comment.
For these, since Gatekeeper webhook is a fallback for VAP, how do we ensure the failure resulted from VAP instead of the GK webhook?
There was a problem hiding this comment.
We check the violation string to make sure it includes ValidatingAdmissionPolicy - https://github.com/open-policy-agent/gatekeeper-library/pull/618/files#diff-3b0266d06db240d302bf9febd17c09cf104dccce0bb2ee8e799e49c85f7082ffR96.
| @@ -36,9 +36,9 @@ deploy: | |||
| ifeq ($(POLICY_ENGINE), rego) | |||
There was a problem hiding this comment.
In general, I think because we are not actually testing the error message, it's hard to tell which engine caused the violation and which enforcement point caused the failure. Not sure how hard it is to add that, could be a follow up issue/PR if you want to track it.
There was a problem hiding this comment.
Since GK does not have fallback between engines, violations are thrown through CEL engine if it is enabled and CT has CEL. Otherwise the source of violation is rego engine.
There was a problem hiding this comment.
Since GK does not have fallback between engines, violations are thrown through CEL engine if it is enabled and CT has CEL. Otherwise the source of violation is rego engine.
yea thats how it works today but if we ever expose priority like the issue you opened, then we need to test the actual violation message. maybe open an issue to track for future?
There was a problem hiding this comment.
In this case, we may want to modify the violation message to include engine information as well. Since as of now the violation message is the same regardless of the engine used to evaluate CTs. IMO, I don't think users apart from CT authors would care about which engine is being used to enforce policies. And CT authors would only care to verify the logic written for policy, which I think can be attained with gator to test CT.
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
JaydipGabani
force-pushed
the
vap-test
branch
from
7fd9ed2 to
5681b2c
Compare
|
This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions. |
|
Not stale |
|
This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions. |
|
This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions. |
|
This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions. |
|
This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions. |
3 participants
What this PR does / why we need it:
Which issue(s) does this PR fix (optional, using
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when the PR gets merged):Fixes #574
Special notes for your reviewer: