Test For command falco -i (ignore default events) #8

Rohith-Raju · 2023-03-29T16:02:11Z

This issue is related to #7,
Checkoff task

i

Implementation:

Move all the ignored events to the events.json file. The reason behind this was that if we were to store all the events in the test, it would look dirty.
Read the event.json file, loop over the events, and match with the regex expression.

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

poiana · 2023-03-29T16:02:20Z

Welcome @Rohith-Raju! It looks like this is your first PR to falcosecurity/testing 🎉

jasondellaluce

Thanks a ton for your first contribution! Much appreciated, specially in this repo!!

jasondellaluce · 2023-03-29T16:40:07Z

tests/falco/events.json

@@ -0,0 +1,278 @@
+{


This makes sense! What about putting this under a tests/data/outputs directory, and masking this through run.NewStringFileAccessor? That would allow the framework to carry around the test file in a runner-agnostic way (it will work whether Falco runs in a container or in the local machine). Accordingly, I would also add the deserialization code in the same package (the part of code responsible of converting the JSON in a Go struct). Take a look at how some of the existing test data is handled.

Sure, will do

jasondellaluce · 2023-03-29T16:41:05Z

tests/falco/commands_test.go

+	}
+
+	runner := tests.NewFalcoExecutableRunner(t)
+	t.Run("all-events-ignored-by-default", func(t *testing.T) {


Since we don't have other tests here, I would avoid having a sub-test with t.Run and would just proceed with the test code in the main TestFalco_Print_IgnoredEvents test body, what do you think?

Test the behaviour and expected output of the Falco CLI depending on its options, considered both individually and in combinations:

Yes, since there no combinations for -i we can proceed with the main TestFalco_Print_IgnoredEvents

jasondellaluce · 2023-03-29T16:41:42Z

tests/falco/commands_test.go

+	type Data struct {
+		Events []string `json:"events"`
+	}
+	evntfile, err := os.Open("events.json")
+	if err != nil {
+		panic(err)
+	}
+	defer evntfile.Close()
+
+	var events Data
+	err = json.NewDecoder(evntfile).Decode(&events)
+	if err != nil {
+		panic(err)
+	}


Look at the comment below: I think this should be part of the test data package under tests/data/outputs, so that it could be accessible by other tests as well and reused.

jasondellaluce · 2023-03-29T16:43:07Z

tests/falco/commands_test.go

+			`Ignored\sEvent\(s\):\n`,
+		), res.Stdout())
+		for _, event := range events.Events {
+			assert.Regexp(t, regexp.MustCompile(`\b`+event+`\b`), res.Stdout())


Maybe assert.Contains would be more lightweight here, since events are simple known strings. Remember, the more we make tests optimized, the less the CI will take to run them!

jasondellaluce · 2023-03-29T16:43:33Z

tests/falco/events.json

@@ -0,0 +1,278 @@
+{
+  "events": [


Out of curiosity, which Falco version did you use to generate this list?

Falco version: 0.34.1 Libs version: 0.10.4 Plugin API: 2.0.0 Engine: 16 Driver: API version: 3.0.0 Schema version: 2.0.0 Default driver: 4.0.0+driver

Falco version: 0.34.1

Rohith-Raju · 2023-03-30T03:52:32Z

Hi, @jasondellaluce thanks for reviewing my pr. I'll make the requested changes by Saturday as I'm caught up with college exams until tomorrow 😄

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

jasondellaluce · 2023-04-03T08:28:18Z

tests/data/outputs/event.go

+	Events []string `json:"events"`
+}
+
+func deserialize() string {


Since we have a fairly recent version of Go, let me suggest considering go:embed https://pkg.go.dev/embed. I think it's a better approach than going through runtime.Caller and reinvent the wheel in this case! What do you think?

Plus, if the only point of having a JSON file is to deserialize it and encode the inner list with comma-separated concatenation, why not just encoding all strings as comma-separated in a text file right away? This would also save you the trouble of defining a Data struct. I see the attempt of making something reusable, but I would suggest to 1) tackle reusability later when needed and keep the scope of this PR smaller, and 2) the JSON encoding is arbitrary, so it would be less intuitive for future contributors to reproduce.

TL;DR: Great work! My last suggestions are:

Encode the file as a comma-separated text file containing the list of event names

Include it in the project using go:embed, and then making it available through the NewStringFileAccessor construct like you already do.

Since we have a fairly recent version of Go, let me suggest considering go:embed https://pkg.go.dev/embed. I think it's a better approach than going through runtime.Caller and reinvent the wheel in this case! What do you think?

Oh, wow I didn't know something like this existed 😮 . Yes, this is great.

Encode the file as a comma-separated text file containing the list of event names

Include it in the project using go:embed, and then making it available through the NewStringFileAccessor construct like you already do.

Will do!!

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

jasondellaluce · 2023-04-03T13:49:24Z

Looks good now! Your test is failing in the latest version of Falco master. That's reasonable, because we're changing the logic with which events are selected/ignored for the next Falco release. Please make sure you use the latest master version of Falco to generate your syscall list (you can use the :master tag of the falcosecurity/falco-no-driver image).

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

Rohith-Raju · 2023-04-04T15:11:59Z

Hey @jasondellaluce, So I exec ed into the falcosecurity/falco-no-driver:master and ran falco -i and it gave me these

root@933f5c41b8e2:/# falco -i
2023-04-04T14:57:26+0000: Falco version: 0.34.1-125+2b29ff7 (x86_64)
2023-04-04T14:57:26+0000: Falco initialized with configuration file: /etc/falco/falco.yaml
Ignored I/O syscall(s):
- recv
- pwrite64
- pread64
- sendfile64
- read
- write
- writev
- pwritev
- readv
- sendfile
- preadv
- sendto
- recvfrom
- send
- sendmsg
- recvmsg
- sendmmsg
- recvmmsg

I took these and re-generated the event file. I wasn't able to build falco master locally (ran into some build errors), so the tests are still using the old falco (which the tests are not passing) version and hence failing in my setup. Is there a way I can confirm my tests locally? I would love to know your local dev setup.

jasondellaluce · 2023-04-04T15:20:42Z

What error are you encountering when compiling Falco? Either way, you can install locally the latest dev version by downloading it from https://download.falco.org/?prefix=packages/bin-dev/. Then, once installed, the test suite should run successfully by following the readme instructions.

Either way, the tests are now just failing in the CI because your regex must be updated to match Ignored I/O syscall(s): instead of Ignored Events:. The restricted looks go to me, because from v0.35 onwards Falco will only effectively ignore IO-related system calls, whereas all the others will be activated dynamically based on the rulesets and config.

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

Rohith-Raju · 2023-04-05T07:33:53Z

Oh, I see that there was a new image 14h ago. let me retry

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

jasondellaluce · 2023-04-05T07:42:20Z

tests/data/outputs/events.txt

@@ -0,0 +1 @@
+recv,pwrite64,pread64,sendfile64,read,write,writev,pwritev,readv,sendfile,preadv,sendto,recvfrom,send,sendmsg,recvmsg,sendmmsg,recvmmsg


Suggested change

recv,pwrite64,pread64,sendfile64,read,write,writev,pwritev,readv,sendfile,preadv,sendto,recvfrom,send,sendmsg,recvmsg,sendmmsg,recvmmsg

sendfile,pwritev,preadv,writev,read,pwrite,write,sendto,recv,send,sendmsg,recvfrom,recvmsg,pread,sendmmsg,recvmmsg,readv

The latest Falco dev fixed a couple of things, this should be the most up to date and hopefully definitive list.

tests/data/outputs/events.txt

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

Rohith-Raju · 2023-04-05T08:01:36Z

tests/data/outputs/events.txt

@@ -0,0 +1 @@
+sendfile,recvfrom,readv,sendto,send,read,recvmmsg,write,recvmsg,pwrite,sendmmsg,sendmsg,pread,writev,recv,pwritev,preadv


@jasondellaluce my list is now matching with the latest list.

What error are you encountering when compiling Falco? Either way, you can install locally the latest dev version by downloading it from https://download.falco.org/?prefix=packages/bin-dev/. Then, once installed, the test suite should run successfully by following the readme instructions.

There was some deps issue but now I've solved it. I built falco master and used the same for the test and it has since been passing. I think we are good for now @jasondellaluce

jasondellaluce

/approve

Thanks a lot and congrats for your first contribution here 🎉

poiana · 2023-04-06T09:31:16Z

LGTM label has been added.

Git tree hash: 884a0c6714a4cd4aabb129d0c1fd34ac6d47e46a

poiana · 2023-04-06T09:31:17Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jasondellaluce, Rohith-Raju

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Rohith-Raju · 2023-04-06T11:44:16Z

/approve

Thanks a lot and congrats for your first contribution here 🎉

Thank you more to come 😄

Added test to print ignored events

ae3b1da

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

poiana added the dco-signoff: yes label Mar 29, 2023

poiana requested review from Andreagit97 and leogr March 29, 2023 16:02

poiana added the size/XL label Mar 29, 2023

jasondellaluce reviewed Mar 29, 2023

View reviewed changes

Rohith-Raju added 2 commits April 3, 2023 13:22

Made changes as per review requests

d465af0

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

doc changes

4cae337

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

Rohith-Raju requested a review from jasondellaluce April 3, 2023 07:55

jasondellaluce reviewed Apr 3, 2023

View reviewed changes

use embed to read the output file

506f3dd

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

poiana added size/M and removed size/XL labels Apr 3, 2023

Rohith-Raju requested a review from jasondellaluce April 3, 2023 10:52

made changes to syscall events

3813047

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

made changes to regex

5fce27d

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

made changes to events

0dfb739

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

jasondellaluce reviewed Apr 5, 2023

View reviewed changes

Rohith-Raju commented Apr 5, 2023

View reviewed changes

tests/data/outputs/events.txt Outdated Show resolved Hide resolved

final changes

b3bd0f6

Signed-off-by: Rohith-Raju <rohithraju488@gmail.com>

Rohith-Raju commented Apr 5, 2023

View reviewed changes

Rohith-Raju requested a review from jasondellaluce April 5, 2023 08:44

Rohith-Raju mentioned this pull request Apr 6, 2023

Falco cmd tests #10

Closed

3 tasks

jasondellaluce approved these changes Apr 6, 2023

View reviewed changes

poiana assigned jasondellaluce Apr 6, 2023

poiana added the lgtm label Apr 6, 2023

poiana added the approved label Apr 6, 2023

poiana merged commit 7e2011f into falcosecurity:main Apr 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test For command falco -i (ignore default events) #8

Test For command falco -i (ignore default events) #8

Rohith-Raju commented Mar 29, 2023 •

edited

Loading

poiana commented Mar 29, 2023

jasondellaluce left a comment

jasondellaluce Mar 29, 2023

Rohith-Raju Apr 3, 2023

jasondellaluce Mar 29, 2023

Rohith-Raju Apr 3, 2023

jasondellaluce Mar 29, 2023

jasondellaluce Mar 29, 2023

Rohith-Raju Apr 3, 2023

jasondellaluce Mar 29, 2023

Rohith-Raju Apr 3, 2023

Rohith-Raju commented Mar 30, 2023 •

edited

Loading

jasondellaluce Apr 3, 2023

Rohith-Raju Apr 3, 2023

jasondellaluce commented Apr 3, 2023

Rohith-Raju commented Apr 4, 2023 •

edited

Loading

jasondellaluce commented Apr 4, 2023

Rohith-Raju commented Apr 5, 2023

jasondellaluce Apr 5, 2023

Rohith-Raju Apr 5, 2023 •

edited

Loading

Rohith-Raju Apr 5, 2023 •

edited

Loading

jasondellaluce left a comment

poiana commented Apr 6, 2023

poiana commented Apr 6, 2023

Rohith-Raju commented Apr 6, 2023

		@@ -0,0 +1 @@
		recv,pwrite64,pread64,sendfile64,read,write,writev,pwritev,readv,sendfile,preadv,sendto,recvfrom,send,sendmsg,recvmsg,sendmmsg,recvmmsg

	recv,pwrite64,pread64,sendfile64,read,write,writev,pwritev,readv,sendfile,preadv,sendto,recvfrom,send,sendmsg,recvmsg,sendmmsg,recvmmsg
	sendfile,pwritev,preadv,writev,read,pwrite,write,sendto,recv,send,sendmsg,recvfrom,recvmsg,pread,sendmmsg,recvmmsg,readv

		@@ -0,0 +1 @@
		sendfile,recvfrom,readv,sendto,send,read,recvmmsg,write,recvmsg,pwrite,sendmmsg,sendmsg,pread,writev,recv,pwritev,preadv

Test For command falco -i (ignore default events) #8

Test For command falco -i (ignore default events) #8

Conversation

Rohith-Raju commented Mar 29, 2023 • edited Loading

poiana commented Mar 29, 2023

jasondellaluce left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Rohith-Raju commented Mar 30, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasondellaluce commented Apr 3, 2023

Rohith-Raju commented Apr 4, 2023 • edited Loading

jasondellaluce commented Apr 4, 2023

Rohith-Raju commented Apr 5, 2023

Choose a reason for hiding this comment

Rohith-Raju Apr 5, 2023 • edited Loading

Choose a reason for hiding this comment

Rohith-Raju Apr 5, 2023 • edited Loading

Choose a reason for hiding this comment

jasondellaluce left a comment

Choose a reason for hiding this comment

poiana commented Apr 6, 2023

poiana commented Apr 6, 2023

Rohith-Raju commented Apr 6, 2023

Rohith-Raju commented Mar 29, 2023 •

edited

Loading

Rohith-Raju commented Mar 30, 2023 •

edited

Loading

Rohith-Raju commented Apr 4, 2023 •

edited

Loading

Rohith-Raju Apr 5, 2023 •

edited

Loading

Rohith-Raju Apr 5, 2023 •

edited

Loading