Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump go to 1.20 #5027

Merged
merged 1 commit into from
Apr 18, 2024
Merged

Bump go to 1.20 #5027

merged 1 commit into from
Apr 18, 2024

Conversation

siddarthkay
Copy link
Contributor

@siddarthkay siddarthkay commented Apr 8, 2024

Summary

This PR attempts to upgrade go version to 1.20
This PR also removes the following items from lint checks :

  • goconst
  • structcheck
  • deadcode
  • golint
  • varcheck

Mobile PR for QA purposes -> status-im/status-mobile#19564

Todo

@siddarthkay siddarthkay self-assigned this Apr 8, 2024
@siddarthkay siddarthkay marked this pull request as draft April 8, 2024 17:56
@status-im-auto
Copy link
Member

status-im-auto commented Apr 8, 2024

Jenkins Builds

Click to see older builds (115)
Commit #️⃣ Finished (UTC) Duration Platform Result
✖️ 9a39cf1 #1 2024-04-08 17:59:32 ~2 min tests 📄log
✔️ 9a39cf1 #1 2024-04-08 18:00:56 ~4 min linux 📦zip
✔️ 9a39cf1 #1 2024-04-08 18:01:59 ~5 min ios 📦zip
✔️ 9a39cf1 #1 2024-04-08 18:02:37 ~5 min android 📦aar
✔️ 199eed7 #2 2024-04-08 18:10:13 ~1 min android 📦aar
✔️ 199eed7 #2 2024-04-08 18:11:46 ~3 min ios 📦zip
✔️ 199eed7 #2 2024-04-08 18:12:41 ~4 min linux 📦zip
✔️ 199eed7 #2 2024-04-08 18:48:19 ~39 min tests 📄log
✔️ 3150f35 #3 2024-04-09 07:13:59 ~2 min linux 📦zip
✔️ 3150f35 #3 2024-04-09 07:14:36 ~3 min android 📦aar
✔️ 3150f35 #3 2024-04-09 07:15:15 ~3 min ios 📦zip
✔️ 3150f35 #3 2024-04-09 07:51:27 ~40 min tests 📄log
✔️ b2c7b72 #4 2024-04-09 07:24:08 ~1 min android 📦aar
✔️ b2c7b72 #4 2024-04-09 07:24:48 ~2 min linux 📦zip
✔️ b2c7b72 #4 2024-04-09 07:25:37 ~3 min ios 📦zip
✔️ b2c7b72 #4 2024-04-09 08:30:24 ~38 min tests 📄log
✔️ 923d124 #5 2024-04-09 09:17:53 ~2 min android 📦aar
✔️ 923d124 #5 2024-04-09 09:18:00 ~2 min linux 📦zip
✔️ 923d124 #5 2024-04-09 09:19:33 ~4 min ios 📦zip
✔️ 923d124 #5 2024-04-09 09:58:02 ~42 min tests 📄log
✔️ 7fcf11b #6 2024-04-09 09:28:06 ~2 min android 📦aar
✔️ 7fcf11b #6 2024-04-09 09:29:16 ~3 min ios 📦zip
✔️ 7fcf11b #6 2024-04-09 09:31:02 ~5 min linux 📦zip
✔️ 7fcf11b #6 2024-04-09 10:37:50 ~39 min tests 📄log
✔️ 6d28f90 #7 2024-04-09 11:22:47 ~3 min android 📦aar
✔️ 6d28f90 #7 2024-04-09 11:25:12 ~5 min linux 📦zip
✔️ 6d28f90 #7 2024-04-09 12:00:41 ~40 min tests 📄log
✔️ 3b8f754 #8 2024-04-09 11:43:48 ~1 min android 📦aar
✔️ 3b8f754 #8 2024-04-09 11:45:40 ~3 min linux 📦zip
✔️ 3b8f754 #8 2024-04-09 12:38:50 ~37 min tests 📄log
✔️ ddff9e0 #9 2024-04-09 15:18:54 ~2 min android 📦aar
✔️ ddff9e0 #9 2024-04-09 15:19:11 ~2 min linux 📦zip
✔️ ddff9e0 #9 2024-04-09 15:23:26 ~6 min ios 📦zip
✔️ ddff9e0 #9 2024-04-09 15:56:03 ~39 min tests 📄log
✖️ caaa3ed #10 2024-04-11 06:02:54 ~1 min tests 📄log
✔️ caaa3ed #10 2024-04-11 06:04:29 ~2 min linux 📦zip
✔️ caaa3ed #10 2024-04-11 06:05:44 ~4 min android 📦aar
✔️ caaa3ed #10 2024-04-11 06:06:34 ~5 min ios 📦zip
✔️ 88c8b2e #11 2024-04-11 06:07:45 ~2 min linux 📦zip
✔️ 88c8b2e #11 2024-04-11 06:08:03 ~1 min android 📦aar
✔️ 88c8b2e #11 2024-04-11 06:10:11 ~3 min ios 📦zip
✔️ 88c8b2e #12 2024-04-11 07:23:36 ~39 min tests 📄log
✔️ a474965 #11 2024-04-11 06:44:12 ~40 min tests 📄log
✔️ 0e46202 #12 2024-04-12 07:33:06 ~2 min linux 📦zip
✔️ 0e46202 #12 2024-04-12 07:33:13 ~2 min android 📦aar
✔️ 0e46202 #12 2024-04-12 07:34:29 ~3 min ios 📦zip
✔️ 0e46202 #13 2024-04-12 08:10:58 ~40 min tests 📄log
✖️ 3288d38 #14 2024-04-13 11:25:29 ~1 min tests 📄log
✔️ 3288d38 #13 2024-04-13 11:26:06 ~2 min android 📦aar
✔️ 3288d38 #13 2024-04-13 11:26:16 ~2 min ios 📦zip
✔️ 3288d38 #13 2024-04-13 11:27:01 ~3 min linux 📦zip
✖️ 70603a0 #15 2024-04-16 10:33:12 ~1 min tests 📄log
✔️ 70603a0 #14 2024-04-16 10:33:20 ~1 min ios 📦zip
✔️ 70603a0 #14 2024-04-16 10:35:44 ~3 min linux 📦zip
✔️ 70603a0 #14 2024-04-16 10:36:32 ~4 min android 📦aar
✖️ 0212f93 #16 2024-04-16 10:34:09 ~44 sec tests 📄log
✔️ 0212f93 #15 2024-04-16 10:35:30 ~1 min ios 📦zip
✖️ 473bb92 #17 2024-04-16 10:35:28 ~42 sec tests 📄log
✔️ 473bb92 #15 2024-04-16 10:38:49 ~2 min linux 📦zip
✔️ 473bb92 #16 2024-04-16 10:39:53 ~3 min ios 📦zip
✔️ 473bb92 #15 2024-04-16 10:42:34 ~5 min android 📦aar
✖️ cc89910 #18 2024-04-16 10:38:36 ~1 min tests 📄log
✔️ cc89910 #16 2024-04-16 10:41:42 ~2 min linux 📦zip
✔️ cc89910 #17 2024-04-16 10:43:47 ~3 min ios 📦zip
✔️ 3f3bd6c #17 2024-04-16 10:44:21 ~2 min linux 📦zip
✔️ 3f3bd6c #16 2024-04-16 10:45:39 ~2 min android 📦aar
✔️ 3f3bd6c #18 2024-04-16 10:47:28 ~3 min ios 📦zip
✖️ 3f3bd6c #19 2024-04-16 10:47:40 ~6 min tests 📄log
✔️ 2695e61 #17 2024-04-16 10:54:11 ~1 min android 📦aar
✔️ 2695e61 #18 2024-04-16 10:54:55 ~2 min linux 📦zip
✔️ 2695e61 #19 2024-04-16 10:56:30 ~3 min ios 📦zip
✖️ 2695e61 #20 2024-04-16 10:58:35 ~6 min tests 📄log
✔️ c7bbecd #18 2024-04-16 11:07:13 ~2 min android 📦aar
✔️ c7bbecd #19 2024-04-16 11:07:34 ~2 min linux 📦zip
✔️ c7bbecd #20 2024-04-16 11:08:38 ~3 min ios 📦zip
✔️ c7bbecd #21 2024-04-16 11:43:42 ~38 min tests 📄log
✔️ 4ca3435 #20 2024-04-17 14:55:57 ~2 min linux 📦zip
✔️ 4ca3435 #19 2024-04-17 14:56:12 ~3 min android 📦aar
✔️ 4ca3435 #21 2024-04-17 14:57:04 ~3 min ios 📦zip
✔️ 4ca3435 #22 2024-04-17 15:33:27 ~40 min tests 📄log
✔️ 6dfb180 #21 2024-04-17 14:58:43 ~2 min linux 📦zip
✔️ 6dfb180 #20 2024-04-17 14:58:46 ~2 min android 📦aar
✔️ cc57586 #21 2024-04-17 15:00:54 ~1 min android 📦aar
✔️ cc57586 #22 2024-04-17 15:01:09 ~2 min linux 📦zip
✔️ cc57586 #22 2024-04-17 15:01:29 ~3 min ios 📦zip
✔️ cc57586 #23 2024-04-17 15:05:13 ~3 min ios 📦zip
✔️ cc57586 #23 2024-04-17 16:13:33 ~39 min tests 📄log
✔️ 0b81ffd #22 2024-04-18 07:51:15 ~2 min android 📦aar
✔️ 0b81ffd #23 2024-04-18 07:51:38 ~2 min linux 📦zip
✔️ 0b81ffd #24 2024-04-18 07:52:30 ~3 min ios 📦zip
✔️ 0b81ffd #24 2024-04-18 08:29:05 ~39 min tests 📄log
✔️ 71d8aa5 #23 2024-04-18 08:08:57 ~1 min android 📦aar
✔️ 71d8aa5 #24 2024-04-18 08:09:25 ~2 min linux 📦zip
✔️ 71d8aa5 #25 2024-04-18 08:14:05 ~7 min ios 📦zip
✔️ 06096b8 #24 2024-04-18 08:28:52 ~2 min android 📦aar
✔️ 06096b8 #25 2024-04-18 08:29:42 ~2 min linux 📦zip
✔️ 06096b8 #26 2024-04-18 08:30:47 ~4 min ios 📦zip
✔️ 06096b8 #25 2024-04-18 09:09:49 ~40 min tests 📄log
✔️ 7955d76 #25 2024-04-18 09:01:53 ~2 min android 📦aar
✔️ 94c19e8 #27 2024-04-18 09:02:22 ~2 min linux 📦zip
✔️ 94c19e8 #26 2024-04-18 09:06:21 ~4 min android 📦aar
✔️ 94c19e8 #28 2024-04-18 09:07:49 ~8 min ios 📦zip
✖️ 94c19e8 #26 2024-04-18 09:15:28 ~5 min tests 📄log
✔️ e133495 #28 2024-04-18 11:06:37 ~2 min linux 📦zip
✔️ e133495 #27 2024-04-18 11:06:59 ~3 min android 📦aar
✔️ e133495 #29 2024-04-18 11:07:39 ~3 min ios 📦zip
✔️ e133495 #27 2024-04-18 11:46:24 ~42 min tests 📄log
✔️ 3255da8 #28 2024-04-18 12:16:56 ~2 min android 📦aar
✔️ 3255da8 #29 2024-04-18 12:17:07 ~2 min linux 📦zip
✔️ 3255da8 #30 2024-04-18 12:18:04 ~3 min ios 📦zip
✔️ 3255da8 #28 2024-04-18 12:55:29 ~40 min tests 📄log
✔️ 7a1af10 #29 2024-04-18 14:43:06 ~2 min android 📦aar
✔️ 7a1af10 #30 2024-04-18 14:43:16 ~2 min linux 📦zip
✔️ 7a1af10 #31 2024-04-18 14:56:10 ~15 min ios 📦zip
✔️ 7a1af10 #29 2024-04-18 15:23:06 ~42 min tests 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
ef96e22 #32 2024-04-18 15:11:09 ~59 sec ios 📄log
✔️ ef96e22 #31 2024-04-18 15:12:34 ~2 min linux 📦zip
✔️ ef96e22 #30 2024-04-18 15:12:38 ~2 min android 📦aar
✔️ ef96e22 #30 2024-04-18 16:03:19 ~39 min tests 📄log
✔️ f521530 #32 2024-04-18 16:01:44 ~2 min linux 📦zip
✔️ f521530 #31 2024-04-18 16:01:51 ~2 min android 📦aar
✔️ f521530 #33 2024-04-18 16:03:06 ~3 min ios 📦zip
✔️ f521530 #31 2024-04-18 16:43:20 ~39 min tests 📄log

@siddarthkay siddarthkay marked this pull request as ready for review April 8, 2024 18:15
@siddarthkay siddarthkay changed the title [WIP] Bump go to 1.20 Bump go to 1.20 Apr 8, 2024
@Samyoul
Copy link
Member

Samyoul commented Apr 8, 2024

Hey @siddarthkay! Thank you for your PR and your great work upgrading basically everything. Just to give my opinion, I'm uneasy about dropping gosec.

Other two linter checks I'm not too bothered about, they do help keep the code idiomatic though, particularly structcheck, it enforces struct usage in the same way the compiler treats var and import usage. I'm least attached to goconst, I don't mind repeated string literals.

@siddarthkay
Copy link
Contributor Author

siddarthkay commented Apr 9, 2024

make lint gives these warnings so it seems safe to remove these from enabled section of .golangci.yml

WARN [runner] The linter 'deadcode' is deprecated (since v1.49.0) due to: The owner seems to have 
abandoned the linter. Replaced by unused.
WARN [runner] The linter 'golint' is deprecated (since v1.41.0) due to: The repository of the linter has 
been archived by the owner. Replaced by revive.
WARN [runner] The linter 'varcheck' is deprecated (since v1.49.0) due to: The owner seems to have 
abandoned the linter. Replaced by unused.

@siddarthkay
Copy link
Contributor Author

siddarthkay commented Apr 9, 2024

I enabled gosec to see the offending code and to fix them and following are the issues :

protocol/communities/persistence.go:963:11: G202: SQL string concatenation (gosec)
	query := "SELECT sig, timestamp, topic, payload, padding, hash, third_party_id FROM 
	waku_messages WHERE timestamp >= " + fmt.Sprint(from) + " AND timestamp < " + fmt.Sprint(to) +
	 " AND ("
	         ^
transactions/testhelpers.go:147:12: G601: Implicit memory aliasing in for loop. (gosec)
					tx := &sum.tx
					      ^
transactions/testhelpers.go:169:12: G601: Implicit memory aliasing in for loop. (gosec)
					tx := &sum.tx
					      ^
transactions/transactor_test.go:303:12: G601: Implicit memory aliasing in for loop. (gosec)
				Return(&scenario.nonceFromNetwork, nil)
				       ^
services/wallet/transfer/block_ranges_sequential_dao.go:118:11: G202: SQL string concatenation (gosec)
	query := "SELECT address, blk_start, blk_first, blk_last, token_blk_start, token_blk_first, 
	token_blk_last, balance_check_hash FROM blocks_ranges_sequential WHERE address IN (" +
		addressesPlaceholder + ") AND network_id = ?"
		
protocol/activity_center_persistence.go:856:12: G202: SQL string concatenation (gosec)
	update := "UPDATE activity_center_notifications SET deleted = 1, updated_at = ? WHERE id IN (" + 
	inVector + ") AND NOT deleted"
	          ^
multiaccounts/accounts/keycard_database.go:266:11: G202: SQL string concatenation (gosec)
	query := `
		DELETE
		FROM
			keycards_accounts
		WHERE
			keycard_uid = ?
		AND
			account_address	IN (?` + inVector + `)`
			
protocol/message_persistence.go:3019:9: G202: SQL string concatenation (gosec)
	sql := "UPDATE user_messages SET response_to = ? WHERE id IN 
	(?" + strings.Repeat(",?", len(statusMessagesToUpdate)-1) + ")"
	       ^
	       
make: *** [lint] Error 1

The key offenders are :

  • G202: SQL string concatenation
  • G601: Implicit memory aliasing in for loop.

@siddarthkay
Copy link
Contributor Author

Regarding G202: SQL string concatenation -> https://securego.io/docs/rules/g201-g202#the-right-way
It seems that rule exists to avoid SQL injection.
Which is nice, However we rarely take in any input from the user to generate these SQL statements so it seems safe to skip the rule in places by adding a // nolint: gosec
I do see presence of // nolint: gosec in many places .. 121 times :D

@siddarthkay
Copy link
Contributor Author

next up is G601: Implicit memory aliasing in for loop

An easy fix for such cases is defining a local variable for memory references in for loop and use that variable ahead.
Only drawback of such an approach is weird looking code like this -> 3150f35

We could also exclude just G601 and still keep other rules of gosec.

which approach do you prefer @Samyoul ?

@stefandunca
Copy link
Contributor

stefandunca commented Apr 9, 2024

...
We could also exclude just G601 and still keep other rules of gosec.

There were few ugly bugs due to ignoring this check, mainly trying to modify a temporary variable instead of the original data. I think it is better to keep the check and fix the aliasing using temporary variables or index until we upgrade to 1.22 when this "optimization" is finally fixed

@siddarthkay
Copy link
Contributor Author

Thanks for weighing in @stefandunca, just to confirm, the changes in this commit -> 3150f35 seem good to you ?

@stefandunca
Copy link
Contributor

Thanks for weighing in @stefandunca, just to confirm, the changes in this commit -> 3150f35 seem good to you ?

They seems right if you need a quick fix and don't have time to look if the G202 can be fixed instead of ignoring. I added some comments for minor improvements and to add a follow up tasks for G202. However, please note that Run from tests can be parallelized and the aliasing become a real issue, therefore the G601 being a serious thing.

@siddarthkay
Copy link
Contributor Author

siddarthkay commented Apr 9, 2024

Thanks @stefandunca : reversing the alias is a good idea, I'll push a commit and fix that.
Regarding G202: SQL string concatenation : we could modify those SQL queries to make sure we don't generate them via string concatenation via + or fmt.sprintf and instead pass the correct parameters as suggested in https://securego.io/docs/rules/g201-g202#the-right-way
I'll make a new issue for it, as I believe its better to do in a separate PR and would require some extensive QA since modifying queries could potentially break certain functionalities if we miss something.

@siddarthkay
Copy link
Contributor Author

issue created to track fixing G202: SQL string concatenation -> #5033

Copy link
Collaborator

@igor-sirotin igor-sirotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! 🎉

@siddarthkay siddarthkay marked this pull request as draft April 16, 2024 09:52
@siddarthkay siddarthkay force-pushed the bump-go-to-1-20 branch 7 times, most recently from 6dfb180 to cc57586 Compare April 17, 2024 14:57
@siddarthkay siddarthkay marked this pull request as ready for review April 17, 2024 14:58
@siddarthkay
Copy link
Contributor Author

@igor-sirotin : don't worry we found a dumb fix for the storenode problem :)

@siddarthkay siddarthkay force-pushed the bump-go-to-1-20 branch 7 times, most recently from 3255da8 to 7a1af10 Compare April 18, 2024 14:40
vendor/modules.txt Outdated Show resolved Hide resolved
This commit attempts to upgrade go version to 1.20.12
This commit also removes the following items from lint checks :
* `goconst`
* `structcheck`
* `deadcode`
* `golint`
* `varcheck`

Mobile PR for QA purposes -> status-im/status-mobile#19564
@siddarthkay siddarthkay merged commit d6c4682 into develop Apr 18, 2024
7 checks passed
@siddarthkay siddarthkay deleted the bump-go-to-1-20 branch April 18, 2024 16:48
siddarthkay added a commit to status-im/status-mobile that referenced this pull request Apr 18, 2024
## Summary

This commit also points to status-go branch where we have upgraded go to 1.20
Related statusgo PR -> status-im/status-go#5027

### Testing notes
Please test everything, specially the store node stuff.

#### Platforms
- Android
- iOS

status: ready
siddarthkay added a commit to status-im/status-mobile that referenced this pull request Apr 18, 2024
status-im/status-go@a39c01d...5e9f961

## Summary

This commit also points to status-go branch where we have upgraded go to 1.20
Related statusgo PR -> status-im/status-go#5027

### Testing notes
Please test everything, specially the store node stuff.

#### Platforms
- Android
- iOS

status: ready
siddarthkay added a commit to status-im/status-mobile that referenced this pull request Apr 18, 2024
## Summary

This commit also points to status-go branch where we have upgraded go to 1.20
Related status-go PR -> status-im/status-go#5027

### Testing notes
Please test everything, specially the store node stuff.

#### Platforms
- Android
- iOS

status: ready
@@ -27,6 +28,8 @@ const defaultBackoff = 10 * time.Second
const graylistBackoff = 3 * time.Minute
const isAndroidEmulator = runtime.GOOS == "android" && runtime.GOARCH == "amd64"
const findNearestMailServer = !isAndroidEmulator
const overrideDNS = runtime.GOOS == "android"
const bootstrapDNS = "8.8.8.8:53"
Copy link
Member

@jakubgs jakubgs May 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this. We should at least leave a FIXME comment explaining that this should be changed.

It's a bard practice because:

  • It's bad in terms of preserving privacy of our users.
  • It doesn't respect user DNS settings on device or in their local network.
  • It unreliable since it depends on only on server.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry I missed adding the FIXME.. there is an issue open for this DNS entry, we also do the same thing for waku nodes in mobile config -> #3024

Copy link
Member

@jakubgs jakubgs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants