[needs discussion] Use decred.org in package paths

[jrick]

Go package paths commonly include a domain so that tools such as 'go get' are able to automatically find and download the source code. Currently all of Decred's Go projects use package paths beginning with github.com/decred/. This means that when performing an operation such as go get github.com/decred/dcrd (ignoring for now this is not the correct way to build dcrd), go will query github.com directly for the source code.

Go has a feature to use custom package paths with any domain while maintaining the source code repository elsewhere. This would allow us to modify all package paths to begin with decred.org while still using Github for the issue tracker, pull requests, and hosting the git repository.

The decred.org web server has already been updated to support this feature. For example:

PS C:\Users\jrick> go get -v decred.org/dcrd
Fetching https://decred.org/dcrd?go-get=1
Parsing meta tags from https://decred.org/dcrd?go-get=1 (status code 200)
get "decred.org/dcrd": found meta tag get.metaImport{Prefix:"decred.org/dcrd", VCS:"git", RepoRoot:"https://github.com/decred/dcrd.git"} at https://decred.org/dcrd?go-get=1
decred.org/dcrd (download)
decred.org/dcrd
# decred.org/dcrd
go\src\decred.org\dcrd\rpcserver.go:3182:18: chain.LocateHeaders undefined (type *blockchain.BlockChain has no field or method LocateHeaders)
go\src\decred.org\dcrd\server.go:348:14: addrManager.SetServices undefined (type *addrmgr.AddrManager has no field or method SetServices)
go\src\decred.org\dcrd\server.go:734:19: chain.LocateBlocks undefined (type *blockchain.BlockChain has no field or method LocateBlocks)
go\src\decred.org\dcrd\server.go:775:18: chain.LocateHeaders undefined (type *blockchain.BlockChain has no field or method LocateHeaders)
go\src\decred.org\dcrd\server.go:1647:13: cannot use sp.OnVersion (type func(*peer.Peer, *wire.MsgVersion) *wire.MsgReject) as type func(*peer.Peer, *wire.MsgVersion) in field value

The project does not compile using this command because this is not the correct way to build dcrd, but it will be in the future as vgo is integrated into the official toolchain. What is important is that decred.org was queried for the source code location and redirected to github.com for the actual location of the git repo to use.

The primary advantage of this change would be the ability to move away from Github or any other code hosting provider in the future without the churn of renaming all packages again, while still using Github in the short (or long) term if there was no reason to switch. This is our exit strategy so to speak, in case we needed to move away from Github at a moment's notice without disrupting the project too significantly.

If packages are renamed, we should also consider annotating all package statements with the canonical package path. This prevents the same code from being referenced by two different names, which are in fact two distinct and incompatible packages according to the Go toolchain.

[chappjc]

I love this in general, but security of the decred.org web server becomes even more important. If the server returns a meta tag redirecting to a malicious VCS system, bad news.

Is there any mitigation like verifying signed commits or something similar?

[jrick]

No, and vgo will actually prefer to fetch a module bundle (zip file) instead of using git directly (this was the original vgo default but was disabled for github.com recently due to unauthorized API rate limiting).

I understand the concerns regarding the security of the decred.org domain, but don't see this being any worse than the current situation where we don't control github.com.

[chappjc]

It's worse because it's more attack surface. We'd still be redirecting to GitHub, at least for now, so it's adding one more security concern, not trading one.

[jrick]

If the security of multiple servers, and especially servers we don't control, is a concern, I believe we should go entirely self hosted. This would be the first step towards doing so.

[ghost]

I also started thinking a bit about this during all of these events unfolding with GitHub. This is a really great idea. Like you previously discussed, including the commit hashes of releases on the blockchain and then maybe emigrate over to self hosting somehow would be a very interesting project.

[dajohi]

I vote for it.

[jrick]

See decred/dcrwallet#1538 which performs this conversion for the dcrwallet main module.

[davecgh]

I'm on board with this idea, however, several things in the main module need to be split out and made internal first in order to avoid causing massive issues due to semver breaks in code that is actually treated as internal code.