-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add links.validate.config
flag
#33
Conversation
73c1990
to
27376ec
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Amazing work. I would aim however for something more flexible, so we can add other smart "validators" WDYT? (:
README.md
Outdated
@@ -63,6 +63,10 @@ Flags: | |||
--links.validate.without-address-regex=^$ | |||
If specified, all links will be validated, | |||
except those matching the given target address. | |||
--links.validate.without-github-links="" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, this is great, but I wonder if we can introduce better composability here.
E.g what if we find good mechanism for understanding GH wiki links, notifications or releases?
What if someone has website X and knows that links like a/b/c will work there as long as host is ok?
I think we should allow some flexibility for users to add those, WDYT?
I would suggest something like:
--links.validate.config="<yaml content of filepath>"
(pathorcontent
flag) with yaml like:
validate:
validators:
- regex: "(^http[s]?:\/\/)(www\.)github\.com"
type: "github"
# by default things are checked with `type: roundtrip`
.. so similar flow as with globs
and transform yaml WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that would be great! It would be better to allow users to pass in yaml config and maybe have multiple validators. Will try and implement this!
Number int `json:"number"` | ||
} | ||
|
||
func getGitHubRegex(reponame string) (*regexp.Regexp, int, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's make it a separate validator, totally separate file etc as this is complex. For readability it's best to isolate complexities from each other and abstract them away (:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure!
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
51ff48d
to
88790de
Compare
links.validate.config
flag
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, Looks quite good, some nits.
README.md
Outdated
--links.validate.config="" | ||
Path to YAML file for skipping link check, with | ||
spec defined in | ||
github.com/bwplotka/mdox/pkg/linktranformer.Config |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
github.com/bwplotka/mdox/pkg/linktranformer.Config | |
github.com/bwplotka/mdox/pkg/linktransformer.Config |
README.md
Outdated
@@ -63,6 +63,10 @@ Flags: | |||
--links.validate.without-address-regex=^$ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's kill this flag and use below one maybe?
(We can adjust flag names before first release - later it will be pain for users so let's make sure we got it right now)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure! Would be easier!
main.go
Outdated
@@ -127,6 +127,7 @@ This directive runs executable with arguments and put its stderr and stdout outp | |||
// TODO(bwplotka): Add cache in file? | |||
linksValidateEnabled := cmd.Flag("links.validate", "If true, all links will be validated").Short('l').Bool() | |||
linksValidateExceptDomains := cmd.Flag("links.validate.without-address-regex", "If specified, all links will be validated, except those matching the given target address.").Default(`^$`).Regexp() | |||
linksValidateConfig := cmd.Flag("links.validate.config", "Path to YAML file for skipping link check, with spec defined in github.com/bwplotka/mdox/pkg/linktranformer.Config").Default("").String() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
linksValidateConfig := cmd.Flag("links.validate.config", "Path to YAML file for skipping link check, with spec defined in github.com/bwplotka/mdox/pkg/linktranformer.Config").Default("").String() | |
linksValidateConfig := cmd.Flag("links.validate.config", "Path to YAML file for skipping link check, with spec defined in github.com/bwplotka/mdox/pkg/linktransformer.Config").Default("").String() |
main.go
Outdated
@@ -127,6 +127,7 @@ This directive runs executable with arguments and put its stderr and stdout outp | |||
// TODO(bwplotka): Add cache in file? | |||
linksValidateEnabled := cmd.Flag("links.validate", "If true, all links will be validated").Short('l').Bool() | |||
linksValidateExceptDomains := cmd.Flag("links.validate.without-address-regex", "If specified, all links will be validated, except those matching the given target address.").Default(`^$`).Regexp() | |||
linksValidateConfig := cmd.Flag("links.validate.config", "Path to YAML file for skipping link check, with spec defined in github.com/bwplotka/mdox/pkg/linktranformer.Config").Default("").String() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use pathorcontent
flag?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure!
type Config struct { | ||
Version int | ||
|
||
Validate struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not stright Validators
field here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was trying to match the structure in your previous review. Will change it!
|
||
type Validator struct { | ||
_regex *regexp.Regexp | ||
_maxnum int |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_maxnum int | |
_maxNum int |
return v, nil | ||
} | ||
|
||
// MustNewValidator returns mdformatter.LinkTransformer that crawls all links. | ||
func MustNewValidator(logger log.Logger, except *regexp.Regexp, anchorDir string) mdformatter.LinkTransformer { | ||
v, err := NewValidator(logger, except, anchorDir) | ||
func MustNewValidator(logger log.Logger, except *regexp.Regexp, linksValidateConfig string, anchorDir string) mdformatter.LinkTransformer { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel it would be best to get parsed Config here WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure! Would be better here!
} | ||
|
||
// Match link with any one of provided validators. | ||
func CheckValidators(dest string, v Config) bool { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
func CheckValidators(dest string, v Config) bool { | |
func (v Config).GetValidatorForURL(url string) SomeValidator interface { |
Let's have some interface for each validator so it's cleaner, WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure!
} | ||
|
||
// If type is "github", change regex and add maxnum. | ||
func CheckGitHub(v Config) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it need to be public? What check means? You mean func (v Config) validateGH() error {
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll try to implement this!
func getGitHubRegex(repoRe string) (*regexp.Regexp, int, error) { | ||
if repoRe != "" { | ||
// Get reponame from regex. | ||
idx := strings.Index(repoRe, `\`) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, it is bit confusing. What about either expecting named regex group? `(?P.*)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll try to implement this!
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot! Much better (: Some thoughts. Almost perfect!
--links.validate.without-address-regex=^$ | ||
If specified, all links will be validated, | ||
except those matching the given target address. | ||
--links.validate.config-file=<file-path> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💪🏽
// Regex for type github is reponame matcher, like `bwplotka\/mdox`. | ||
Regex string `yaml:"regex"` | ||
// By default type is `roundtrip`. Could be `github`. | ||
Type string `yaml:"type"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's make it a type ValidatorType string
and then provide 2 consts with those expected values (: It will be easier to use, at least in code level
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure! Currently, after inverting fixes, only one const is used(roundtrip
) so, commenting the second one until it is used in future.
} | ||
|
||
func ParseConfig(c []byte) (Config, error) { | ||
if string(c) == "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can surprise. Imagine someone will pass empty bytes by mistake and parse will just succeed. Can we do this on caller side?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure!
func (v Config) GetValidatorForURL(url string) URLValidator { | ||
u := URLValidator{Matched: false} | ||
for _, val := range v.Validators { | ||
if val._regex.MatchString(url) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's invert. Prefer keep the complex logic with as small indent as possible. We found invert here and continue if not matched avoiding one indent. Same below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright! Have made the same mistake in a number of places. Will rectify!
u := URLValidator{Matched: false} | ||
for _, val := range v.Validators { | ||
if val._regex.MatchString(url) { | ||
if val.Type == "github" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's use proposed const
if val.Type == "github" { | ||
// Find rightmost index of match i.e, where regex match ends. | ||
// This will be where issue/PR number starts. Split incase of section link and convert to int. | ||
idx := val._regex.FindStringIndex(url) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
idx?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This stores the rightmost index of the match, i.e gives the index of an issue/PR number. Will change the name!
// If number in link does not exceed then link is valid. Otherwise will be checked by v.c.Visit. | ||
if val._maxNum >= num && err == nil { | ||
u.Matched = true | ||
return u |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why return u if when this block ends also return u?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I seem to have missed this. Will remove!
return u | ||
} | ||
|
||
// If type is "github", change regex and add maxnum. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comments should always start with the function name.
// If type is "github", change regex and add maxnum. | ||
func (v Config) validateGH() error { | ||
for i := range v.Validators { | ||
if v.Validators[i].Type == "github" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto - invert fo readability
|
||
// Get GitHub pulls/issues regex from repo name. | ||
func getGitHubRegex(repoRe string) (*regexp.Regexp, int, error) { | ||
if repoRe != "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto and please remember about this rule. If you invert this and improve readability a lot.
Also having one big IF at the beginning is a code smell. Why invoking this function in the first place if repoRe is empty? 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure! Will keep this in mind in future PRs.
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
af2c1ce
to
246930a
Compare
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
246930a
to
c46b6e4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some comments during our 1:1, great job so far!
"gopkg.in/yaml.v3" | ||
) | ||
|
||
const roundtrip ValidatorType = "roundtrip" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's common to put this just below the ValidatorType
definition. Let's also group those into one const ( )
.
const roundtrip ValidatorType = "roundtrip" | |
const roundtripValidator ValidatorType = "roundtrip" |
err = v.validateConfig.validateGH() | ||
if err != nil { | ||
return nil, err | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
err = v.validateConfig.validateGH() | |
if err != nil { | |
return nil, err | |
} | |
return v.validateConfig.validateGH() |
if !val._regex.MatchString(url) { | ||
continue | ||
} | ||
if val.Type == roundtrip { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's use switch
since we want to have other validators soon
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Explicit might be better
} | ||
|
||
// validateGH changes regex and adds maxNum, if type is "github". | ||
func (v Config) validateGH() error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we do this during parsing? (It's tied to parsing - we don't need to it anytime else)
// validateGH changes regex and adds maxNum, if type is "github". | ||
func (v Config) validateGH() error { | ||
for i := range v.Validators { | ||
if v.Validators[i].Type == roundtrip { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is where we can do explicit switch
as I mentioned in Parse
because we want to error out if user specifies something unknown
) | ||
|
||
// PathOrContent is a flag type that defines two flags to fetch bytes. Either from file (*-file flag) or content (* flag). | ||
type PathOrContent struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add env substitution here, and then we can think about proposing this upstream.
So the idea would be to add env substitution using e.g Kubernetes
format, which is $(...)
and document it.
e.g our config could be Token: $(GITHUB_TOKEN)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Then we can upstream it e.g through this: efficientgo/tools#9
} | ||
|
||
// GetValidatorForURL matches link with any one of provided validators. | ||
func (v Config) GetValidatorForURL(url string) URLValidator { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to work on different abstrations here...
What about something else:
type Validator interface {
IsValid(URL string) (bool, error)
}
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That looks solid to me (:
pkg/extflag/pathorcontent.go
Outdated
} | ||
|
||
// RegisterPathOrContent registers PathOrContent flag in kingpinCmdClause. | ||
func RegisterPathOrContent(cmd FlagClause, flagName string, help string, required bool, envSubstitution bool) *PathOrContent { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Time for variadic options here (:
func RegisterPathOrContent(cmd FlagClause, flagName string, help string, required bool, envSubstitution bool) *PathOrContent { | |
func RegisterPathOrContent(cmd FlagClause, flagName string, help string, opts ...Option) *PathOrContent { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure!
pkg/extflag/pathorcontent.go
Outdated
// substituteEnvVars returns content of YAML file with substituted environment variables. | ||
// Will be substituted with empty string if env var isn't set. | ||
// Follows K8s convention, i.e $(...), as mentioned here https://kubernetes.io/docs/tasks/inject-data-application/define-interdependent-environment-variables/. | ||
func substituteEnvVars(content string) []byte { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's take in bytes if we want to split out bytes (consistency)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we reuse exsiting code? https://github.com/thanos-io/thanos/blob/7d118f416be98cf502b9b5e0df4e98b37478f3f3/pkg/reloader/reloader.go#L419
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright! Will try to reuse code from here!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have taken existing code but used os.Getenv
instead of os.LookupEnv
since Token
here is optional and don't want to error out if variable is unset. 🙂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why we don't want to error out? I think that is a correct behaviour
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The GITHUB_TOKEN
variable is only expected to be set in CI runners so that authenticated API calls are made and it's not rate limited(Same IP across macos runners problem). But if user decides not to set the token i.e, isn't getting rate limited and doesn't need to make auth'd requests then there shouldn't be any error I think. So should I put the error back in?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But if user decides not to set the token i.e, isn't getting rate limited and doesn't need to make auth'd requests then there shouldn't be any error I think.
If user decided not to set this token, why she/he would add this to yaml as $(....)
construct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If they wanted to run CI job locally but then they can have a make-docs-local command as well. I see what you mean, I'll add error back in! 🙂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
explicit configuration is better than surprising ones
e5634b3
to
98e791a
Compare
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
98e791a
to
3282c03
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot! Super close to merge
pkg/extflag/pathorcontent.go
Outdated
} | ||
|
||
// WithRequired allows you to override default envSubstitution option. | ||
func WithEnvSubstitution(e bool) Option { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
func WithEnvSubstitution(e bool) Option { | |
func WithEnvSubstitution() Option { |
pkg/extflag/pathorcontent.go
Outdated
} | ||
|
||
// WithRequired allows you to override default required option. | ||
func WithRequired(r bool) Option { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
func WithRequired(r bool) Option { | |
func WithRequired() Option { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't need to expose both bools, as we have by default false, so we need to have just method for true, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. Will change this!
} | ||
|
||
// RoundTripValidator.IsValid returns false if url matches, to ensure it is visited by colly. | ||
func (v RoundTripValidator) IsValid(URL string) (bool, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This still needs some love 🙈
"RountTripValidator says URL is invalid" - that does not sounds like this would mean it will be check by colly. I would rather put colly logic directly here into this method, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright, will put this logic directly! 🙂
if !val.rtValidator._regex.MatchString(URL) { | ||
continue | ||
} | ||
u = val.rtValidator |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need u
variable, we can just return, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure! I missed this, will rectify!
} | ||
} | ||
// By default all links are ignored. | ||
u = IgnoreValidator{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm... I don't think, I think we want to round trip rest?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I might've misunderstood earlier. Will change!
a9b740a
to
d3d8606
Compare
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
d3d8606
to
d1c885d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! LGTM (: Two issues, but it's ok to fix in later PRs 💪🏽
r.rMu.Lock() | ||
defer r.rMu.Unlock() | ||
// We need to check again here to avoid race. | ||
if _, ok := r.remoteLinks[k.dest]; ok { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is some sort of "caching" that we can use on TOP of validators, so other validators can leverage it no? It's ok to think about it later PRs (:
} | ||
return val.igValidator | ||
default: | ||
panic("unexpected validator type") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
panic("unexpected validator type") | |
panic(fmt.Sprintf("unexpected validator type %v", val.Type)) |
This PR adds in the
--links.validate.config
flag which allows users to pass in path to YAML config with composable regex matchers.A sample yaml config would be,
Here, all links are checked with default type
ignore
, i.e, links are not visited. If type is set toroundtrip
, then links matching provided regex are visited by colly. For typegithub
only repo name regex is taken, which then directs mdox to not visit any GitHub issue/PR link of that repo. User can also provide a token(env var substitution) to tackle GitHub API rate limiting.Also, we don't check if a particular number is an issue or a PR since GitHub can automatically redirect such links to the correct page, for example the link
https://github.com/bwplotka/mdox/pull/30
is actually an issue and will be redirected. So such links can be safely skipped.