Add config file for gh-jira sync

[keirthana]

Adding a preliminary config file will allow the gh-jira-sync-bot to set up a sync between the GitHub repo and Jira. It can further be configured depending on the level of synchronisation we need.

Once this is added, I can get the server side configuration done with Maksim's help.

[pmatulis]

@keirthana Can you provide details on usage and benefits? What will users be able to do once this is implemented - open issues from Jira? Anything else?

[keirthana]

(Sorry I clicked the wrong button earlier and closed the PR and then reopened it)

@pmatulis Ruth had a concern about having issues on GitHub and hence the work was already being tracked there. When we use Jira, ideally she is required to duplicate the same stuff on Jira for tracking which is not a good use of her or anyone's time. So when this is implemented, the issues on GitHub should be synced to Jira automatically.

[ru-fu]

I generally like the idea and would like to test this out - however, I'm not sure if it's a good idea to do this on the starter pack.

The idea of the starter pack is that others can just use it as it is and set it up. Adding this kind of GitHub action means that they will also copy this action - so the minimum we'd need to do is to add instructions to the README to edit or remove this file for a doc repo. Otherwise, we'll get doc PRs for all repos that use the starter pack. ;)

I guess the first question to answer is whether we want to recommend using this sync for all documentation that uses the starter pack?

[keirthana]

I generally like the idea and would like to test this out - however, I'm not sure if it's a good idea to do this on the starter pack.

The idea of the starter pack is that others can just use it as it is and set it up. Adding this kind of GitHub action means that they will also copy this action - so the minimum we'd need to do is to add instructions to the README to edit or remove this file for a doc repo. Otherwise, we'll get doc PRs for all repos that use the starter pack. ;)

I agree, we should add more instructions to the README.

I guess the first question to answer is whether we want to recommend using this sync for all documentation that uses the starter pack?

I'd say it is a useful addition but let's discuss in the team catchup once to know everyone's opinion.

[s-makin]

I guess the first question to answer is whether we want to recommend using this sync for all documentation that uses the starter pack?

Is it possible to add it (or at least, instructions for it) as an optional extra? I think the key point about the starter pack being a starter pack is that it should be fairly lightweight and minimal. The most common complaint I was hearing about it was that it's hard to adopt because it's got a lot of extras in it that should be optional and not required. We should rationalise features like this to make sure we're not adding an undue burden on teams that want to adopt it, especially if we want to encourage people to track the repo in the future to stay up to date.

I can definitely see the use in this functionality, but one thing I'm not sure about if we add it as default for every repo is whether it would be useful for teams who get a lot of community contributions. Since GH (and not Jira) is the interface between teams and their community, we should document i.e. what the expected behaviour is (whether someone submitting an issue has access to Jira or not), and how to customise the different options in this action. What happens if an issue is closed as a duplicate, for example? Does the Jira issue disappear? Does it only create the Jira ticket, or does it track updates to the GH issue and add them? If you manually change the Jira ticket, is the GH issue updated?

From a roadmap planning perspective as well, having GH issues auto-sync to Jira needs to be curated in some way. How can we tell if an issue is urgent and should be prioritised, vs something that can be added to the next cycle's roadmap candidates? Would a team using this feature need to use Jira in a specific way? It's one thing for small projects to use this for tracking, but for teams that take a lot of issues on GH, Jira could easily become overwhelmed with unaddressed issues, which might lead to additional overheads (particularly if there are duplicates) in managing the auto-populated issues manually. Definitely think it's worth bikeshedding on exactly how this should work in our next TA meeting!

[ru-fu]

Is it possible to add it (or at least, instructions for it) as an optional extra?

If we want to use it for the starter pack, it will be included for everyone who uses the starter pack. If we make it optional to use, it will be optional (and thus turned off by default) for the starter pack as well. :(

In general, for each feature that I add, I decide if it's something I would recommend everyone to use. If yes, it will be enabled, otherwise it will be optional. (With the exception of some extensions that are always included but won't be visible in any way until someone actually makes use of them.)

[keirthana]

Is it possible to add it (or at least, instructions for it) as an optional extra?

If we want to use it for the starter pack, it will be included for everyone who uses the starter pack. If we make it optional to use, it will be optional (and thus turned off by default) for the starter pack as well. :(

So you mean to say that we can't provide it as optional (hence turned off by default) but just have it enabled manually for the starter pack repository?

In general, for each feature that I add, I decide if it's something I would recommend everyone to use. If yes, it will be enabled, otherwise it will be optional. (With the exception of some extensions that are always included but won't be visible in any way until someone actually makes use of them.)

[keirthana]

I guess the first question to answer is whether we want to recommend using this sync for all documentation that uses the starter pack?

Is it possible to add it (or at least, instructions for it) as an optional extra? I think the key point about the starter pack being a starter pack is that it should be fairly lightweight and minimal. The most common complaint I was hearing about it was that it's hard to adopt because it's got a lot of extras in it that should be optional and not required. We should rationalise features like this to make sure we're not adding an undue burden on teams that want to adopt it, especially if we want to encourage people to track the repo in the future to stay up to date.

I can definitely see the use in this functionality, but one thing I'm not sure about if we add it as default for every repo is whether it would be useful for teams who get a lot of community contributions. Since GH (and not Jira) is the interface between teams and their community, we should document i.e. what the expected behaviour is (whether someone submitting an issue has access to Jira or not), and how to customise the different options in this action. What happens if an issue is closed as a duplicate, for example? Does the Jira issue disappear? Does it only create the Jira ticket, or does it track updates to the GH issue and add them? If you manually change the Jira ticket, is the GH issue updated?

From a roadmap planning perspective as well, having GH issues auto-sync to Jira needs to be curated in some way. How can we tell if an issue is urgent and should be prioritised, vs something that can be added to the next cycle's roadmap candidates? Would a team using this feature need to use Jira in a specific way? It's one thing for small projects to use this for tracking, but for teams that take a lot of issues on GH, Jira could easily become overwhelmed with unaddressed issues, which might lead to additional overheads (particularly if there are duplicates) in managing the auto-populated issues manually. Definitely think it's worth bikeshedding on exactly how this should work in our next TA meeting!

Very valid points! We need to test this out to see how the issues appear in Jira to be able to answer some of these questions. I did not get a lot of information from Maksim on how this would actually appear on Jira except that for GitHub issues to sync to Jira, we need this config file in place before he could enable that on the Jira board. Maybe testing this on a separate test branch of the repo without affect the main branch could help?

[pmatulis]

I agree that we need to have real-world testing done before merging this.

[ru-fu]

So you mean to say that we can't provide it as optional (hence turned off by default) but just have it enabled manually for the starter pack repository?

Correct. The configuration that we pick for the starter pack is the default configuration for anybody that starts using the starter pack.

[keirthana]

After the discussions in one of the author's catchup and a demo session with Ruth, submitting an updated PR with the config file and updates to readme.

[ru-fu]

This breaks the spelling check - you'll need to add "Jira" to the wordlist. And "gh-jira-sync-bot" fails as well ... maybe easiest to call it "GitHub/Jira sync bot"?

[keirthana]

Done, thanks!

[ru-fu]

You probably need to change it in this line here as well.

[ru-fu]

How about:

Suggested change

	By default, this file in the starter pack contains configuration for syncing
	issues from the starter pack repository to its documentation Jira board and
	hence does not work out of the box for other repositories in GitHub.
	The ``.jira_sync_config.yaml`` file that is included in the starter pack contains configuration for syncing
	issues from the starter pack repository to its documentation Jira board.
	Therefore, it does not work out of the box for other repositories in GitHub, and you must update it if you want to use the synchronisation feature.

[pmatulis]

By exposing real Doc values in the YAML file, isn't there the risk of users of remote repositories interacting with the Doc Jira board?

[ru-fu]

Not by accident because you need the configuration on the GitHub repo server side for it to work. So just including the default file will not do anything.
There might be an issue if someone includes the default file, does the server configuration for the Canonical Jira, but does NOT change the file. But in that case, they clearly want it to work, so they would probably track it down quickly.

[ru-fu]

Can you resolve the conflict?

[ru-fu]

You probably need to change it in this line here as well.

[keirthana]

I am not sure why the link check fails, I am able to open the flagged links and earlier today, the checks passed. Am I missing something?

[ru-fu]

I am not sure why the link check fails, I am able to open the flagged links and earlier today, the checks passed. Am I missing something?

It might just be a hick-up - I've restarted the check. 🤞

[lblythen]

Re. broken links from accessible addresses: I'm getting the same thing in another project, for working links to public repos that are copy-pasted from the targets. In each case they link to an anchor not a page. Couldn't work out why and in the end it wasn't worth pursuing further. I just confirmed they work, then added them as exclusions to avoid the errors.

[ru-fu]

Yeah, the linkcheck isn't great with anchors. :(

Let's try using https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-linkcheck_anchors_ignore_for_url to exclude checking anchors on GitHub URLs. Should be something like (not tested now ;) ):

linkcheck_anchors_ignore_for_url = [
    r'https://github\.com/.*'
]

Can you try adding that to custom_conf.py, Keirthana? Or do you prefer me to test this?

[keirthana]

Yeah, the linkcheck isn't great with anchors. :(

The inconsistency threw me off because the link check passed for a while with my changes.

Let's try using https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-linkcheck_anchors_ignore_for_url to exclude checking anchors on GitHub URLs. Should be something like (not tested now ;) ):

linkcheck_anchors_ignore_for_url = [
    r'https://github\.com/.*'
]

Can you try adding that to custom_conf.py, Keirthana? Or do you prefer me to test this?

Thanks Ruth! I will give it a try.

[keirthana]

The URL ignore logic seems to have worked :) But we are ignoring checks for all github URLs, I am not sure if this won't cause more trouble.

[ru-fu]

The URL ignore logic seems to have worked :) But we are ignoring checks for all github URLs, I am not sure if this won't cause more trouble.

We're still checking the GitHub URLs, but we're excluding the anchors from the checks. So we know that the link leads to an existing page - just maybe not to the correct section. But I've seen it before that GitHub anchors don't work, so this is probably a general problem.

[ru-fu]

Looks good to me. :)