MSC4073: Shepherd teams #4073
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| # MSC4073: Shepherd teams | ||
|
|
||
| Matrix currently has significant problems with its specification change process. There are several common complaints | ||
| from contributors: | ||
|
|
||
| 1. "Nobody responds to my MSC" | ||
| 2. "It takes forever to get an MSC passed" | ||
| 3. "It's not clear what the status of my MSC is, or what I am supposed to do next" | ||
|
|
||
| From conversations with various Spec Core Team (SCT) members, all of these have (at least) one identical root cause: | ||
| the SCT does not have enough bandwidth to manage, review, and approve all of the submitted MSCs. This causes | ||
| communication to become minimal, and for MSCs considered "less critical" to languish and eventually become abandoned | ||
| by their authors, who often give up on Matrix protocol development entirely due to their bad experiences. The SCT | ||
| cannot be bypassed, because its final approval is required for any MSC to be considered mergeable. | ||
|
|
||
| ## The current process | ||
|
|
||
| In the current MSC process, it is the responsibility of the SCT to manage the discussion and direction of every MSC, | ||
| and ultimately give its final approval before a merge is possible. A shepherd may be assigned to guide the MSC | ||
| process independently of the SCT - but they ultimately only serve the role of a *guide*, and do not hold any | ||
| authority over the final decision to merge or reject. A shepherd is also not required, and frequently is not | ||
| assigned. Altogether, this often leaves an impression with contributors of "appealing to a disinterested SCT". | ||
|
|
||
| ## Proposal | ||
|
|
||
| This MSC aims to solve these problems, or at least make a first significant step towards doing so, by changing the | ||
| scope of responsibility of the SCT. It introduces the concept of a "shepherd *team*", and makes them primarily | ||
| responsible for the MSC's progress. This team is assigned on a *per-MSC* basis, and it is modelled after | ||
| [the "shepherd teams"](https://github.com/NixOS/rfcs/blob/master/rfcs/0036-rfc-process-team-amendment.md) in | ||
| [the NixOS RFC process](https://github.com/NixOS/rfcs#readme). | ||
|
|
||
| The new process will work as follows: | ||
|
There was a problem hiding this comment. There are some cases where an MSC is fairly trivial, and everyone just agrees that it should be done (MSC3828 might be a good example of this). For such MSCs, creating a shepherd team for it might be unnecessary administrative overhead, and I wonder if it might be worthwhile to keep a "shortcut" of having the SCT accept the MSC. I'm not sure if it happens often enough, though. There was a problem hiding this comment. That is a fair point; I'm not opposed to adding such a shortcut, but I do think that it should be a well-defined one, such that that path is only taken when there's reasonable certainty that it can be passed quickly. My main concern is that if there isn't such a defined threshold, it can become appealing for SCT members to take on more work than they can realistically handle, out of excitement/optimism. I'm thinking something along the lines of:
... but I'm open to other suggestions as well. There was a problem hiding this comment. Personally, I'm not a fan of making too many rules unless necessary. If we do want to add this shortcut, I'd be more in favour of either making it a guideline rather than a rule ("if the SCT hasn't managed to get their act together within 3 weeks, then they should really consider assigning it to a shepherd team"), or leaving it open and then fixing any bugs if they occur later on. |
||
|
|
||
| 1. Community members submit nominations for shepherds, in the MSC's discussion thread. The nominees should be people | ||
|
There was a problem hiding this comment. Obviously, this will depend on peoples' willingness to be shepherds, so we'll need to see how much involvement we can get. I suspect that the community is at a point where we should be able to get shepherds in at least most subject areas. Some other areas, I'm not so sure about. We may need to find out how many people there are who would be willing to be shepherds, in what areas, and how much time they can commit to. There was a problem hiding this comment. I suspect that initially, it will be difficult to find enough shepherds, because a lot of people have already (halfway) given up on the spec process due to past experiences, and it will take time to 'rebuild trust' on that point. Eventually, I expect this problem to resolve itself; it is already common for implementers to involve small communities of people for review, and for people outside of that community to have critical feedback in more informal ways. I think that a lot of that feedback and motivation can be moved into shepherd roles in the long term. Given that I expect a somewhat rocky start, it may be a good idea to deal with the existing MSCs by slowly and gradually moving them over to the shepherd model, to get them out of the weeds. There are a number of MSCs I can think of that a lot of people are passionate about, and that have also raised criticisms, and which would probably be good candidates to try out the shepherd process on to start with. If this goes well, that can be expanded to other existing MSCs. How to deal with existing MSCs should probably be part of the proposal too... I will add that soon as well. There was a problem hiding this comment. Relatedly: clear guidance will be necessary for shepherds, on how to conduct their part of the job, so that they can work independently without burning out. This is more of a note-to-self, in that I should add this to the proposal. There was a problem hiding this comment. (Both of those points have now been added to the proposal.) |
||
| who are closely familiar with the subject matter of the MSC, but not be the author. Community members may also | ||
|
There was a problem hiding this comment. Another minor point: some times, multiple conflicting MSCs are submitted that solve a problem in different ways. I think it makes sense to have the same shepherd team to those MSCs to sort out which approach to take. In this special case, excluding the authors may unnecessarily limit the pool of shepherds (as, instead of just excluding one person, we may now be excluding, say, three people), and it may even be beneficial in such cases to have the authors of conflicting MSCs co-shepherd the proposals. I'm not sure what would be the best way to handle this exception. There was a problem hiding this comment. Thinking through it a bit, I'm not certain that that approach would be ideal. The final state of an MSC does not necessarily look much like its initial draft, depending on how much refinement and feedback it has gone through, and especially for more complex design problems, it is very possible that what initially seemed to be the worse solution, actually ends up being the optimal one after a few rounds of refinement. Given that, I think it would actually be beneficial for conflicting proposals to proceed independently, with the understanding that coordination between the shepherd teams is needed before any FCP is called; so that each proposal can independently develop, and the comparison ends up being between the refined proposals, rather than between drafts. In the current model, this would place a significant burden on the SCT, but in a shepherd-driven model, that would not be a concern; they would be separate teams, like with any other two unrelated MSCs. Eventually, the author may decide to withdraw their proposal because they feel that the alternative is better (in which case the relevant shepherd team is disbanded), or when both proposals have reached a near-FCP stage, the shepherd teams might in their coordination decide that only one of them is worth moving ahead with, or even decide that a merge between the two should take place (in which case the SCT could reorganize the shepherd team with an exemption from the usual no-authors rule). That would be dependent on the specific case. I feel like this is more a matter of "providing good guidance to shepherd teams, for coordinating this stuff", than a matter of having a strict written no-conflict policy. There was a problem hiding this comment. Yeah, it may be better for conflicting MSCs to proceed independently. I think we should just leave this open as something to consider. |
||
| nominate themselves. | ||
| 2. The SCT selects multiple nominees to become the "shepherd team"; ideally 3 or 4 people. | ||
| - The SCT may likewise nominate candidates (eg. if there are too few nominees), but should leave the opportunity | ||
| for other community members to object to the SCT's nominations. | ||
| - No more than half of the shepherds may be SCT members. | ||
| - The shepherd selection should avoid conflicts of interest, and be a diverse representation of perspectives on | ||
uhoreg marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| the topic (within the boundaries of the Code of Conduct). | ||
| 3. The shepherd team will be responsible for guiding the discussion on the proposal. This discussion should | ||
| incrementally improve the proposal until outstanding concerns have been resolved. They may request administrative | ||
| assistance from the SCT if eg. moderator intervention is required. | ||
|
There was a problem hiding this comment. Due to the use of GitHub the shepherds probably would need to depend on the SCT to resolve comment threads and such. This is a bit annoying, but hopefully not a huge deal. They might also need to rely on them to modify the text (via merging PRs?) or getting access to the original source repository to modify the pull request. There was a problem hiding this comment.
I assume this comment is under the assumption of the second option in your other comment? If so, it would not be relevant in this case, as the author does principally retain control over the MSC, and the author can also resolve conversations themselves (so SCT intervention would only be necessary in the case of a dispute between author and shepherd team).
Good point, this will require some sort of solution. Given that "editing responsibility is transferred to shepherd team" should be the exception rather than the rule, I feel like it's acceptable for this to require some setup - even if a contributor does not wish to grant the shepherd team access to their fork as a whole (I don't remember to what degree this can be branch-restricted), in the worst case, I imagine a dedicated fork with shared access could be set up just for the purpose of shepherding that one specific MSC. I'm not sure whether it's possible to repoint an existing MSC PR to source itself from a different fork after-the-fact, though; if not, that may be a problem for this approach. Do you feel that the exact procedure for such a responsibility transfer should be included in this MSC? Or would it be acceptable to leave it as an unspecified implementation detail, and/or only address it in a separate MSC after this one? There was a problem hiding this comment. It might be possible to just add shepherds as maintainers to this repository, and make sure that MSCs have the "Allow edits and access to secrets by maintainers " setting enabled. Presumably, shepherds should be able to be trusted to behave properly and not abuse their privileges (e.g. by interfering with other MSCs). There was a problem hiding this comment. That's a point I hadn't considered yet. I think that's a viable option, but it's also going to depend on whether all the current SCT members feel comfortable with that approach. There was a problem hiding this comment.
Only somewhat -- if the author is unresponsive or if the shepherds are deciding what to do with an MSC, they seem like they would need some editorial power to resolve threads, etc. Just to be upfront -- I don't think the technology not being perfect here is something that should block this proposal, just wanted to point out a particular sticking point. There was a problem hiding this comment.
Right. I'd be inclined to classify this as an exceptional case requiring administrative intervention; if the author is unresponsive, and no explicit transfer of editing responsibility has taken place, then (assuming shepherds do not straight-up get access to the spec repository) it would be up to the shepherd team to request administrative intervention from the SCT. As long as this doesn't happen frequently, it sounds sufficiently workable to me as a solution. And if it does happen frequently, and authors frequently abandon MSCs, then that is IMO a bug in the specification process that needs to be addressed - because we shouldn't be seeing that sort of abandonment rate in a healthy spec process. There was a problem hiding this comment. Speaking from my experience of being a shepherd (on #2666), and from a past idea of mine; This problem can be fixed by introducing a bot that can answer to commands like This may increase noise, but it would at least allow the shepherd some control over otherwise locked functions of a shepherded PR. One issue I also had was the inability to push commits to the PR branch; this can be solved by instructing the bot to push commits onto the PR branch, targeting some branch the user is currently working on. (Force pushes would be disallowed, and only fast-forward merges/tracking would work) |
||
| 4. If the MSC proposes to standardize functionality that is already informally supported in one or more protocol | ||
|
There was a problem hiding this comment. Note: I have belatedly added this section to explicitly call out that shepherds are responsible for seeking review from existing implementers; I believe that this is currently a task that the SCT does, but it requires insight into the details of pending proposals that the SCT may no longer (always) have under this proposal. |
||
| implementations in a different form, the shepherd team should additionally reach out to the relevant implementation | ||
| developers to notify them of the proposal, and to invite their review. | ||
| 5. Once the shepherd team feels that all concerns have been resolved satisfactorily, and that there is no meaningful | ||
| refinement left to be done, they can carry out a final technical review, and call FCP (Final Comment Period). Alternatively, if the shepherd team feels that the proposal is not salvageable, they may choose to reject it. | ||
| - This requires unanimous agreement from all shepherds. | ||
| - The SCT must be informed immediately by the shepherds when the FCP starts, so that the SCT can make a public | ||
| 'last call' for feedback. Likewise, the SCT must be informed of a decision to reject. | ||
| - The FCP process itself remains as it was before. | ||
| 6. If FCP passes without objection, the shepherds declare the MSC accepted, and notify the SCT. The SCT will then | ||
| merge the proposal. | ||
|
|
||
| Some of the most important takeaways from this: | ||
|
|
||
| - The SCT is no longer the final reviewer and approver; this responsibility is delegated to the shepherd team for | ||
| that MSC. | ||
| - The shepherd team is trusted to ensure that: | ||
| 1. the proposal represents diverse interests, and | ||
| 2. the proposal does not conflict with other proposals, coordinating with shepherds for other MSCs if necessary. | ||
| - The SCT no longer concerns itself with the *content* of MSCs, but only with supervising the process. | ||
| - The SCT may still intervene where necessary to resolve administrative disputes, or replace shepherds if they become | ||
| unavailable. | ||
|
|
||
| Throughout the process, the original author(s) remain responsible for updating the proposal in response to feedback, | ||
| as is currently already the case. If the author(s) wish to step back, they may - in agreement with the shepherd team - | ||
| choose to transfer this responsibility to the shepherd team (or another community member) instead. | ||
|
|
||
| These changes are a significant step towards a more community-driven protocol, by putting more of the specification | ||
| process in the hands of community members, and more crucially, it frees up significant capacity from the SCT. | ||
|
|
||
| This proposal does not currently encompass the question of "who writes the final specification text?" - that is left | ||
| unchanged from the current situation, and may be clarified or changed in a separate MSC. | ||
|
|
||
| ### Tasks for the SCT | ||
|
|
||
| For the implementation of this proposal, the SCT will carry out the following tasks: | ||
|
|
||
| 1. Develop guidance for shepherd teams, explaining their responsibilities and how to go about implementing these, as | ||
| well as defining any additional circumstances under which the SCT should be notified. Particular attention should | ||
| be paid to the topics of preventing burnout, and constructive conflict resolution. | ||
| 2. Develop and publish guidance for the SCT itself, on how to select a representative shepherd team for a given MSC, | ||
| and which concerns (eg. organizational distribution, diverse viewpoints) to take into account in their selection. | ||
|
|
||
| ### Transitioning existing proposals | ||
|
|
||
| There is currently a large set of existing MSCs that is in some kind of stalled state. These are currently under the | ||
| responsibility of the SCT directly, as that has been the process thus far. To prevent organizational chaos and to | ||
| account for the expected low initial availability of shepherds, it would be unwise to immediately transition all of | ||
| these proposals to the shepherd team model. | ||
|
|
||
| Instead, these proposals will be gradually transitioned; starting with languished proposals for which there is a known | ||
| "support base" of proponents and, conversely, a known interest in critical review. These can serve as the initial | ||
| testcases for this new policy, and it may even be worth starting this process experimentally before this MSC has been | ||
| fully accepted. | ||
|
|
||
| Drawing lessons from these initial experiences, the process can then be improved where necessary and, over time, be | ||
| applied to the remainder of the pending MSCs that were started under the old policy. | ||
|
|
||
| ## Anticipated outcomes | ||
|
|
||
| Primarily, this should unblock the many MSCs that some community members feel strongly about, but that the SCT has | ||
| not had the capacity to advance, eg. because it accommodates a relatively specialized need. This should overall | ||
| significantly improve on the throughput and responsiveness of the MSC process, much like it has already done for | ||
| NixOS. | ||
|
|
||
| Indirectly, this would improve community trust and engagement regarding protocol development, because contributors | ||
| would no longer feel ignored, and they can continue building additional work on top of previous work - where that | ||
| previously would have been blocked on a 'stuck' MSC, and the uncertainty of it ever getting merged. | ||
|
|
||
| Another anticipated indirect effect is increased transparency in the specification design process; it is currently an | ||
| issue that a lot of specification design happens 'internally', behind closed doors, and this can currently be gotten | ||
| away with due to every MSC going through a central point of review (the SCT) before merging, where any | ||
| incompatibilities can be ironed out. | ||
|
|
||
| In the 'shepherd team' model, this is no longer possible, as one cannot expect 'internal' and undocumented decisions | ||
| to be taken into account by other shepherd teams in their final review, and so all such specification development | ||
| *must* happen out in the open (as is ultimately desirable for an open protocol). | ||
|
|
||
| ## Security concerns | ||
|
|
||
| It is *technically* possible for an organized effort to disrupt the specification by passing something without SCT | ||
| review. However, this has not been a problem for NixOS at all, and would require compromising the *entire* shepherd | ||
| team. Additionally, administrative intervention from the SCT remains possible in exceptional cases (like when a | ||
| shepherd has become unavailable), and this could likewise be applied to such cases. | ||
|
|
||
| ## Alternative approaches | ||
|
|
||
| - __Scaling up the SCT:__ Organizations become more unwieldy as they try to scale up monolithically. Additionally, | ||
| this requires more full-time commitments than anyone currently seems to be willing and able to fund, and would | ||
| bring conflict-of-interest concerns with it. This does not seem viable. | ||
| - __Doing nothing:__ Increasing amounts of potential contributors are bouncing off Matrix due to the problematic MSC | ||
| process, and it is well-documented that there are currently very few external contributions. This would likely lead | ||
| to a slow death of Matrix as an open protocol. | ||
| - __Bigger MSC scopes:__ This would reduce the amount of units for the SCT to review and manage, but make changesets | ||
| so large that the discussion of the implementation details would likely exponentially explode, making the problem | ||
| worse on another axis. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Process-wise, I think it's actually best if we sit on this and think about it for a while (ironically). The MSC is largely written in response to recent events, and we should be cognisant that those events may be clouding judgement. Similarly, we should not rush into decisions when it comes to process: they need careful thought and consideration, even if appearing simple on the surface.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is not quite accurate; none of these issues are new, and all of them have been brought up in a more informal manner many times, including suggestions to have the SCT do less. What has changed recently, are two things:
So rather than this being a sudden novel idea, it's an idea that has been floating around in various form for quite some time already, and has already proven its worth in another community (which has actually partly modeled its spec process on that of Matrix, to my knowledge), and it is now at a point where concrete solutions need to start appearing, so as to not to lose the remaining trust of the community.
Of course, careful consideration is important, that is part of why I submitted this as an MSC rather than trying to backchannel it. But this does not need to translate into a long approval process in terms of time, and it's not clear to me in what way this proposal would be expected improve by letting it sit, in and of itself.
In this consideration, given the circumstances and the worsening community trust issues, I also think it is important to ask: what possible worst-case scenarios are there for the outcome of this proposal, that are worse than the outcome of not changing the process?
Because governance change proposals do not need to be perfect to be useful, they just need to be an improvement over what we currently have, so if it turns out that something was overlooked, this can always be rectified down the line (especially since the SCT still retains the power to intervene if really necessary).