Determine whether introduced work has actually delayed work

[dspasojevic]

When performing sprint-based forecasting, it is reasonably simple to determine whether introduced work delayed other work - there is a clear signal that work was brought into a sprint and then other work was prioritised over it.

Outside sprints, we can consider other signals that indicate a ticket was intended to be completed:

1. top X of backlog
2. a particular (perhaps project specific) status

[danielnixon]

top X of backlog

I think we can do something even simpler than this. I think a new ticket delayed work if:

• it is now completed (at the time of forecasting)
• it is now in progress (at the time of forecasting)
• it is in the backlog above the ticket we care about at the time of forecasting

[danielnixon]

There's also the question of when and if existing tickets delayed (planned) work, e.g. existing tickets being moved up in the backlog.

[danielnixon]

We could consider looking at the way the backlog looked at various points in the past. Then we get an idea of how many tickets were moved ahead of our target ticket over time, regardless of whether those promoted tickets were new or old.

Could Jira’s APIs tell us what we need to know? Can we see historical backlog rankings? Or do we need to keep track of past state snapshots locally?

[danielnixon]

There's also the question of when and if existing tickets delayed (planned) work, e.g. existing tickets being moved up in the backlog.

And the inverse of this too I suppose. Previously prioritized tickets being demoted below the ticket that we care about. So this number (rate of new work overtaking planned work) could technically be negative.

[shouvSarker]

From what I understand, to deliver this ticket we need the exact backlog history, like how exactly the tickets were moving on the board in the past "X" amount of time. I have not noticed any out of the box solution in JIRA for this, but jql was and changed operators can help us construct queries to dig out this piece of information. We do not want to maintain a separate database, at least not yet while this is a script, so need to come up with a reliable jql query.

How we fit that into our forecasting algorithm is part 2 of the problem. 🦖

[shouvSarker]

Possible fix:

1. Rank stuff (the rank field seems to have a bit of black magic characters but should be able to parse, just grab the black magic character for our ticket's rank and do gt/lt with eyes closed)
2. Query with changed (rank went from down to up, or up to down). example: where rank changed from <ticketRank to >ticketRank before dateRange1 after dateRange2

From this, we would know the tickets that were behind us in the backlog but got lifted up later or vice versa. Would this information be enough or are we looking for something more specific?

[kn-ad]

So, just to restate it for clarity, the problem is that we want the script to predict how many tickets the team will do before they get to the target ticket. It assumes that the tickets ahead of the target in the backlog will get done before the target ticket, which is probably OK for now. It also assumes that all new tickets will be done before the target ticket, but that's not realistic for most projects.

We could ignore any tickets that are currently below the target ticket in the backlog, but that wouldn't help with tickets that are/were only ahead of the target ticket because they were created before it. Even if the team was prioritising tickets perfectly and was never swayed by the age of a ticket, there would still be tickets that were completed before the target ticket was created, but were less important than it.

We could try to estimate the team's rate of creating (or promoting) tickets that, if they were created today, would be prioritised above the target ticket in the current backlog. If there's a way to use Jira's API to get the history of inserting/moving tickets in the backlog, we could calculate the probability of a ticket in position a of the backlog being moved to position b. For new tickets, a would be the bottom of the backlog.

When a ticket is moved up in the backlog, every ticket between it's old and new position gets moved down one position. So if the history was Array<{ fromPosition: number, toPosition: number}>, when you drag a ticket up n positions in Jira, n+1 elements would get added to the history array. And I think our probability calculation would use all of the elements.

Actually, I think we could simplify things by only calculating two probabilities for each position, the probability of a ticket in that position being moved up by one position and the probability of a ticket in that position being moved down by one position. So the history would be Array<{ fromPosition: number, moveDirection: 'up' | 'down'}> and when you drag a ticket up n positions in Jira, we'd add n 'up' elements for the ticket you dragged and a 'down' element for each of the tickets between it's starting and ending positions.

Then we could use the results from the probability calculation to write a function like (positionInBacklog: number, durationInDays: number) => { expectedPosition: number }. If a ticket at positionInBacklog is more likely to move up, expectedPosition will be positive. If it's more likely to move down, expectedPosition will be negative.

I'm not sure how we would account for bug vs feature tickets after that, though. Or other properties like story points. Maybe we should try to just feed everything into some machine learning library instead and let it work out what matters.

[shouvSarker]

I'm not sure how we would account for bug vs feature tickets after that, though. Or other properties like story points. Maybe we should try to just feed everything into some machine learning library instead and let it work out what matters.

Maybe this is a bit oversimplification, but we can run another simulation to see how tickets are moving could introduce delay, and add/substract that from the original prediction? 💁‍♂️

[kn-ad]

I just tried searching with this JQL Rank was 1 and got an error:

History searches do not support the 'Rank' field.

So it's looking like that the historical rank data might not be stored at all.