Timed window of opportunity to delete artefacts.

[danielsz]

Please allow authors to delete artefacts just after publishing. The exact amount of time during which they are allowed to do that doesn't matter. The principle does.
Mistakes will happen. This sentence comes in many flavors, as it is such an essential trait in humans. Currently, Clojars doesn't allow authors to undo their mistakes.
This is not a breach in the overall immutability of the system. Not in any meaningful way. There is no race to fetch a mistakenly published artefact. If it was up to me, nobody would have time to. I realized my mistake the second I published it.
Also, it looks like this would take care of many of the pleads for deletion people resort to as a desperate measure.
Thank you for listening.

[danielcompton]

I've talked with Toby and while I understand where you're coming from, allowing users to make deletions isn't a goal for Clojars. Our aim is to provide an immutable and stable repository.

Clojars was created in part as a reaction to Maven Central requiring deploys to be manually promoted to the repository after being released at the command line. If the ability to tightly control which versions of your software are released, and to prevent accidental deploys, then you could look at publishing through Maven Central, as it has the controls you're looking for. It would make no difference to your consumers as all Clojure tooling has Clojars and Maven Central loaded.

[danielsz]

You have granted 400 deletions requests. Its all there in the issues. What are you even talking about?

[tobias]

It's unclear where you are getting the 400 deletions number from, but
I don't think the number of deletions that have occurred are relevant
to this feature request.

There are a few ways to implement what you are asking:

1. allow deletion from the primary repo, but only if the artifact is
   younger than a certain age.

I have a few concerns about this approach:

• what constitutes a good age? 15 minutes? an hour? a day? a week?
  Whatever we choose, someone is going to request a longer time.
• what happens when a user deploys a version, others download it, then
  the user deletes it and redeploys the same version with different
  content? That leads to inconsistent behavior and difficult to track
  down issues when a someone else on the team downloads the same
  version, but later, so actually has a different jar.

2. store artifacts in a staging repo, moving them in after a time
   period in which you could delete the artifact.

Concerns here are:

• the same age concern as above
• we now have the burden of maintaining more repositories, and have to
  implement code not only to delete on request, but to prevent that
  deletion from occurring during the move
• artifacts would no longer be available right after deployment -
  users would have to wait until the waiting period has expired.

3. store artifacts in a staging repo, and require users to manually
   release them to the primary repo when they are satisfied with
   them. This is the approach that Sonatype Nexus takes.

Concerns here are:

• library authors now would have to take an extra step for each deploy
  to release their software, which would be a major change. The lack
  of this step was an explicit design decision early on in Clojars
  development. Having to leave mistaken deploys or occasionally
  manually delete deployments is the price of that convenience.

Please keep in mind that Clojars is maintained by volunteers, so any
solution that increases the operational complexity of the system needs
to be considered carefully. Given the effort to implement and maintain
this feature, I'm not sure it would be the effort, given the low
number of mistaken deploys that occur that are anything more than mild
embarrassments.

[danielcompton]

An alternative to deletions, that would help prevent accidental deploys, without affecting the immutability of the repo is to provide a dry run option for lein release and lein deploy. This would show people what version would be deployed, and to where - technomancy/leiningen#2229.

[kahunamoore]

I suppose it might be worth considering why people want to delete deployments. If they accidentally embedded passwords/creds then having a way to allow users delete their own in such an emergency might be valuable but I don't think that is very common. Do we know why most deletions are requested? If not we might want to ask.

One possible way to do this is having the existing "delete me" trigger an automated delete. Given that redeploying requires authentication this would be secure. It only eliminates the manual step (no UI changes, etc)

[danielsz]

@kahunamoore +1

@tobias What consitutes a good age? This is a good question. 1 minute is certainly a good age. Maybe 10 minutes is reasonable too. Within that window of opportunity, authors are granted full permissions to delete the artefact in the same way that deployment was granted. This is in line with @kahunamoore's suggestion. This would take care of many deletion requests, and yes, there will always be those cases that fall outside of the granted scope, and then you will need to resort to the current deletion requests/approval/rejection ballet that takes place in the issues. But those will be dramatically reduced. Win win situation for both users and admins.

All this without impacting the immutable nature of the repository.

[danielsz]

@tobias An alternative to the timed window of opportunity would be a check on the number of downloads. As long as that number is zero, it is safe to delete. Nobody is using it yet. My snapshot version that I asked to delete currently has 0 downloads. If this number changes, upon whose shoulders would the responsibility rest in your opinion?

[kahunamoore]

Please take my comments with a grain of salt. I'm not a maintainer and don't have the history/knowledge of Clojars like others do.

Also, congratulations on getting into SFC! Way to go... thank for all your hard work.

[johanatan]

+1 to checking download count & if zero, allowing deletions.

[tobias]

@kahunamoore:

The most common cause of deletions that we have performed in the past have been private code and/or credentials being published, and those generally come through email. Of those, I'd say less than half have been out for a while (days to months) before the issue was discovered, so a cooling off period wouldn't help with those.

Using the description contents as a trigger is an interesting idea, but wouldn't work - say you push 0.3.0, and realize you want it (and not the entire project) deleted. You can't push to 0.3.0 again, so would need to push an 0.3.1 (or whatever), and we'd have no way to know you want 0.3.0 + 0.3.1 deleted.

And thanks for the kind words :)

@danielsz:

I would only be comfortable with an age period if there were a staging repository. We can say that it's unlikely that a user will download the release within that period, but we can't guarantee that.

@danielsz + @johanatan:

WRT the download count: we can't see the download count for today, only yesterday and before. This is because we only process logs once a day, which includes the nginx log and several hundred logs from our CDN provider, so looking at downloads isn't a viable option.

I'm open to a solution for this, but will likely need someone in the community to drive the implementation - I don't have a lot of time to devote to Clojars currently.

[puredanger]

Not to be flip, but why not just deploy a new version that fixes whatever was in the broken one?

This happened to me in the past and that seemed like the obvious way to "fix" it.

My presumption is that deletes create a lot of implementation and policy complexity and it doesn't seem worth that in comparison to the frequency or severity of occurrence.

The Maven Central staging model is a pain and certainly everything we deploy for Clojure and contrib is done in an automated way such that it creates the identical problem, so I don't see staging as any improvement.

[johanatan]

@puredanger I'm not sure how the suggestion to "just deploy a new one" would fix the case that spawned this issue: the 'defect' was in the version identifier itself (namely the accidental inclusion of "SNAPSHOT" when it was not intended to be a SNAPSHOT).

[johanatan]

I agree that releasing X.Y.Z after releasing X.Y.Z-SNAPSHOT is strange, however, since they would both contain identical code, the SNAPSHOT semantics would still be maintained (i.e., SNAPSHOT is >= the non-SNAPSHOT in terms of functionality/api surface area/etc).

[danielsz]

@puredanger @johanatan Correct, there is no danger of shipping broken code. But I believe we should examine other avenues when dealing with accidental deployments. Some thoughts in #615,

[ustun]

Just another data point:

npm made some changes for unpublishing after last year's leftpad fiasco.

http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy

If the version is less than 24 hours old, you can unpublish it. The package will be completely removed from the registry. No new packages can be published using the same name and version.

If the version is older than 24 hours, then the unpublish will fail, with a message to contact support@npmjs.com.

If you contact support, they will check to see if removing that version of your package would break any other installs. If so, we will not remove it. You’ll either have to transfer ownership of the package or reach out to the owners of dependent packages to change their dependency.

If every version of a package is removed, it will be replaced with a security placeholder package, so that the formerly used name will not be susceptible to malicious squatting.

If another member of the community wishes to publish a package with the same name as a security placeholder, they’ll need to contact support@npmjs.com.  npm will determine whether to grant this request. (Generally, we will.)

[tobias]

@johanatan Releasing X.Y.Z after X.Y.Z-SNAPSHOT is the proper practice, and not strange at all. The snapshots lead up to the actual release, not vice-versa.

[awb99]

Is it possible to exclude DEPRECIATED artefacts from search? Or at least show them AFTER the non depreciated artefacts. We have reorganized our artefact system, and want to make sure that users do not get confused and by mistake use old versions. It is not very helpful, if the first couple of pages only show depreciated projects.

[russellwhitaker]

Five years and a number of world-changing events later... thread necromancy time! Did anything interesting happen apropos this topic? I found myself scrambling in the wee hours earlier today, having goofed on the packaging of an artifact - missed a piece after having converted a project to use a git submodule - to deploy a fixed artifact, after having issued lein deploy clojars on the broken one.

[tobias]

No, there hasn't been any further discussion or movement on this topic. It hasn't come up in the intervening 5 years.

[russellwhitaker]

Thanks @tobias , much appreciated.