[CloudMirror] When will CF_CALLBACK_TYPE_NOTIFY_DEHYDRATE be invoked

[ho-229]

When the user clicks Free up space in the Explorer menu, CF_CALLBACK_TYPE_NOTIFY_DEHYDRATE and CF_CALLBACK_TYPE_NOTIFY_DEHYDRATE_COMPLETION are not invoked.

In the Cloud Mirror Sample, it uses ReadDirectoryChangesW and checks FILE_ATTRIBUTE_UNPINNED flag to detect if the placeholder is dehydrated by the user, then call a preview API CfDehydratePlaceholder for unknown reasons.

[PrimalZed]

So far as I can tell, "Free up space" doesn't actually itself trigger dehydration. It just sets the Unpinned attribute. That's why the example sync provider calls CfDehydratePlaceholder when it is unpinned.

I think the Dehydrate then removes the Unpinned attribute? At least I'm pretty sure that's not something in my implementation removing it.

Presumably, Windows can also trigger dehydration based on the Storage Sense. I don't know whether it actually merely sets the Unpinned attribute and still relies on the storage provider reacting to the Unpinned attribute.

Since CfDehydratePlaceholder is no longer in the official documentation (only in previous 2017 version), I'm guessing we "should" trigger dehydration by calling CfUpdatePlaceholder with the CF_UPDATE_FLAG_DEHYDRATE flag (or with the DehydrateRangeArray parameter). Doing that, it will be up to the sync provider to clear the Unpinned attribute (or have the check for the Unpinned attribute also check for the Offline attribute before calling CfUpdatePlaceholder).

I'm still trying to figure out how we're supposed to have a file with the Pinned attribute trigger fetch after calling CfUpdatePlaceholder. OneDrive seems to automatically update the content of a pinned file when the cloud file changes.

[ho-229]

Something related I was found: https://www.userfilesystem.com/programming/faq/

[PrimalZed]

I'm still trying to figure out how we're supposed to have a file with the Pinned attribute trigger fetch after calling CfUpdatePlaceholder. OneDrive seems to automatically update the content of a pinned file when the cloud file changes.

I think I ended up getting this working, but I'm not sure if it's the intended way. It also handles the Dehydrate command, instead of using the CfDehydratePlaceholder that has been removed from documentation. It took a lot of futzing to figure this out.

My goals are:

• Given a file that is downloaded but not pinned, when updating the placeholder, then the file re-hydrates with new data from the server
• Given a file that is downloaded and pinned, when updating the placeholder, then the file re-hydrates with new data from the server and is still pinned
• Given a file that is downloaded and unpinned (the dehydrate command), when updating the placeholder, then the file dehydrates (but does not need to keep the unpinned attribute)
• Given a file that is not downloaded, when updating the placeholder, then the file does not hydrate

(I'm not doing dehydration ranges, because I don't have a way to know what part(s) of the cloud file changed.)

Here's what I did to achieve it (with some helper functions) (to dehydrate, call with force: true)

/// <param name="force">Used to dehydrate even when files appear equal</param>
private Task UpdateFile(string serverFile, bool force = false) {
	var clientFile = _pathMapper.GetClientPath(serverFile);
	var serverFileInfo = new FileInfo(serverFile);
	var clientFileInfo = new FileInfo(clientFile);

	if (!CloudFilter.IsPlaceholder(clientFile)) {
		CloudFilter.ConvertToPlaceholder(clientFile);
	}

	if (!force && _fileComparer.Equals(serverFileInfo, clientFileInfo)) {
		return Task.CompletedTask;
	}

	var pinned = clientFileInfo.Attributes.HasAnySyncFlag(SyncAttributes.PINNED);
	if (pinned) {
		// Clear Pinned to avoid 392 ERROR_CLOUD_FILE_PINNED
		CloudFilter.SetPinnedState(clientFile, 0);
	}
	var downloaded = !clientFileInfo.Attributes.HasFlag(FileAttributes.Offline);
	var redownload = downloaded && !clientFileInfo.Attributes.HasAnySyncFlag(SyncAttributes.UNPINNED);
	long usn;
	using (var hfile = CloudFilter.CreateHFileWithOplock(clientFile, MyFileAccess.Exclusive | MyFileAccess.Write)) {
		var relativePath = _pathMapper.GetSubpath(clientFile, _clientOptions.Directory, string.Empty);
		// With MarkInSync flag, and Dehydrate flag when dehydrate param is true
		usn = CloudFilter.UpdateFilePlaceholder(hfile, serverFileInfo, relativePath, dehydrate: downloaded);
	}
	if (pinned) {
		// ClientWatcher calls HydratePlaceholder when both Offline and Pinned are set
		CloudFilter.SetPinnedState(clientFile, SyncAttributes.PINNED);
	}
	else if (redownload) {
		CloudFilter.HydratePlaceholder(clientFile);
	}
	return Task.CompletedTask;
}

I expect I'll have to refine this further for when it cannot get an exclusive lock on the file. It might need something like merely use ClearInSync flag on the update, and have a separate process that periodically checks for not-in-sync files to update.

[PrimalZed]

Back to the original topic, it looks like we should specify AutoDehydrationAllowed on our sync root:
StorageProviderHydrationPolicyModifier Enum

I think this is for the Storage Sense dehydration that I mentioned in #353 (comment).