Missing features for offline first architecture

[1mike12]

I apologize for the wall of text, but this is something I've been grapling with for months and I finally can put into words what I'm even trying to ask haha. So I'm building something with a strong offline component, as well as the ability to infinitely browse content when online. For the sake of example, let's pretend I'm re-creating a simple twitter with the following requirements:

1. I can always read my own tweets and craft new tweets offline, which will then be shown when online again.
2. I can scroll an infinite list of tweets out in the public
3. I have a list of people I follow available offline
4. I have a list of people who follow me available offline

Here's the basic code

export const UserModel = types.model("User").props({
   id: types.identifier,
   name: types.string,
})

export const TweetModel = types.model("Tweet").props({
   id: types.identifier,
   text: types.string,
   user: types.reference(UserModel),
})

export const UserStore = types.model("UserStore").props({
   userMap: types.map(UserModel),
   followers: types.array(types.reference(UserModel)),
   following: types.array(types.reference(UserModel)),
})
export const TweetStore = types.model("TweetStore").props({
   tweetMap: types.map(TweetModel),
   myTweets: types.array(types.reference(TweetModel)),
   feed: types.array(types.reference(TweetModel)),
})

const RootStore = types.model("RootStore").props({
   tweetStore: TweetStore,
   userStore: UserStore,
})

Herein lies the problem that the offline first and infinite scroll pattern seems to create. The store grows infinitely large and never drops in size when the user is scrolling the feed. So we need to clear the public tweets list every once in a while. But if we only delete the tweets in the feed, we are left with dangling users. This is because the majority of users in the array are not your friends, so this user list will grow infinitely as well unless we get rid of the users on the deletion of the feed tweets.

BUT you can't just delete users willy nilly since they could be one of your followers or followings. So now there needs to be a check written to let us know if a user is safe to delete. The rootstore becomes this:

const RootStore = types
   .model("RootStore")
   .props({
      tweetStore: TweetStore,
      userStore: UserStore,
   })
   .actions((self) => ({
      isUserSafeToDelete(userId: string) {
         const isInFollowers = self.userStore.followers.includes(userId)
         const isInFollowing = self.userStore.following.includes(userId)
         return !isInFollowers && !isInFollowing
      },
   }))

This is great, but it only works from the perspective of deleting a potential user that is referenced by a tweet in the feed. If I wanted to delete a user from the follower list, I would have to change the logic since we are looking from a different part of the tree. This is the complexity I'm seeing from a contrived example. In a real app with lots of stores and references, it would not be possible to write all permutations of the isUserSafeToDelete method.

What I really need is a getReferenceCount(Node) function, which would entail reference counting under the hood somewhere. But afaik this doesn't exist. Am I just going about this the wrong way? I have no idea.

In a normal app, you wont need to hold onto the state forever. Ie, when the user logs back into the website again, we start with a fresh rootStore and start growing it without fear

[EmilTholin]

Hi @1mike12!

This is one of the least explored areas of MST, and it was discussed a while back here. The main takeaway from that discussion is that a getReferenceCount type solution might be hard to implement since references are evaluated lazily in MST. This is almost always desirable, but unfortunate in this case.

It might be worth looking into mst-reference-pool by Jamon Holmgren for this use case, or at least take a look at the code for inspiration.