Consider mutating the state

[edusperoni]

I'm diving a bit into pinia-orm as we intend on using it for storing a lot of data and need it to be blazing fast (which is why I opened #1206).

Another big issue for us is that store manipulation is painfully slow when you have lots of entities.

I managed to narrow it down to the way the entities are inserted/updated/removed from the store. For example:

 save(this: DataStore, records: Elements, triggerQueryAction = true) {
      this.data = { ...this.data, ...records }

While this is fine and usually fast in plain JS, Vue's reactivity make it awfully slow, as it goes through all changed items (in this case, everything, since we changed data itself), so take this example:

https://stackblitz.com/edit/vue-3-composition-api-demo-ez5csp?file=src%2Fpinia.ts

export class User extends Model {
  static entity = 'user' as const;
  @Attr(null) declare id: number;
  @Attr('') declare email: string;
}

export const pinia = createPinia().use(createORM());

export const userRepo = useRepo(User, pinia);

for (let i = 0; i < 1000; i++) {
    const start = Date.now();
    userRepo.save({
      id: i,
    });
    const timeToRun = Date.now() - start;
    console.log(timeToRun)
  }

you'll see that timeToRun will increase progressively. With a big enough size, time to save an item can take over 15ms. I believe the complexity of this code with vue reactivity enabled is O(n²), as each insert operation is O(n) (goes through all items).

In sum, if your store is big enough, every save, insert, delete, etc will take a significant amount of time to run. Deletes are particularly heavy because delete complexity is already O(n²) by iterating over existing items and using array.includes.

This all, of course, can be mitigated in 2 ways:

1. Make data non-reactive. By using markRaw we make the data object itself non-reactive, reducing the complexity to O(n). I'm not sure about the full implications of this though. From what I gathered, the entities themselves would still be reactive.

export function useDataStore<S extends DataStoreState, T extends DataStore = DataStore>(
  id: string,
  options: DefineStoreOptionsBase<S, T>,
  query?: Query,
) {
  return defineStore(id, {
    state: () => ({ data: markRaw({}) } as S),
    actions: useStoreActions(query),
    ...options,
  })
}

2. Mutate the state through Object.assign. Although I'm not a fan of state mutation at all, that seems to be the strength of vue's reactivity. By doing this the complexity goes down massively as vue will only iterate over changed items. The same has to be done to delete as well (use the delete operator instead of changing the state).

 save(this: DataStore, records: Elements, triggerQueryAction = true) {
      Object.assign(this.data, records)

I'd be happy to implement these changes in a PR, but wanted to discuss them first as they can change architecture/behavior for (advanced) users

[edusperoni]

@CodeDredd I noticed you've pushed a new update and wanted to bring your attention to this idea if you have some spare time. Number 2 is a pretty simple change with massive gains and doesn't seem to have any downsides since vue uses proxies anyway to notify changes

[CodeDredd]

@edusperoni Thanks for the hint.
I think it sounds good for a poc. If all tests are still green i don't see why this cannot be done. Would be awesome if you could make a PR.
I right now try to make a namespace feat as next.

[vesper8]

@CodeDredd @edusperoni now that #1604 has been merged and released as part of 1.7.0, I was wondering.. are you satisfied with the performance now? For me, I'm just now migrating from vuex-orm and the act of saving a large response that essentially contains hundreds (not thousands) of records and nested relationships, still feels much much slower than it was with vuex-orm. I can't even imagine how slow it may have been before this PR was merged. But unfortunately.. it's still unacceptably slow.. so I have quite the dilemma on my hands. Do you feel like it's fast enough for your use-case now? Any insights as to why it might be much slower than it was with vuex-orm in my case?

Many thanks

[CodeDredd]

@vesper8 Hey vesper...i don't know your usecase exactly. without a reproduction this is hard to say.
There are performance unit tests and they all dont take long. But of course i dont know your complexity.
Normally it shoudln't be that slow. What relations do you have the most? belongsToMany ? STI? Morph Relations ?
There is also find performance test like this one: Find Speed Test

• unit test: https://github.com/CodeDredd/pinia-orm/tree/master/packages/pinia-orm/tests/performance

[vesper8]

@CodeDredd So.. I think I figured out what was causing it to be so slow. I was using withAllRecursive in a lot of places. With vuex-orm this didn't cause performance issues but with pinia-orm it causes major major slowdowns. So I began commenting out these withAllRecursive statements and it made a huge difference. In some cases I replaced them with multiple with('someTable) statements. And in other cases.. I was surprised to realize that it appears just doing a regular useRepo(Project).find(someId) returns the first level of relations already loaded ? This felt odd.. I'm pretty sure vuex-orm didn't work that way.. but indeed.. I printed the my model after doing the above query and it's got all the first level of relations loaded even though I didn't ask for it.

All in all I got it to work fast now.. I don't know why there's such a big difference between how vuex-orm and pinia-orm handle withAllRecursive but in my use-case I'd say that using withAllRecursive is completely unusable on pinia-orm. That's just forcing me to specifically load the relationships I need whereas before I was being lazy about it because it didn't really seem to affect performance.

[Graphmaxer]

I created a PR #1773 because this implementation is breaking Vue 2 reactivity