Difference from version 11 when directly setting a onetomany collection and calling update #3285

tom-williams · 2023-12-05T13:18:22Z

tom-williams
Dec 5, 2023

I am in the process of upgrading from version 11.15.12 to 13.25.1, and am confused by a difference of behavior when setting a @onetomany annotated collection directly and calling update.
I have a collection:

    @OneToMany(cascade = CascadeType.ALL, mappedBy = "parentBlock")
    private List<StimuliBlockChild> children;

An existing parentBlock has 2 existing children.
I directly set the one to many collection to 6 newly created children objects.

    parentBlock.setChildren(aListOfSixNewlyCreatedChildren);
    db.update(parentBlock);

Without any code changes -> this is the log output from setting loglevel to TRACE for version 11.15.12:

DEBUG [2023-12-05 13:54:41,051] io.ebeaninternal.server.persist.DefaultPersister: Update skipped as bean is unchanged:com.imotions.model.stimulusblock.StimuliBlock@df0951df
DEBUG [2023-12-05 13:54:41,051] io.ebean.SQL: txn[1370] delete from stimuli_block_child where parent_block_id = ?
DEBUG [2023-12-05 13:54:41,054] io.ebean.SUM: txn[1370] DeleteSql table[stimuli_block_child] rows[2] bind[73a6a4f0-3949-43e6-863c-c2d713da741e]
DEBUG [2023-12-05 13:54:41,055] io.ebean.SUM: txn[1370] BatchControl flush [StimuliBlockChild:101 i:6]
DEBUG [2023-12-05 13:54:41,056] io.ebean.SQL: txn[1370] insert into stimuli_block_child (id, fixed, block_order, parent_block_id, block_id, stimuli_id) values (?,?,?,?,?,?)
DEBUG [2023-12-05 13:54:41,057] io.ebean.SQL: txn[1370] insert into stimuli_block_child (id, fixed, block_order, parent_block_id, block_id, stimuli_id) values (?,?,?,?,?,?)
DEBUG [2023-12-05 13:54:41,057] io.ebean.SQL: txn[1370] insert into stimuli_block_child (id, fixed, block_order, parent_block_id, block_id, stimuli_id) values (?,?,?,?,?,?)
DEBUG [2023-12-05 13:54:41,057] io.ebean.SQL: txn[1370] insert into stimuli_block_child (id, fixed, block_order, parent_block_id, block_id, stimuli_id) values (?,?,?,?,?,?)
DEBUG [2023-12-05 13:54:41,058] io.ebean.SQL: txn[1370] insert into stimuli_block_child (id, fixed, block_order, parent_block_id, block_id, stimuli_id) values (?,?,?,?,?,?)
DEBUG [2023-12-05 13:54:41,058] io.ebean.SQL: txn[1370] insert into stimuli_block_child (id, fixed, block_order, parent_block_id, block_id, stimuli_id) values (?,?,?,?,?,?)
DEBUG [2023-12-05 13:54:41,064] io.ebean.SUM: txn[1370] Inserted [StimuliBlockChild] [dadb00c6-6736-4138-a965-0c1879807927]
DEBUG [2023-12-05 13:54:41,064] io.ebean.SUM: txn[1370] Inserted [StimuliBlockChild] [910281ba-c51f-4e02-b318-7bce9a2df385]
DEBUG [2023-12-05 13:54:41,064] io.ebean.SUM: txn[1370] Inserted [StimuliBlockChild] [c314936f-7997-4b49-bbc9-f2952fbe6de6]
DEBUG [2023-12-05 13:54:41,064] io.ebean.SUM: txn[1370] Inserted [StimuliBlockChild] [77d566ec-d63f-44ef-bc6a-fc77169d4e5d]
DEBUG [2023-12-05 13:54:41,064] io.ebean.SUM: txn[1370] Inserted [StimuliBlockChild] [749a5570-e247-4981-888a-c7ea1fa626e5]
DEBUG [2023-12-05 13:54:41,064] io.ebean.SUM: txn[1370] Inserted [StimuliBlockChild] [37cf2e4c-b7ba-485e-a4b0-c8891cf12b4f]
DEBUG [2023-12-05 13:54:41,067] io.ebean.TXN: txn[1370] Commit

Where it first deletes all child elements from the parent block and inserts the 6 new elements.

However in version 13.25.1:

DEBUG [2023-12-05 13:51:15,706] io.ebean.internal: Update skipped as bean is unchanged: StimuliBlock@0(id:16c0f481-3054-414e-989f-728012d4f926, ...
DEBUG [2023-12-05 13:51:15,707] io.ebean.SUM: txn[1346] BatchControl flush [StimuliBlockChild:100 i:6]
DEBUG [2023-12-05 13:51:15,708] io.ebean.SQL: txn[1346] insert into stimuli_block_child (id, fixed, block_order, parent_block_id, block_id, stimuli_id) values (?,?,?,?,?,?)
DEBUG [2023-12-05 13:51:15,708] io.ebean.SQL: txn[1346]  -- bind(5f27bb0e-87a8-4906-8fd1-47a9154d205a,true,0,16c0f481-3054-414e-989f-728012d4f926,null,9552fbb2-6ffc-4372-b1ea-a070ecad78c2)
DEBUG [2023-12-05 13:51:15,709] io.ebean.SQL: txn[1346]  -- bind(60db59ef-cd5e-4eb8-a2ab-7062e95402fe,true,1,16c0f481-3054-414e-989f-728012d4f926,null,72717438-c7ac-4c3d-8d0a-8f79ac82ce13)
DEBUG [2023-12-05 13:51:15,709] io.ebean.SQL: txn[1346]  -- bind(7d675fc2-651f-42f8-9bb2-8b72847139b8,true,2,16c0f481-3054-414e-989f-728012d4f926,null,73de9427-74b1-4377-8dc2-e0753f0e9cff)
DEBUG [2023-12-05 13:51:15,710] io.ebean.SQL: txn[1346]  -- bind(675dea17-41fd-47ee-91d3-f9669e1ab69c,true,3,16c0f481-3054-414e-989f-728012d4f926,null,d94cd5f4-a3c6-4fb5-96c8-f9b487693428)
DEBUG [2023-12-05 13:51:15,710] io.ebean.SQL: txn[1346]  -- bind(103625a8-c0e4-4fa2-bdf5-988616dced5b,true,4,16c0f481-3054-414e-989f-728012d4f926,null,758348ed-ef3b-41b9-9938-08b44c22a812)
DEBUG [2023-12-05 13:51:15,711] io.ebean.SQL: txn[1346]  -- bind(944e9a13-0da2-409f-a3fc-56d75c4fd671,true,5,16c0f481-3054-414e-989f-728012d4f926,null,a8df5483-cea5-40d6-bfd8-3a8369ee644f)
DEBUG [2023-12-05 13:51:15,715] io.ebean.SQL: txn[1346]  -- executeBatch() size:6 sql:insert into stimuli_block_child (id, fixed, block_order, parent_block_id, block_id, stimuli_id) values (?,?,?,?,?,?)
DEBUG [2023-12-05 13:51:15,716] io.ebean.SUM: txn[1346] Inserted [StimuliBlockChild] [5f27bb0e-87a8-4906-8fd1-47a9154d205a]
DEBUG [2023-12-05 13:51:15,716] io.ebean.SUM: txn[1346] Inserted [StimuliBlockChild] [60db59ef-cd5e-4eb8-a2ab-7062e95402fe]
DEBUG [2023-12-05 13:51:15,716] io.ebean.SUM: txn[1346] Inserted [StimuliBlockChild] [7d675fc2-651f-42f8-9bb2-8b72847139b8]
DEBUG [2023-12-05 13:51:15,716] io.ebean.SUM: txn[1346] Inserted [StimuliBlockChild] [675dea17-41fd-47ee-91d3-f9669e1ab69c]
DEBUG [2023-12-05 13:51:15,717] io.ebean.SUM: txn[1346] Inserted [StimuliBlockChild] [103625a8-c0e4-4fa2-bdf5-988616dced5b]
DEBUG [2023-12-05 13:51:15,717] io.ebean.SUM: txn[1346] Inserted [StimuliBlockChild] [944e9a13-0da2-409f-a3fc-56d75c4fd671]
DEBUG [2023-12-05 13:51:15,719] io.ebean.TXN: txn[1346] Commit

Here it just inserts the newly created elements, so when you fetch the parentBlock from the database again, It now has 8 children, instead of the 6 it has been set to.

Is this intended new behavior? And is it possible to make it behave as it used to. Otherwise our legacy code will behave in an incorrect manner.

I suspect that the previous version behaved incorrectly, and is it related to the @onetomany annotation attribute orphanRemoval by default is set to false?

jnehlmeier · 2023-12-05T16:20:33Z

jnehlmeier
Dec 5, 2023

Yes, you need to use orphanRemoval correctly.

You always have to decide whether or not the many side can exist on its own. For example in the classic Order -> OrderItem scenario an OrderItem can only exist if the Order exists. So if you replace all OrderItems in an Order you should set orphanRemoval = true to tell the ORM to delete the old, now orphaned, OrderItems.

But there is also a scenario like Meeting -> Person. If you replace all persons in a meeting you don't want to delete all old persons just because they do not attend in a given meeting anymore. In that case orphanRemoval needs to be false.

Old ebean versions also have an annotation called @PrivateOwned which was basically the same as orphanRemoval. This has been deleted in 12.x in favor of orphanRemoval. There were numerous issues regarding orphanRemoval with some still being open: https://github.com/ebean-orm/ebean/issues?q=is%3Aissue+orphanRemoval+is%3Aopen

This might also affect OneToOne as it also has orphanRemoval defaulting to false. I don't know how ebean behaves in that case. But generally you would need to decide on a case by case basis whether or not you want to delete the orphaned entity if you set a OneToOne relation to null.

I don't think you will get back the old mode and you would need to carefully update your code. See release notes https://github.com/ebean-orm/ebean/releases/tag/ebean-12.1.1

0 replies

tom-williams · 2023-12-06T09:04:02Z

tom-williams
Dec 6, 2023
Author

Thanks for your answer.

I do understand the implications of the different types of behavior, however, it is a relatively large legacy code base that I then need to go through on a per annotation basis and see all usages, to see where I am affected by this, if I understand correctly.

As far as I can tell the javax.persistence @onetomany implementation in version 11.15.12 still had the orphanRemoval default set to false, but it doesn't actually behave that way. Reading the release notes again, I still cannot see that this was deliberately changed.

What I was hoping to ascertain was whether or not I could just set orphanRemoval to true on all the annotations, and then have the same behavior as I do currently, without any other repercussions.

It does seem weird to me that default behavior on a @onetomany relationship with cascade = CascadeType.ALL has:
parentBlock.setChildren(listOfXNumberOfChildren)

behave the same as:
parentBlock.getChildren().addAll(listOfXNumberOfChildren)

My biggest concern is, if the above is true and not a misunderstanding on my part, are there any other changes that have been made in other relationship annotations, specifically @manytomany which doesn't have an attribute of orphanRemoval, but if I recall correctly if I had the following relationship:

    @ManyToMany
    @JoinTable(name = "SEGMENT_RESPONDENT")
    private List<Respondent> respondents;

Then the following operation:
segment.getRespondents().add(aNewRespondent);
db.update(segment)

Resulted in:
delete from segment_respondent where segment.id = <segment.getId()>

followed by inserting all of the elements in the list.

This behavior I remember because we recently had a DuplicateKeyException being thrown in a race condition when adding two separate Respondents in different concurrent transactions with a isolation level of READ_COMMITTED against a postgresql database.
Here if both transactions are trying to delete the same rows (the existing respondents in the list), the first transaction will row lock all the elements in the collection for deletion then add them all again plus the new one.

When the first transaction commits, the second transaction will not execute the delete, because the rows it was waiting to delete, were deleted by the first transaction (from the postgres doc: If the first updater commits, the second updater will ignore the row if the first updater deleted it), but it will try and add the elements of the collection again, so try to add the existing respondents to the segment once again which are already there causing the exception.

4 replies

rbygrave Dec 6, 2023
Maintainer

I still cannot see that this was deliberately changed.

It was, this was a deliberate change, this was the reason for the major version bump to 12.x.

The change was to bring ebean behaviour in line with the JPA spec around orphanRemoval.

I could just set orphanRemoval to true on all the annotations, and then have the same behavior as I do currently, without any other repercussions.

Pretty much (excluding "stateless updates" which as ebean only feature not part of JPA).

are there any other changes that have been made in other relationship annotations, specifically @manytomany

No there have not changes to manyToMany. There have been changes to @DbArray around supporting null (vs non null array with empty arrays).

tom-williams Dec 7, 2023
Author

Thanks. That is really helpful.

excluding "stateless updates"

regarding use of stateless update, I assume that the line in the release note linked to above

#1824 Stateless updates - Remove update deleteMissingChildren option, instead always use orphanRemoval behaviour change breaking-api

that this is the only change regarding this?!

Again thanks for your quick replies and help!

rbygrave Dec 7, 2023
Maintainer

this is the only change regarding this

I don't think I understand your question so I'll add some comments and see if that helps.

There is a reason-for-version-bump tag and we can see all the issues that have this tag like:
https://github.com/ebean-orm/ebean/issues?q=is%3Aissue+is%3Aclosed+label%3Areason-for-version-bump
... and this is a way people can review those more interesting issues when upgrading etc [across all versions since 12.1.1]. People could read all the release notes but I suspect many people don't do that and this is an alternative to look at each of the issues with this reason-for-version-bump tag.

12.1.1

This is the first 12.x release version, there is no 12.0.0 etc. The 12.1.1 release notes are for this major version change and it's all around the behaviour change on orphanRemoval.

only change / release notes

The release notes are best effort but yes they should include everything. There is no intention to leave anything out of the release notes or hide any changes. Back at 12.1.1 those release notes were manually created but these days the release notes are automatically generated [but then I do tend to add some organisation to them when we bump major or minor].

Note that 12.1.1 is now just over 4 years ago so things are "not as fresh" in the memory - but there is no reason to think those release notes are not accurate or representative etc. For more details we could look at the specific issues and related PRs/commits/diff and maybe that can help if the summary in the release notes isn't clear enough?

Also thanks Jens for answering this, I thought your comments were really good.

tom-williams Dec 9, 2023
Author

I hope you both know that I appreciate any time and effort you spend on trying to answer my questions.

I understand it has been a long time since these changes were made, and I will look at the PR's to clear up any questions I have.
I think this is most an issue of how we use and change/edit a collection where we just set the collection to what we want and hope all goes well behind the scenes.

But thanks for explicitly clearing up what the breaking change was.

tom-williams · 2023-12-12T13:39:19Z

tom-williams
Dec 12, 2023
Author

I think that the PR that fixed this issue:
#1472

is the main issue that our legacy code is having trouble with.

I have a similar relationship as before:

  public class Study {
    @Id
    UUID id;
    
    @OneToMany(CascadeType.ALL)
    List<Segment> segments;
  }
  
  public class Segment {
    @Id
    UUID id;
    
    @ManyToOne
    Study study;
  }

where an update of of an incoming Study object with an existing Segment object in its segments list
db.update(study);

Before with version 11.15.12 resulted in:

  select t0.id from segment t0 where study_id=?  and not (t0.id = any(?)) ; --bind(f48c37ea-9045-4209-9066-2d71530fa841,Array[1]={f15d19cc-8856-4d7a-895f-a52b4b132544})
  update segment set name=?, PROC_STATE=?, study_id=? where id=?; --bind(All Respondents,PENDING,f48c37ea-9045-4209-9066-2d71530fa841,f15d19cc-8856-4d7a-895f-a52b4b132544)

Now produces:

select t0.id from segment t0 where study_id=?  and not (t0.id = any(?)); --bind(11690b60-43a8-4b83-87c1-5bc6137bdaf7,Array[1]={a5b1e33e-cd9c-4789-b262-fe0e103f0990})
insert into segment (id, name, PROC_STATE, histogram_url, study_id)
values (?,?,?,?,?); -- bind(a5b1e33e-cd9c-4789-b262-fe0e103f0990,All Respondents,PENDING,null,11690b60-43a8-4b83-87c1-5bc6137bdaf7)

Which results in a DuplicateKeyException.

But does this make sense for objects whose @id's are generated as it automatically is with a UUID?
The incoming segment objects already have an Id that was generated when they were created by an earlier request.

If I understand the bug report and the description, this is for when the id's are provided and not generated (is this what is meant by managed id?). I, therefore, tried to add the @GeneratedValue after the @Id annotation, to see if that made a difference, but id did not.

4 replies

rbygrave Dec 13, 2023
Maintainer

Note sure but you might want to set idGeneratorAutomatic to false in DatabaseConfig.

tom-williams Dec 13, 2023
Author

Thanks. I will try that.

rbygrave Dec 13, 2023
Maintainer

The next steps I think are to look for a similar test (stateless update using UUID id's cascading a OneToMany) and compare &/or create an example [with a failing test] that shows the issue [typically by forking https://github.com/ebean-orm-examples/example-minimal ]

tom-williams Dec 14, 2023
Author

I tried setting the value in DatabaseConfig without effect.

I have forked the example-minimal project and created a failing test.
What is the next step?

As far as I can determine, this issue is the result of the commit that fixed the issue #1472
And maybe this is now behaving more correctly than before, and I will have to try and deal with it in some other way.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Difference from version 11 when directly setting a onetomany collection and calling update #3285

{{title}}

Replies: 3 comments 8 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Difference from version 11 when directly setting a onetomany collection and calling update #3285

tom-williams Dec 5, 2023

Replies: 3 comments · 8 replies

jnehlmeier Dec 5, 2023

tom-williams Dec 6, 2023 Author

rbygrave Dec 6, 2023 Maintainer

tom-williams Dec 7, 2023 Author

rbygrave Dec 7, 2023 Maintainer

tom-williams Dec 9, 2023 Author

tom-williams Dec 12, 2023 Author

rbygrave Dec 13, 2023 Maintainer

tom-williams Dec 13, 2023 Author

rbygrave Dec 13, 2023 Maintainer

tom-williams Dec 14, 2023 Author

tom-williams
Dec 5, 2023

Replies: 3 comments 8 replies

jnehlmeier
Dec 5, 2023

tom-williams
Dec 6, 2023
Author

rbygrave Dec 6, 2023
Maintainer

tom-williams Dec 7, 2023
Author

rbygrave Dec 7, 2023
Maintainer

tom-williams Dec 9, 2023
Author

tom-williams
Dec 12, 2023
Author

rbygrave Dec 13, 2023
Maintainer

tom-williams Dec 13, 2023
Author

rbygrave Dec 13, 2023
Maintainer

tom-williams Dec 14, 2023
Author