Importing data: selecting the molecule

[PavelBal]

Let us specify what options to map the molecule to imported data set DURING import process should be available. I have a feeling that we are still not clear about that. There are some open questions and I propose to vote how we should proceed.

1. There is a text field for manual entry of "Molecule" value
2. There is no field for manual entry of MW
   • This contradicts with the use cases I drafted in the word document, as I assumed that there is such a field and the user can manually enter MW during import process. However, I now see that this would lead to unnecessary complexity when importing sheets with multiple compounds.
3. There is a checkbox/switch "Map molecule from column"
   2. There is a checkbox/switch "Map MW from column"
4. The user can map a column to molecular weight

"Map molecule from column" is on:

• The text field "Molecule" is disabled
• User selects a column with molecule names.
  • If no valid column is selected (e.g., there are only columns for "Time" and "Measurement" and no additional column is available), the import process cannot continue.
  • If a valid column is selected, xy-points are grouped by the value of "Molecule"-column of this row.
  • Empty "Molecule"-entry creates a data set without assigned molecule (empty metainfo).

"Map molecule from column" is off:

• User has to entry the molecule name in the "Molecule" text field. With this setting, only one data set can be created from the sheet. The user can also pre-select a molecule name from the list of compounds available in the project.
• If the entered molecule name is present in the project, the MW is taken from the compounds-BB. If the molecule is not present in the project OR no molecule has been defined, the MW property of this data set is not set.

No column is mapped to MW

• If the "Molecule"-field of a data set is filled out with the name that is present in the project, its MW is set from the compounds-BB
• If the "Molecule" of the data set does not exist in the project or "Molecule" is empty, the MW property of the data set is empty.

"A column is mapped to MW:
For each data set:

• Name of the molecule is present in the project:
  • MW in excel matches MW in compounds BB -> MW is applied to the data set
  • MW in excel is empty -> MW from compounds BB is applied to the data set
  • MW in excel is different from MW in compounds BB -> Warning to the user, cannot proceed
• Name of the molecule is not present in the project or no name is set:
  • MW from excel is applied to the data set
  • If no MW is defined in excel, the data set has no MW
• If the same dataset has different values for MW, an error is shown.

[Yuri05]

1. There is a text field for manual entry of "Molecule" value

Agree

2. There is no field for manual entry of MW

Agree

3. There is a checkbox/switch "Map molecule from column"

Agree

4. There is a checkbox/switch "Map MW from column"

I think we don't need this. Either MW is mapped or not. s. below

"Map molecule from column" is on:

Agree with your proposal (except that I don't understand what's the difference between "xy-points are grouped by the value of "Molecule"-column of this row" and "the data are grouped by the molecule name" - for me it's the same)

"Map MW from column" is off:

Agree with your proposal. Remark: as I think we don't need the on/off switch for the MW-mapping - this use case would just correspond to "MW column is not mapped"

"Map MW from column" is on:

Agree with your proposal. With the same remark as above (this use case would correspond to "MW column IS mapped")

[msevestre]

Yes, I can live with this.
One could have shown the value of MW as readonly but it becomes complicated in case of multiple compounds

[abdelr]

I would like to add one quick comment. Not a proposal at all, but rather just a piece of information.

The code is general right now so there is no special molecule metadata, just a list of metadata elements. The way to make the importer treat the molecule different is to provide different settings for it. This being said, we could perfectly obtain the same behavior for the Organ for example, i.e. the Organ could also be mapped from a column or from a set of predefined set of values... also any other metadata. Again, I do not want to start a new thread here but keep in mind that this is possible with little to none effort, just a matter of passing the right configuration (which could be easily mimic from the Molecule metadata settings).

Second topic and now, a question:
How do you propose to add the checkbox/switch. Currently we have a column in the grid where the excel column is selected. Now the column header should not be "excel column" anymore but this is not the main issue. The main issue would be to add this checkbox/switch. We could have an editor (as we do for the additional fields) where we select the checkbox/switch and the column/molecule. Keep in mind that this will be similar for the whole column so the Organ editor will also show the same editor which makes little sense if we do not support the same behavior for all (or at least some) metadata. We could also have the checkbox/switch not in the column itself but on top of the table or somewhere detached since it is for only one filed and not for all metadata rows. What if we would omit the checkbox/switch at all? I actually started today to work on this since I thought this part was already clear and I proposed to populate the list of elements for the "excel column" with both, the list of columns union the list of existing molecule names on the project, then what I did was to change the regular combobox to image combobox so it is visually clear when the user selects a column or a value from the list. The application internally will handle this but for the user it will be transparent (except for the icons on the combobox element). So, should we keep it like this or should we move into the checkbox/switch option?

[Yuri05]

@abdelr

The code is general right now so there is no special molecule metadata, just a list of metadata elements. The way to make the importer treat the molecule different is to provide different settings for it. This being said, we could perfectly obtain the same behavior for the Organ for example, i.e. the Organ could also be mapped from a column or from a set of predefined set of values... also any other metadata. Again, I do not want to start a new thread here but keep in mind that this is possible with little to none effort, just a matter of passing the right configuration (which could be easily mimic from the Molecule metadata settings).

I really like this proposal and vote for it! :)

Regarding the 2nd topic:

We could have an editor (as we do for the additional fields) where we select the checkbox/switch and the column/molecule. Keep in mind that this will be similar for the whole column so the Organ editor will also show the same editor which makes little sense if we do not support the same behavior for all (or at least some) metadata.

It's the same behaviour for Molecule, Organ,Compartment,Species, ...: the user can either map a column or select a (single) value from the predefined list of values. Thus having the same editor for all those metadata would make perfectly sense for me.

On the other side,

I proposed to populate the list of elements for the "excel column" with both, the list of columns union the list of existing molecule names on the project, then what I did was to change the regular combobox to image combobox so it is visually clear when the user selects a column or a value from the list.

sounds also reasonable for me. If we:

• sort the entries in the combobox in the way that all predefined values come first (alphabetically sorted); and all excel columns come after that (alphabetically sorted as well) and
• the entries are with the icon as you already implemented

then I think it's a good solution as well.

[StephanSchaller]

Nice one. Thanks Pavel for setting this up!
--> So now we can mark "the accepted" answer?

[abdelr]

I would also like to propose a slight change in the UI.
Pavel proposal is:

No column is mapped to MW

• If the "Molecule"-field of a data set is filled out with the name that is present in the project, its MW is set from the compounds-BB
• If the "Molecule" of the data set does not exist in the project or "Molecule" is empty, the MW property of the data set is empty.

"A column is mapped to MW:
For each data set:

• Name of the molecule is present in the project:

  • MW in excel matches MW in compounds BB -> MW is applied to the data set
  • MW in excel is empty -> MW from compounds BB is applied to the data set
  • MW in excel is different from MW in compounds BB -> Warning to the user, cannot proceed

• Name of the molecule is not present in the project or no name is set:

  • MW from excel is applied to the data set
  • If no MW is defined in excel, the data set has no MW

• If the same dataset has different values for MW, an error is shown.

Additionally to "Excel column" on the grid, we have "Edit extra fileds" which currently is only used for the mapping rows but not for the metadata rows. Since we are now using the "Excel Column" for the alphabetically sorted list of / image ComboBox, we could extend the editor for the Molecule (and potentially other metadata in the future if necessary) with the necessary extra fields. So, in this case, MW could be an extra field of the Molecule metadata and we could only accept values here if the Molecule has been set to a custom value. All in all:

1. Molecule is set to a column of the excel or existing value in the project -> nothing more to define since the MW will be inferred from the molecule
2. Molecule is set to manual value (available also in the ComboBox) -> the editor shows a switch/check box for type the molecule name manually or select the MW from a column.

The idea would be not to allow the user so select a MW if the molecule has already been mapped since it will not add new information at all (the only possibility is to have a MW that will match the one already in the project) but could lead to inconsistent states that will prevent the import process so why should we allow it at all? Also, notice that like with units and lloq, the MW is strongly related to the molecule in this case so it is somehow an extra field of the molecule rather than a new mapping. We could show the users the data like this. Normally the process will flow with a molecule set or empty and you will only need to set the MW manually in some specific cases so no new row for it, only the extra field of the Molecule. Please, let me know what you think.

[Yuri05]

The idea would be not to allow the user so select a MW if the molecule has already been mapped since it will not add new information at all (the only possibility is to have a MW that will match the one already in the project)

It would add additional information if some of the molecules are not part of the project.
That's why I would still allow MW mapping even if the molecule was mapped

[msevestre]

That's why I would still allow MW mapping even if the molecule was mapped

If we allow this, then potentially the wrong MW could come back into the tool (e..g a MW that does not match what we have in PK-Sim or MoBi).

Maybe the check asking the user to verify the MW should happen at the importer level ? That way, the user does not have to reimport everything again if a mistake was made but can simply change the configuration. For that, instead of just a compound name list, we would also have to pass along the MW associated with the compound => a {name, molWeight} list so to speak

[Yuri05]

Maybe the check asking the user to verify the MW should happen at the importer level ? That way, the user does not have to reimport everything again if a mistake was made but can simply change the configuration. For that, instead of just a compound name list, we would also have to pass along the MW associated with the compound => a {name, molWeight} list so to speak

Maybe 😃