fix for the zeros / NA discrepancy #275
Conversation
Fix normalization from 'per protein' to 'per sample'.
| """ | ||
| square_sum_per_row = array.pow(2).sum(axis=1, skipna=True) | ||
|
|
||
| l2_norms = np.sqrt(square_sum_per_row) |
There was a problem hiding this comment.
you could check how https://numpy.org/doc/stable/reference/generated/numpy.linalg.norm.html works with NaNs
There was a problem hiding this comment.
Couldn't get it to work properly, I don't think it works well with NaNs
| n_jobs=2, | ||
| random_state=0, | ||
| verbose=0, # random forest takes a while print progress | ||
| imp = sklearn.ensemble.HistGradientBoostingRegressor( |
There was a problem hiding this comment.
is this the same as RandomForest ? maybe introduce this as another method ?
There was a problem hiding this comment.
It's also based on multiple trees, so I don't know what would be right. I could call it gradient boosting, but this could be less known for non-technical people, although it would be more correct
| @@ -30,45 +31,58 @@ def preprocess_print_info(self): | |||
| print(pd.DataFrame(self.preprocessing_info.items())) | |||
|
|
|||
| def _remove_na_values(self, cut_off): | |||
| if ( | |||
| self.preprocessing_info.get("Missing values were removed") | |||
There was a problem hiding this comment.
I don't know much about this preprocessing_info, but it seems to store information in human-readable keys.
This could lead to problems (say, I access something as self.preprocessing_info.get("Missing values were removed.") (did you spot the trailing dot? ;-))
I would suggest introducing a set of string constants, e.g.
MISSING_VALUES_REMOVED = "Missing values were removed"
somewhere and access this store exclusively through them
(not now, just for the future)
Some of the tools' output which is used in alphapeptstats use zero meaning missing data, some use NA. Previously they were not unified so the preprocessing in
_remove_na_valueswas wrong. Other changes are so the downstream code can work with NAs; style, and tests.