Skip to content
This repository has been archived by the owner on Jan 2, 2024. It is now read-only.

Feature/#336 - Filter numpy array exposed type #768

Merged
merged 11 commits into from
Nov 3, 2023

Conversation

trgiangdo
Copy link
Member

#336

A few things we need to consider:

  • We should probably also consider to expose the Operator and JoinOperator at the taipy init level.
  • If the exposed type is numpy, then the key is the index of the column. If the user wants to filter by row value, we need to find a way to allow that.

@trgiangdo trgiangdo changed the title Feature/#336 - Filter numpy array exposed type [BLOCKED UNTIL 3.0 RELEASE] Feature/#336 - Filter numpy array exposed type Oct 2, 2023
@github-actions
Copy link

github-actions bot commented Oct 2, 2023

☂️ Python Cov

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
8175 7760 95% 85% 🟢

New Files

No new covered files...

Modified Files

File Coverage Status
src/taipy/core/_entity/_migrate/_utils.py 85% 🟢
src/taipy/core/data/_abstract_sql.py 92% 🟢
src/taipy/core/data/_filter.py 91% 🟢
src/taipy/core/data/csv.py 95% 🟢
src/taipy/core/data/data_node.py 99% 🟢
src/taipy/core/data/parquet.py 98% 🟢
src/taipy/core/data/sql.py 97% 🟢
src/taipy/core/data/sql_table.py 98% 🟢
TOTAL 94% 🟢

updated for commit: bb54978 by action🐍

toan-quach
toan-quach previously approved these changes Oct 3, 2023
Copy link
Member

@toan-quach toan-quach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Member

@jrobinAV jrobinAV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. excel.py should also be updated with the return type of method _read_as_numpy.

  2. I would like to challenge the design:

    • I am not sure to understand the purpose of the _FilterDataNode class
    • I would like to move the whole filtering logic out of the DataNode class. The datanode.filter method could delegate the filtering logic to a dedicated class like Filter. What do you think?

@trgiangdo
Copy link
Member Author

  1. That method of excel can also return a dict of array, I should update it as well.

2.1. Me neither 🥲 I think we can remove it since it is not documented at all.

2.2. I agree with moving the logic to a dedicate file. And if I understand correctly, the filter() method should still return the filtered data directly, not returning the Filter class right?

@trgiangdo trgiangdo changed the title [BLOCKED UNTIL 3.0 RELEASE] Feature/#336 - Filter numpy array exposed type Feature/#336 - Filter numpy array exposed type Oct 30, 2023
@trgiangdo trgiangdo force-pushed the feature/#336-filter-numpy-array branch from fb09f79 to d83b52d Compare November 1, 2023 18:16
@trgiangdo trgiangdo marked this pull request as ready for review November 1, 2023 18:17
@trgiangdo
Copy link
Member Author

In this PR:

New feature:

  • Ability to filter numpy exposed_type

Refactoring:

  • Refactor the _FilterDataNode class. Move the logic of the DataNode.filter() method to the _FilterDataNode class.
  • Simplify the getitem method of DataNode, make it consistent with DataNode.filter() method.
  • Add unit tests for filtering all data node types.

Bug fixes:

  • Fix unable to to filter excel data node (because excel data node return a dictionary of exposed_type, not the direct exposed_type).
  • Fix unable to filter JSONDataNode when the data is a list of dictionaries.
  • Fix unable to filter a list of objects (custom exposed type) when some object has an attribute that others don't.
  • Fix unable to write to SQLDataNode due to not converting the string query into SQL text query (raised by issue BUG-Can not write to a SQL Data node #816).

jrobinAV
jrobinAV previously approved these changes Nov 3, 2023
src/taipy/core/data/_filter.py Outdated Show resolved Hide resolved
@trgiangdo trgiangdo merged commit 72f4f69 into develop Nov 3, 2023
41 of 42 checks passed
@trgiangdo trgiangdo deleted the feature/#336-filter-numpy-array branch November 3, 2023 16:30
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants