Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CALCITE-6690] Arrow adapter support DECIMAL with precision and scale #4092

Closed
wants to merge 15 commits into from

Conversation

caicancai
Copy link
Member

No description provided.

@caicancai caicancai changed the title [CALCITE-6690] Refactor the Arrow adapter type system [CALCITE-6690] Arrow adapter support DECIMAL with precision and scale Dec 16, 2024
@caicancai caicancai force-pushed the 6690-2 branch 4 times, most recently from 81e9176 to 89d1317 Compare December 16, 2024 14:55

ArrowFieldType(Class<?> clazz) {
this.clazz = clazz;
private ArrowFieldType() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks like a factory, so the name of the class is probably not appropriate.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need me to optimize it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need to optimize anything, just use a name that is appropriate. This is not a type, it is a class which creates types, so it is a factory.

@@ -237,35 +239,38 @@ public void writeArrowDataType(File file) throws IOException {
vectorSchemaRoot.setRowCount(numRows);
for (Field field : vectorSchemaRoot.getSchema().getFields()) {
FieldVector vector = vectorSchemaRoot.getVector(field.getName());
switch (vector.getMinorType()) {
case TINYINT:
switch (field.getName()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why change these?

Copy link
Member Author

@caicancai caicancai Dec 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for taking so long to reply.
This is because I set two fields for decimal type, one with scale 2 and one with scale 3. Since there are two decimal types, it is no longer appropriate to distinguish based on data type.

I chose the field name to distinguish

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, this is just test code. Can you change the file name to ArrowTestData?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mihaibudiu thanks, done

@caicancai caicancai requested a review from mihaibudiu December 19, 2024 13:51
@caicancai caicancai force-pushed the 6690-2 branch 3 times, most recently from ce2e5c5 to 7c22892 Compare December 20, 2024 13:50
ILuffZhe and others added 14 commits December 21, 2024 00:17
…ameter is Boolean, String, Date, Time and Timestamp types
… is too narrow

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>
Signed-off-by: Mihai Budiu <mbudiu@feldera.com>
Signed-off-by: Mihai Budiu <mbudiu@feldera.com>
Signed-off-by: Mihai Budiu <mbudiu@feldera.com>
…y is affected by the order of hosts

Update ElasticsearchSchemaFactory.java

Add unit test

add unit test

Modify code implementation and unit test

Reformat code

Test unit is compatible with Java8

Reformat code

Reformat code

Reformat code

Add jira link
…s, maps without regard to element types

Signed-off-by: Mihai Budiu <mbudiu@feldera.com>
For certain query patterns RelMdUniqueKeys handler generates an
exponentially large number of unique keys that results into crashes
and OOM errors. The limit guards against the combinatorial explosion
that may appear for such use-cases and provides the users of a way to
tune further the upper bound if needed.

Close apache#4089
…hich are arguments of unused aggregate functions
Signed-off-by: Mihai Budiu <mbudiu@feldera.com>
@caicancai
Copy link
Member Author

I'm sorry I messed up the branches, I'll make a new PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants