Skip to content
This repository has been archived by the owner on Jul 7, 2022. It is now read-only.

Adding preliminary support for converting json schema to apache parquet schema. #19

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

lvde21
Copy link

@lvde21 lvde21 commented Oct 9, 2020

Related to issue: #2
Objective: hoping to get advice about approach.

Simple test:

EDIT: fixed a small bug in object field names.

$ aptos convert -format parquet tests/schema/person 
message ParquetSchema {
  optional group Person {
    required binary firstName (UTF8);
    optional binary lastName (UTF8);
    required int32 age;
    optional group Address {
      optional binary street_name (UTF8);
      optional int32 zipcode;
    }
  }
}

lalade added 2 commits October 9, 2020 16:02
…et schema.

Related to issue: pennsignals#2
Objective: hoping to get advice about approach.

Simple test:
$ aptos convert -format parquet tests/schema/person
message ParquetSchema {
  Person {
    required binary firstName (UTF8);
    optional binary lastName (UTF8);
    required int32 age;
    Address {
      optional binary street_name (UTF8);
      optional int32 zipcode;
    }
  }
}
Sample run:
$ aptos convert -format parquet tests/schema/person
message ParquetSchema {
  optional group Person {
    required binary firstName (UTF8);
    optional binary lastName (UTF8);
    required int32 age;
    optional group Address {
      optional binary street_name (UTF8);
      optional int32 zipcode;
    }
  }
}
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant