-
Notifications
You must be signed in to change notification settings - Fork 112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OECD_student_teacher_ratio #1058
Draft
swethammkumari
wants to merge
1
commit into
datacommonsorg:master
Choose a base branch
from
swethammkumari:OECD_student_teacher_ratio
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
42 changes: 42 additions & 0 deletions
42
statvar_imports/oecd/oecd_student_teacher_ratio/Places_Resolved.csv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
place_name,dcid | ||
Australia,country/AUS | ||
Austria,country/AUT | ||
Belgium,country/BEL | ||
Canada,country/CAN | ||
Czech Republic,country/CZE | ||
Denmark,country/DNK | ||
Finland,country/FIN | ||
France,country/FRA | ||
Germany,country/DEU | ||
Greece,country/GRC | ||
Hungary,country/HUN | ||
Iceland,country/ISL | ||
Ireland,country/IRL | ||
Italy,country/ITA | ||
Japan,country/JPN | ||
Korea,country/KOR | ||
Luxembourg,country/LUX | ||
Mexico,country/MEX | ||
Netherlands,country/NLD | ||
New Zealand,country/NZL | ||
Norway,country/NOR | ||
Poland,country/POL | ||
Portugal,country/PRT | ||
Slovak Republic,country/SVK | ||
Spain,country/ESP | ||
Sweden,country/SWE | ||
Switzerland,country/CHE | ||
Türkiye,country/TUR | ||
United Kingdom,country/GBR | ||
United States,country/USA | ||
Brazil,country/BRA | ||
Chile,country/CHL | ||
Colombia,country/COL | ||
Costa Rica,country/CRI | ||
Estonia,country/EST | ||
Israel,country/ISR | ||
Latvia,country/LVA | ||
Lithuania,country/LTU | ||
Slovenia,country/SVN | ||
European Union 22 members in OECD,#ignore | ||
OECD - Average,#ignore |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# OECD - Student Teacher Ratio | ||
|
||
- source: https://stats.oecd.org/Index.aspx?datasetcode=EAG_PERS_RATIO | ||
|
||
- how to download data: Manual download from source based on filter - `Student Teacher Ratio`. | ||
|
||
- type of place: Country. | ||
|
||
- statvars: Education | ||
|
||
- years: 2005 to 2020 | ||
|
||
- place_resolution: Resolved manually. | ||
|
||
### How to run: | ||
|
||
`python3 stat_var_processor.py --input_data=<input_file>.csv --pv_map=statvar_imports/oecd/oecd_student_teacher_ratio/pv_map/<filename>_pvmap.csv --places_resolved_csv=statvar_imports/oecd/oecd_student_teacher_ratio/Places_Resolved.csv --config=statvar_imports/oecd/oecd_student_teacher_ratio/metadata.csv --output_path=--output_path=<filepath/filename>` | ||
|
||
#### Example | ||
`python3 stat_var_processor.py --input_data=/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/oecd_student_teacher_ratio.csv --pv_map=/statvar_imports/oecd/oecd_student_teacher_ratio/pv_map.csv --places_resolved_csv=/statvar_imports/oecd/oecd_student_teacher_ratio/Places_Resolved.csv --config=/statvar_imports/oecd/oecd_student_teacher_ratio/metadata.csv --output_path=/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_output/oecd_student_teacher_ratio` |
16 changes: 16 additions & 0 deletions
16
statvar_imports/oecd/oecd_student_teacher_ratio/metadata.csv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
parameter,value | ||
url,https://stats.oecd.org/Index.aspx?datasetcode=EAG_PERS_RATIO# | ||
description,Student-teacher ratio and average class size | ||
#place_type,Country | ||
#places_within, | ||
start_date,2005 | ||
end_date,2020 | ||
release_frequency,1Year | ||
process, | ||
comments, | ||
output_columns,"observationAbout, observationDate, value, variableMeasured, #input" | ||
places_resolved_csv,https://docs.google.com/spreadsheets/d/1MBzEfsn-IJm4iy6LkXvPOce7vWdSktypy0_zwJ3WE0Q/edit?resourcekey=0-c7VdLxzHS5JtzlBVW9o4eQ#gid=1930918947 | ||
mapped_rows,1 | ||
mapped_columns,8 | ||
header_rows,1 | ||
drop_statvars_without_svobs,0 |
29 changes: 29 additions & 0 deletions
29
statvar_imports/oecd/oecd_student_teacher_ratio/pv_map.csv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
key,p1,v1,p2,v2,p3,v3 | ||
Country,observationAbout,{Data},,,, | ||
Year,observationDate,{Number},,,, | ||
Value,value,{Number},,,, | ||
Ratio of students to teaching staff by type of institutions,measuredProperty,count,populationType,Student,measurementDenominator,Count_Teacher | ||
Early childhood education,schoolGradeLevel,Nursery,,,, | ||
Early childhood educational development,schoolGradeLevel,OECD_EarlyChildhoodEducationalDevelopment,,,, | ||
Pre-primary education,schoolGradeLevel,LessThanPrimaryEducation,,,, | ||
Primary education,schoolGradeLevel,PrimaryEducation,,,, | ||
Lower secondary education,schoolGradeLevel,LowerSecondaryEducation,,,, | ||
Secondary education,schoolGradeLevel,SecondaryEducation,,,, | ||
Upper secondary education,schoolGradeLevel,UpperSecondaryEducation,,,, | ||
Upper secondary general education,schoolGradeLevel,UpperSecondaryGeneralEducation,,,, | ||
Upper secondary vocational education,schoolGradeLevel,UpperSecondaryVocationalEducation,,,, | ||
Post-secondary non-tertiary education,schoolGradeLevel,PostSecondaryNonTertaryEducation,,,, | ||
Post-secondary non-tertiary general education,schoolGradeLevel,PostSecondaryNonTertaryGeneralEducation,,,, | ||
Post-secondary non-tertiary vocational education,schoolGradeLevel,PostSecondaryNonTertaryVocationalEducation,,,, | ||
Short-cycle tertiary education,schoolGradeLevel,ShortCycleTertiaryEducation,,,, | ||
Tertiary education,schoolGradeLevel,TertiaryEducation,,,, | ||
"Bachelor's, master's and doctoral or equivalent level",schoolGradeLevel,BachelorsDegree__DoctorateDegree__MastersDegree,,,, | ||
All public and private educational institutions,schoolManagement,GovernmentOwned__PrivatelyOwned,,,, | ||
Public educational institutions,schoolManagement,GovernmentOwned,,,, | ||
All private educational institutions,schoolManagement,PrivatelyOwned,,,, | ||
Government dependent private educational institutions,schoolManagement,GovernmentalAided,,,, | ||
Independent private educational institutions,schoolManagement,IndependentPrivatelyOwned,,,, | ||
Ratio of students to teaching staff by type of institutions,#ignore,ignoring the SV,,,, | ||
Ratio of students to total teaching staff by type of institutions,measuredProperty,count,populationType,Student,measurementDenominator,"-populationType,+populationType=Teacher" | ||
Average class size,measuredProperty,count,populationType,Student,measurementDenominator,"-populationType,+populationType=ClassRoom" | ||
Calculate change in avg class size with 2005 as base year,#ignore,ignoring the SV,,,, |
1 change: 1 addition & 0 deletions
1
...ent_teacher_ratio/test_data/sample_input/.~lock.Student_Teacher_Ratio_OECD_test_data.csv#
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
,swethakumari,swethakumari.c.googlers.com,26.07.2024 09:49,file:///usr/local/google/home/swethakumari/.config/libreoffice/4; | ||
21 changes: 21 additions & 0 deletions
21
...ecd_student_teacher_ratio/test_data/sample_input/Student_Teacher_Ratio_OECD_test_data.csv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
COUNTRY,Country,INDICATOR,Indicator,EDUCATION_LEV,Education level,REF_SECTOR,Reference sector,YEAR,Year,Value,Flag Codes,Flags | ||
HUN,Hungary,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_44,Post-secondary non-tertiary general education,INST_PRIV_GOV,Government dependent private educational institutions,2020,2020,,m,Missing value; data cannot exist | ||
GRC,Greece,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_35,Upper secondary vocational education,INST_T,All public and private educational institutions,2005,2005,,m,Missing value; data cannot exist | ||
JPN,Japan,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_45,Post-secondary non-tertiary vocational education,INST_PRIV_GOV,Government dependent private educational institutions,2019,2019,,m,Missing value; data cannot exist | ||
EST,Estonia,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_35,Upper secondary vocational education,INST_T,All public and private educational institutions,2018,2018,,m,Missing value; data cannot exist | ||
SWE,Sweden,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_0,Early childhood education,INST_PRIV,All private educational institutions,2012,2012,,o,Missing value | ||
EST,Estonia,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_5,Short-cycle tertiary education,INST_PRIV_GOV,Government dependent private educational institutions,2020,2020,,m,Missing value; data cannot exist | ||
DEU,Germany,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_34,Upper secondary general education,INST_PRIV_IND,Independent private educational institutions,2020,2020,,k,Data included in another category | ||
COL,Colombia,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_0,Early childhood education,INST_PUB,Public educational institutions,2010,2010,,o,Missing value | ||
COL,Colombia,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_45,Post-secondary non-tertiary vocational education,INST_PRIV_GOV,Government dependent private educational institutions,2014,2014,,m,Missing value; data cannot exist | ||
ITA,Italy,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_6T8,"Bachelor's, master's and doctoral or equivalent level",INST_PUB,Public educational institutions,2019,2019,,m,Missing value; data cannot exist | ||
COL,Colombia,PERS_AVG_CLASS,Average class size,ISCED11_2,Lower secondary education,INST_PRIV_IND,Independent private educational institutions,2005,2005,,o,Missing value | ||
SWE,Sweden,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_44,Post-secondary non-tertiary general education,INST_PRIV_GOV,Government dependent private educational institutions,2011,2011,,m,Missing value; data cannot exist | ||
NOR,Norway,PERS_AVG_CLASS,Average class size,ISCED11_1,Primary education,INST_T,All public and private educational institutions,2018,2018,,m,Missing value; data cannot exist | ||
MEX,Mexico,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_2_3,Secondary education,INST_T,All public and private educational institutions,2017,2017,28.645,, | ||
PRT,Portugal,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_0,Early childhood education,INST_PRIV,All private educational institutions,2013,2013,,o,Missing value | ||
ESP,Spain,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_3,Upper secondary education,INST_PRIV,All private educational institutions,2019,2019,,m,Missing value; data cannot exist | ||
ISR,Israel,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_01,Early childhood educational development,INST_PUB,Public educational institutions,2012,2012,,m,Missing value; data cannot exist | ||
ISR,Israel,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_01,Early childhood educational development,INST_PRIV_GOV,Government dependent private educational institutions,2005,2005,,o,Missing value | ||
LVA,Latvia,PERS_AVG_CLASS,Average class size,ISCED11_1,Primary education,INST_PUB,Public educational institutions,2010,2010,16.122,, | ||
LTU,Lithuania,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_1,Primary education,INST_PRIV,All private educational institutions,2013,2013,,m,Missing value; data cannot exist |
3 changes: 3 additions & 0 deletions
3
...ts/oecd/oecd_student_teacher_ratio/test_data/sample_output/Student_Teacher_Ratio_OECD.csv
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
observationAbout,observationDate,value,variableMeasured,#input | ||
dcid:country/LVA,2010,16.122,dcid:Count_Student_PrimaryEducation_GovernmentOwned_AsAFractionOf_Count_ClassRoom_PrimaryEducation_GovernmentOwned,/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/Student_Teacher_Ratio_OECD_test_data.csv:20:10 | ||
dcid:country/LVA,2010,16.122,dcid:Count_Student_PrimaryEducation_GovernmentOwned_AsAFractionOf_-populationType,/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/Student_Teacher_Ratio_OECD_test_data.csv:20:10 |
7 changes: 7 additions & 0 deletions
7
...s/oecd/oecd_student_teacher_ratio/test_data/sample_output/Student_Teacher_Ratio_OECD.tmcf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
Node: E:Student_Teacher_Ratio_OECD->E0 | ||
observationAbout: C:Student_Teacher_Ratio_OECD->observationAbout | ||
observationDate: C:Student_Teacher_Ratio_OECD->observationDate | ||
value: C:Student_Teacher_Ratio_OECD->value | ||
variableMeasured: C:Student_Teacher_Ratio_OECD->variableMeasured | ||
typeOf: dcs:StatVarObservation | ||
#input: /usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/Student_Teacher_Ratio_OECD_test_data.csv:20:10 |
28 changes: 28 additions & 0 deletions
28
...cd_student_teacher_ratio/test_data/sample_output/Student_Teacher_Ratio_OECD_stat_vars.mcf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
# Auto generated using command: "stat_var_processor.py --input_data=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/*.csv --output_path=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_output/Student_Teacher_Ratio_OECD --pv_map=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/pv_map.csv --config=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/metadata.csv --places_resolved_csv=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/Places_Resolved.csv --existing_statvar_mcf=stat_vars.mcf" on 2024-07-26 09:47:46.327060 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please remove the commented line |
||
|
||
Node: dcid:Count_ClassRoom_PrimaryEducation_GovernmentOwned | ||
typeOf: dcid:StatisticalVariable | ||
populationType: dcid:ClassRoom | ||
measuredProperty: dcid:count | ||
statType: dcid:measuredValue | ||
schoolGradeLevel: dcid:PrimaryEducation | ||
schoolManagement: dcid:GovernmentOwned | ||
|
||
Node: dcid:Count_Student_PrimaryEducation_GovernmentOwned_AsAFractionOf_-populationType | ||
typeOf: dcid:StatisticalVariable | ||
populationType: dcid:Student | ||
measuredProperty: dcid:count | ||
statType: dcid:measuredValue | ||
measurementDenominator: -populationType | ||
schoolGradeLevel: dcid:PrimaryEducation | ||
schoolManagement: dcid:GovernmentOwned | ||
|
||
Node: dcid:Count_Student_PrimaryEducation_GovernmentOwned_AsAFractionOf_Count_ClassRoom_PrimaryEducation_GovernmentOwned | ||
typeOf: dcid:StatisticalVariable | ||
populationType: dcid:Student | ||
measuredProperty: dcid:count | ||
statType: dcid:measuredValue | ||
measurementDenominator: dcid:Count_ClassRoom_PrimaryEducation_GovernmentOwned | ||
schoolGradeLevel: dcid:PrimaryEducation | ||
schoolManagement: dcid:GovernmentOwned | ||
|
23 changes: 23 additions & 0 deletions
23
...ent_teacher_ratio/test_data/sample_output/Student_Teacher_Ratio_OECD_stat_vars_schema.mcf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
Node: dcid:schoolManagement | ||
typeOf: dcs:Property | ||
name: "schoolManagement" | ||
domainIncludes: dcid:ClassRoom,dcid:Student | ||
rangeIncludes: dcid:EstablishmentOwnershipEnum | ||
|
||
Node: dcid:schoolGradeLevel | ||
typeOf: dcs:Property | ||
name: "schoolGradeLevel" | ||
domainIncludes: dcid:ClassRoom,dcid:Student | ||
rangeIncludes: dcid:SchoolGradeLevelEnum | ||
|
||
Node: dcid:MeasurementDenominatorEnum | ||
typeOf: schema:Class | ||
subClassOf: dcs:Enumeration | ||
name: "MeasurementDenominatorEnum" | ||
isProvisional: dcs:True | ||
|
||
Node: -populationType | ||
typeOf: dcid:MeasurementDenominatorEnum | ||
name: "-population Type" | ||
isProvisional: dcs:True | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please remove the ~lock file for eg: statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/.~lock.Student_Teacher_Ratio_OECD_test_data.csv#