Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OECD_student_teacher_ratio #1058

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
place_name,dcid
Australia,country/AUS
Austria,country/AUT
Belgium,country/BEL
Canada,country/CAN
Czech Republic,country/CZE
Denmark,country/DNK
Finland,country/FIN
France,country/FRA
Germany,country/DEU
Greece,country/GRC
Hungary,country/HUN
Iceland,country/ISL
Ireland,country/IRL
Italy,country/ITA
Japan,country/JPN
Korea,country/KOR
Luxembourg,country/LUX
Mexico,country/MEX
Netherlands,country/NLD
New Zealand,country/NZL
Norway,country/NOR
Poland,country/POL
Portugal,country/PRT
Slovak Republic,country/SVK
Spain,country/ESP
Sweden,country/SWE
Switzerland,country/CHE
Türkiye,country/TUR
United Kingdom,country/GBR
United States,country/USA
Brazil,country/BRA
Chile,country/CHL
Colombia,country/COL
Costa Rica,country/CRI
Estonia,country/EST
Israel,country/ISR
Latvia,country/LVA
Lithuania,country/LTU
Slovenia,country/SVN
European Union 22 members in OECD,#ignore
OECD - Average,#ignore
20 changes: 20 additions & 0 deletions statvar_imports/oecd/oecd_student_teacher_ratio/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# OECD - Student Teacher Ratio

- source: https://stats.oecd.org/Index.aspx?datasetcode=EAG_PERS_RATIO

- how to download data: Manual download from source based on filter - `Student Teacher Ratio`.

- type of place: Country.

- statvars: Education

- years: 2005 to 2020

- place_resolution: Resolved manually.

### How to run:

`python3 stat_var_processor.py --input_data=<input_file>.csv --pv_map=statvar_imports/oecd/oecd_student_teacher_ratio/pv_map/<filename>_pvmap.csv --places_resolved_csv=statvar_imports/oecd/oecd_student_teacher_ratio/Places_Resolved.csv --config=statvar_imports/oecd/oecd_student_teacher_ratio/metadata.csv --output_path=--output_path=<filepath/filename>`

#### Example
`python3 stat_var_processor.py --input_data=/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/oecd_student_teacher_ratio.csv --pv_map=/statvar_imports/oecd/oecd_student_teacher_ratio/pv_map.csv --places_resolved_csv=/statvar_imports/oecd/oecd_student_teacher_ratio/Places_Resolved.csv --config=/statvar_imports/oecd/oecd_student_teacher_ratio/metadata.csv --output_path=/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_output/oecd_student_teacher_ratio`
16 changes: 16 additions & 0 deletions statvar_imports/oecd/oecd_student_teacher_ratio/metadata.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
parameter,value
url,https://stats.oecd.org/Index.aspx?datasetcode=EAG_PERS_RATIO#
description,Student-teacher ratio and average class size
#place_type,Country
#places_within,
start_date,2005
end_date,2020
release_frequency,1Year
process,
comments,
output_columns,"observationAbout, observationDate, value, variableMeasured, #input"
places_resolved_csv,https://docs.google.com/spreadsheets/d/1MBzEfsn-IJm4iy6LkXvPOce7vWdSktypy0_zwJ3WE0Q/edit?resourcekey=0-c7VdLxzHS5JtzlBVW9o4eQ#gid=1930918947
mapped_rows,1
mapped_columns,8
header_rows,1
drop_statvars_without_svobs,0
29 changes: 29 additions & 0 deletions statvar_imports/oecd/oecd_student_teacher_ratio/pv_map.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
key,p1,v1,p2,v2,p3,v3
Country,observationAbout,{Data},,,,
Year,observationDate,{Number},,,,
Value,value,{Number},,,,
Ratio of students to teaching staff by type of institutions,measuredProperty,count,populationType,Student,measurementDenominator,Count_Teacher
Early childhood education,schoolGradeLevel,Nursery,,,,
Early childhood educational development,schoolGradeLevel,OECD_EarlyChildhoodEducationalDevelopment,,,,
Pre-primary education,schoolGradeLevel,LessThanPrimaryEducation,,,,
Primary education,schoolGradeLevel,PrimaryEducation,,,,
Lower secondary education,schoolGradeLevel,LowerSecondaryEducation,,,,
Secondary education,schoolGradeLevel,SecondaryEducation,,,,
Upper secondary education,schoolGradeLevel,UpperSecondaryEducation,,,,
Upper secondary general education,schoolGradeLevel,UpperSecondaryGeneralEducation,,,,
Upper secondary vocational education,schoolGradeLevel,UpperSecondaryVocationalEducation,,,,
Post-secondary non-tertiary education,schoolGradeLevel,PostSecondaryNonTertaryEducation,,,,
Post-secondary non-tertiary general education,schoolGradeLevel,PostSecondaryNonTertaryGeneralEducation,,,,
Post-secondary non-tertiary vocational education,schoolGradeLevel,PostSecondaryNonTertaryVocationalEducation,,,,
Short-cycle tertiary education,schoolGradeLevel,ShortCycleTertiaryEducation,,,,
Tertiary education,schoolGradeLevel,TertiaryEducation,,,,
"Bachelor's, master's and doctoral or equivalent level",schoolGradeLevel,BachelorsDegree__DoctorateDegree__MastersDegree,,,,
All public and private educational institutions,schoolManagement,GovernmentOwned__PrivatelyOwned,,,,
Public educational institutions,schoolManagement,GovernmentOwned,,,,
All private educational institutions,schoolManagement,PrivatelyOwned,,,,
Government dependent private educational institutions,schoolManagement,GovernmentalAided,,,,
Independent private educational institutions,schoolManagement,IndependentPrivatelyOwned,,,,
Ratio of students to teaching staff by type of institutions,#ignore,ignoring the SV,,,,
Ratio of students to total teaching staff by type of institutions,measuredProperty,count,populationType,Student,measurementDenominator,"-populationType,+populationType=Teacher"
Average class size,measuredProperty,count,populationType,Student,measurementDenominator,"-populationType,+populationType=ClassRoom"
Calculate change in avg class size with 2005 as base year,#ignore,ignoring the SV,,,,
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
,swethakumari,swethakumari.c.googlers.com,26.07.2024 09:49,file:///usr/local/google/home/swethakumari/.config/libreoffice/4;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please remove the ~lock file for eg: statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/.~lock.Student_Teacher_Ratio_OECD_test_data.csv#

Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
COUNTRY,Country,INDICATOR,Indicator,EDUCATION_LEV,Education level,REF_SECTOR,Reference sector,YEAR,Year,Value,Flag Codes,Flags
HUN,Hungary,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_44,Post-secondary non-tertiary general education,INST_PRIV_GOV,Government dependent private educational institutions,2020,2020,,m,Missing value; data cannot exist
GRC,Greece,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_35,Upper secondary vocational education,INST_T,All public and private educational institutions,2005,2005,,m,Missing value; data cannot exist
JPN,Japan,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_45,Post-secondary non-tertiary vocational education,INST_PRIV_GOV,Government dependent private educational institutions,2019,2019,,m,Missing value; data cannot exist
EST,Estonia,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_35,Upper secondary vocational education,INST_T,All public and private educational institutions,2018,2018,,m,Missing value; data cannot exist
SWE,Sweden,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_0,Early childhood education,INST_PRIV,All private educational institutions,2012,2012,,o,Missing value
EST,Estonia,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_5,Short-cycle tertiary education,INST_PRIV_GOV,Government dependent private educational institutions,2020,2020,,m,Missing value; data cannot exist
DEU,Germany,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_34,Upper secondary general education,INST_PRIV_IND,Independent private educational institutions,2020,2020,,k,Data included in another category
COL,Colombia,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_0,Early childhood education,INST_PUB,Public educational institutions,2010,2010,,o,Missing value
COL,Colombia,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_45,Post-secondary non-tertiary vocational education,INST_PRIV_GOV,Government dependent private educational institutions,2014,2014,,m,Missing value; data cannot exist
ITA,Italy,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_6T8,"Bachelor's, master's and doctoral or equivalent level",INST_PUB,Public educational institutions,2019,2019,,m,Missing value; data cannot exist
COL,Colombia,PERS_AVG_CLASS,Average class size,ISCED11_2,Lower secondary education,INST_PRIV_IND,Independent private educational institutions,2005,2005,,o,Missing value
SWE,Sweden,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_44,Post-secondary non-tertiary general education,INST_PRIV_GOV,Government dependent private educational institutions,2011,2011,,m,Missing value; data cannot exist
NOR,Norway,PERS_AVG_CLASS,Average class size,ISCED11_1,Primary education,INST_T,All public and private educational institutions,2018,2018,,m,Missing value; data cannot exist
MEX,Mexico,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_2_3,Secondary education,INST_T,All public and private educational institutions,2017,2017,28.645,,
PRT,Portugal,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_0,Early childhood education,INST_PRIV,All private educational institutions,2013,2013,,o,Missing value
ESP,Spain,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_3,Upper secondary education,INST_PRIV,All private educational institutions,2019,2019,,m,Missing value; data cannot exist
ISR,Israel,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_01,Early childhood educational development,INST_PUB,Public educational institutions,2012,2012,,m,Missing value; data cannot exist
ISR,Israel,PERS_RATIO_INST,Ratio of students to teaching staff by type of institutions,ISCED11_01,Early childhood educational development,INST_PRIV_GOV,Government dependent private educational institutions,2005,2005,,o,Missing value
LVA,Latvia,PERS_AVG_CLASS,Average class size,ISCED11_1,Primary education,INST_PUB,Public educational institutions,2010,2010,16.122,,
LTU,Lithuania,PERS_RATIO_MANA,Ratio of students to total teaching staff by type of institutions,ISCED11_1,Primary education,INST_PRIV,All private educational institutions,2013,2013,,m,Missing value; data cannot exist
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
observationAbout,observationDate,value,variableMeasured,#input
dcid:country/LVA,2010,16.122,dcid:Count_Student_PrimaryEducation_GovernmentOwned_AsAFractionOf_Count_ClassRoom_PrimaryEducation_GovernmentOwned,/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/Student_Teacher_Ratio_OECD_test_data.csv:20:10
dcid:country/LVA,2010,16.122,dcid:Count_Student_PrimaryEducation_GovernmentOwned_AsAFractionOf_-populationType,/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/Student_Teacher_Ratio_OECD_test_data.csv:20:10
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Node: E:Student_Teacher_Ratio_OECD->E0
observationAbout: C:Student_Teacher_Ratio_OECD->observationAbout
observationDate: C:Student_Teacher_Ratio_OECD->observationDate
value: C:Student_Teacher_Ratio_OECD->value
variableMeasured: C:Student_Teacher_Ratio_OECD->variableMeasured
typeOf: dcs:StatVarObservation
#input: /usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/Student_Teacher_Ratio_OECD_test_data.csv:20:10
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Auto generated using command: "stat_var_processor.py --input_data=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_input/*.csv --output_path=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/test_data/sample_output/Student_Teacher_Ratio_OECD --pv_map=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/pv_map.csv --config=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/metadata.csv --places_resolved_csv=/usr/local/google/home/swethakumari/DC_26_july/data/statvar_imports/oecd/oecd_student_teacher_ratio/Places_Resolved.csv --existing_statvar_mcf=stat_vars.mcf" on 2024-07-26 09:47:46.327060
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the commented line


Node: dcid:Count_ClassRoom_PrimaryEducation_GovernmentOwned
typeOf: dcid:StatisticalVariable
populationType: dcid:ClassRoom
measuredProperty: dcid:count
statType: dcid:measuredValue
schoolGradeLevel: dcid:PrimaryEducation
schoolManagement: dcid:GovernmentOwned

Node: dcid:Count_Student_PrimaryEducation_GovernmentOwned_AsAFractionOf_-populationType
typeOf: dcid:StatisticalVariable
populationType: dcid:Student
measuredProperty: dcid:count
statType: dcid:measuredValue
measurementDenominator: -populationType
schoolGradeLevel: dcid:PrimaryEducation
schoolManagement: dcid:GovernmentOwned

Node: dcid:Count_Student_PrimaryEducation_GovernmentOwned_AsAFractionOf_Count_ClassRoom_PrimaryEducation_GovernmentOwned
typeOf: dcid:StatisticalVariable
populationType: dcid:Student
measuredProperty: dcid:count
statType: dcid:measuredValue
measurementDenominator: dcid:Count_ClassRoom_PrimaryEducation_GovernmentOwned
schoolGradeLevel: dcid:PrimaryEducation
schoolManagement: dcid:GovernmentOwned

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
Node: dcid:schoolManagement
typeOf: dcs:Property
name: "schoolManagement"
domainIncludes: dcid:ClassRoom,dcid:Student
rangeIncludes: dcid:EstablishmentOwnershipEnum

Node: dcid:schoolGradeLevel
typeOf: dcs:Property
name: "schoolGradeLevel"
domainIncludes: dcid:ClassRoom,dcid:Student
rangeIncludes: dcid:SchoolGradeLevelEnum

Node: dcid:MeasurementDenominatorEnum
typeOf: schema:Class
subClassOf: dcs:Enumeration
name: "MeasurementDenominatorEnum"
isProvisional: dcs:True

Node: -populationType
typeOf: dcid:MeasurementDenominatorEnum
name: "-population Type"
isProvisional: dcs:True

Loading