diff --git a/docs/converters/vision_converter.md b/docs/converters/vision_converter.md index 97b9aaae..2146908a 100644 --- a/docs/converters/vision_converter.md +++ b/docs/converters/vision_converter.md @@ -43,4 +43,9 @@ Some components are yet to be modeled for conversions because they might not hav - The conversions for load behaviors of `industry`, `residential` and `business` are not yet modeled. The load behaviors usually do not create a significant difference in power-flow results for most grids when the voltage at bus is close to `1 pu`. Hence, the conversion of the mentioned load behaviors is approximated to be of `Constant Power` type for the time being. - The source bus in power-grid-model is mapped with a source impedance. `Sk"nom`, `R/X` and `Z0/Z1` are the attributes used in modeling source impedance. In Vision, these attributes are used only for short circuit calculations - The load rate for transformer is calculated in Vision by current i.e., `load_rate = max(u1 * I1, u2 * I2) * sqrt(3) / Snom * 100`. Whereas in power-grid-model, loading is calculated by power, i.e., `loading = max(s1,s2)/sn`. (Note: The attribute names are as per relevant notation in Vision and PGM respectively). This gives a slight difference in load rate of transformer. -- A minor difference in results is expected since Vision uses a power mismatch in p.u. as convergence criteria whereas power-grid-model uses voltage mismatch. \ No newline at end of file +- A minor difference in results is expected since Vision uses a power mismatch in p.u. as convergence criteria whereas power-grid-model uses voltage mismatch. + +## Accomdation of UUID-based id system since Vision 9.7 +Vision introduced UUID based identifier system since version 9.7. It is implemented following the Microsoft naming scheme by replacing all the original identifier fields (i.e., '*Number', '*Subnumber', '*Nummber' and '*Subnummer') to GUID. This change brings the many benefits of UUIDs in general while on the other hand adds certain work on the conversion side. Since computations in PGM alone do not benefit from a string based identifier, we hence made the descision to perform UUID to integer conversion while maintaining the 'GUID' information in the `extra_info` field. Note that existing mapping files can still be used without significant changes, apart from adding `GUID` to the `extra` fields of interest. + +An examplery usage can be found in the example notebook as well as in the test cases. diff --git a/docs/examples/vision_example.ipynb b/docs/examples/vision_example.ipynb index e7cc717d..5e952f35 100644 --- a/docs/examples/vision_example.ipynb +++ b/docs/examples/vision_example.ipynb @@ -415,6 +415,60 @@ " display(Markdown(f\"
{json_file.read()}
\"))" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### GUID (Vision 9.7 and later)\n", + "To accomdate GUIDs since version 9.7, we have internal conversion mechanism that converts GUID's to unique integers (persession). This change does not impact anything on the user side. Therefore, loading such new excel files is performed exactly as in previous versions. Here, we only show how one could add corresponding GUID information to the extra_info. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the custom mapping file, add `GUID` to the `extra` field of interest:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# custom yaml mapping file\n", + "...\n", + "grid:\n", + " Nodes:\n", + " node:\n", + " id:\n", + " auto_id:\n", + " key: Number\n", + " u_rated: Unom\n", + " extra:\n", + " - ID\n", + " - Name\n", + " - GUID\n", + " Cables:\n", + " line:\n", + " id:\n", + " auto_id:\n", + " key: Number\n", + " from_node:\n", + " auto_id:\n", + " table: Nodes\n", + " key:\n", + " Number: From.Number\n", + "..." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When run excel conversion with mapping file that contains this change, `GUID` information will be part of the returned `extra_info`." + ] + }, { "cell_type": "markdown", "metadata": {}, diff --git a/src/power_grid_model_io/config/excel/vision_en_9_7.yaml b/src/power_grid_model_io/config/excel/vision_en_9_7.yaml new file mode 100644 index 00000000..62839ee5 --- /dev/null +++ b/src/power_grid_model_io/config/excel/vision_en_9_7.yaml @@ -0,0 +1,863 @@ +# SPDX-FileCopyrightText: Contributors to the Power Grid Model project +# +# SPDX-License-Identifier: MPL-2.0 +--- +id_reference: + nodes_table: Nodes + number: Number + node_number: Node.Number + sub_number: Subnumber +grid: + Nodes: + node: + id: + auto_id: + key: Number + u_rated: Unom + extra: + - ID + - Name + - GUID + Cables: + line: + id: + auto_id: + key: Number + from_node: + auto_id: + table: Nodes + key: + Number: From.Number + from_status: From.Switch state + to_node: + auto_id: + table: Nodes + key: + Number: To.Number + to_status: To.Switch state + r1: R + x1: X + c1: C + tan1: 0 + r0: + power_grid_model_io.functions.both_zeros_to_nan: + value: R0 + other_value: X0 + x0: + power_grid_model_io.functions.both_zeros_to_nan: + value: X0 + other_value: R0 + c0: C0 + tan0: 0 + i_n: Inom' + extra: + - ID + - Name + Lines: + line: + id: + auto_id: + key: Number + from_node: + auto_id: + table: Nodes + key: + Number: From.Number + from_status: From.Switch state + to_node: + auto_id: + table: Nodes + key: + Number: To.Number + to_status: To.Switch state + r1: R + x1: X + c1: C + tan1: 0 + r0: + power_grid_model_io.functions.both_zeros_to_nan: + value: R0 + other_value: X0 + x0: + power_grid_model_io.functions.both_zeros_to_nan: + value: X0 + other_value: R0 + c0: C0 + tan0: 0 + i_n: Inom' + extra: + - ID + - Name + Links: + link: + id: + auto_id: + key: Number + from_node: + auto_id: + table: Nodes + key: + Number: From.Number + from_status: From.Switch state + to_node: + auto_id: + table: Nodes + key: + Number: To.Number + to_status: To.Switch state + extra: + - ID + - Name + Reactance coils: + line: + id: + auto_id: + key: Number + from_node: + auto_id: + table: Nodes + key: + Number: From.Number + from_status: From.Switch state + to_node: + auto_id: + table: Nodes + key: + Number: To.Number + to_status: To.Switch state + r1: R + x1: X + c1: 0 + tan1: 0 + r0: + power_grid_model_io.functions.both_zeros_to_nan: + value: R0 + other_value: X0 + x0: + power_grid_model_io.functions.both_zeros_to_nan: + value: X0 + other_value: R0 + c0: 0 + tan0: 0 + i_n: Inom + extra: + - ID + - Name + Transformers: + transformer: + id: + auto_id: + key: Number + from_node: + auto_id: + table: Nodes + key: + Number: From.Number + from_status: From.Switch state + to_node: + auto_id: + table: Nodes + key: + Number: To.Number + to_status: To.Switch state + u1: Unom1 + u2: Unom2 + sn: Snom + uk: + max: + - divide: + - Pk + - Snom + - power_grid_model_io.functions.value_or_default: + value: uk + default: 0.001 + - 0.001 + pk: Pk + i0: + power_grid_model_io.functions.phase_to_phase.relative_no_load_current: + i_0: Io + p_0: Po + s_nom: Snom + u_nom: Unom2 + p0: Po + winding_from: + power_grid_model_io.functions.phase_to_phase.get_winding_from: + conn_str: Connection + neutral_grounding: N1 + winding_to: + power_grid_model_io.functions.phase_to_phase.get_winding_to: + conn_str: Connection + neutral_grounding: N2 + clock: + power_grid_model_io.functions.phase_to_phase.get_clock: + conn_str: Connection + tap_side: Tap side + tap_pos: Tap + tap_min: Tap min + tap_max: Tap max + tap_nom: Tap nom + tap_size: Tap size + uk_min: + max: + - divide: + - Pk + - Snom + - power_grid_model_io.functions.value_or_default: + value: uk + default: 0.001 + - 0.001 + uk_max: + max: + - divide: + - Pk + - Snom + - power_grid_model_io.functions.value_or_default: + value: uk + default: 0.001 + - 0.001 + pk_min: Pk + pk_max: Pk + r_grounding_from: Re1 + x_grounding_from: Xe1 + r_grounding_to: Re2 + x_grounding_to: Xe2 + extra: + - ID + - Name + Special transformers: + transformer: + id: + auto_id: + key: Number + from_node: + auto_id: + table: Nodes + key: + Number: From.Number + from_status: From.Switch state + to_node: + auto_id: + table: Nodes + key: + Number: To.Number + to_status: To.Switch state + u1: Unom1 + u2: Unom2 + sn: Snom + uk: + max: + - divide: + - Pknom + - Snom + - power_grid_model_io.functions.value_or_default: + value: uknom + default: 0.001 + - 0.001 + pk: Pknom + i0: + power_grid_model_io.functions.phase_to_phase.relative_no_load_current: + i_0: Io + p_0: Po + s_nom: Snom + u_nom: Unom2 + p0: Po + winding_from: 0 + winding_to: 0 + clock: 0 + tap_side: Tap side + tap_pos: Tap + tap_min: Tap min + tap_max: Tap max + tap_nom: Tap nom + tap_size: Tap size + uk_min: + max: + - divide: + - Pkmin + - Snom + - power_grid_model_io.functions.value_or_default: + value: ukmin + default: 0.001 + - 0.001 + uk_max: + max: + - divide: + - Pkmax + - Snom + - power_grid_model_io.functions.value_or_default: + value: ukmax + default: 0.001 + - 0.001 + pk_min: Pkmin + pk_max: Pkmax + r_grounding_from: 0 + x_grounding_from: 0 + r_grounding_to: 0 + x_grounding_to: 0 + extra: + - ID + - Name + Transformer loads: + transformer: + id: + auto_id: + name: transformer + key: + - Node.Number + - Subnumber + from_node: + auto_id: + table: Nodes + key: + Number: Node.Number + to_node: + auto_id: + name: internal_node + key: + - Node.Number + - Subnumber + from_status: Switch state + to_status: 1 + u1: Unom1 + u2: Unom2 + sn: Snom + uk: + max: + - divide: + - Pk + - Snom + - power_grid_model_io.functions.value_or_default: + value: uk + default: 0.001 + - 0.001 + pk: Pk + p0: Po + i0: + power_grid_model_io.functions.phase_to_phase.relative_no_load_current: + i_0: 0 + p_0: Po + s_nom: Snom + u_nom: Unom2 + winding_from: + power_grid_model_io.functions.phase_to_phase.get_winding_from: + conn_str: Connection + winding_to: + power_grid_model_io.functions.phase_to_phase.get_winding_to: + conn_str: Connection + clock: + power_grid_model_io.functions.phase_to_phase.get_clock: + conn_str: Connection + tap_side: Tap side + tap_pos: Tap + tap_min: Tap min + tap_max: Tap max + tap_nom: Tap nom + tap_size: Tap size + uk_min: + max: + - divide: + - Pk + - Snom + - power_grid_model_io.functions.value_or_default: + value: uk + default: 0.001 + - 0.001 + uk_max: + max: + - divide: + - Pk + - Snom + - power_grid_model_io.functions.value_or_default: + value: uk + default: 0.001 + - 0.001 + pk_min: Pk + pk_max: Pk + r_grounding_from: 0 + x_grounding_from: 0 + r_grounding_to: 0 + x_grounding_to: 0 + extra: + - ID + - Name + node: + id: + auto_id: + name: internal_node + key: + - Node.Number + - Subnumber + u_rated: Unom2 + extra: + - ID + - Name + sym_load: + id: + auto_id: + name: load + key: + - Node.Number + - Subnumber + node: + auto_id: + name: internal_node + key: + - Node.Number + - Subnumber + status: Switch state + type: Behaviour + p_specified: + multiply: + - Load.P + - reference: + query_column: Node.Number + other_table: Nodes + key_column: Number + value_column: Simultaneity + q_specified: + multiply: + - Load.Q + - reference: + query_column: Node.Number + other_table: Nodes + key_column: Number + value_column: Simultaneity + extra: + - ID + - Name + sym_gen: + - id: + auto_id: + name: generation + key: + - Node.Number + - Subnumber + node: + auto_id: + name: internal_node + key: + - Node.Number + - Subnumber + status: Switch state + type: 0 + p_specified: Generation.P + q_specified: Generation.Q + extra: + - ID + - Name + - id: + auto_id: + name: pv_generation + key: + - Node.Number + - Subnumber + node: + auto_id: + name: internal_node + key: + - Node.Number + - Subnumber + status: Switch state + type: 0 + p_specified: PV.Pnom + q_specified: + power_grid_model_io.functions.phase_to_phase.reactive_power: + p: PV.Pnom + cos_phi: 1 + extra: + - ID + - Name + Sources: + source: + id: + auto_id: + key: + - Node.Number + - Subnumber + node: + auto_id: + table: Nodes + key: + Number: Node.Number + status: Switch state + u_ref: Uref + sk: Sk"nom + rx_ratio: R/X + z01_ratio: Z0/Z1 + extra: + - ID + - Name + Synchronous generators: + sym_gen: + id: + auto_id: + key: + - Node.Number + - Subnumber + node: + auto_id: + table: Nodes + key: + Number: Node.Number + status: Switch state + type: 0 + p_specified: Pref + q_specified: + multiply: + - power_grid_model_io.functions.phase_to_phase.reactive_power: + p: Pref + cos_phi: cos phi + - Q + extra: + - ID + - Name + Wind turbines: + sym_gen: + id: + auto_id: + key: + - Node.Number + - Subnumber + node: + auto_id: + table: Nodes + key: + Number: Node.Number + status: Switch state + type: 0 + p_specified: + power_grid_model_io.functions.value_or_default: + value: Pref + default: + power_grid_model_io.functions.phase_to_phase.power_wind_speed: + p_nom: Pnom + wind_speed: Wind speed + q_specified: + power_grid_model_io.functions.phase_to_phase.reactive_power: + p: + power_grid_model_io.functions.value_or_default: + value: Pref + default: + power_grid_model_io.functions.phase_to_phase.power_wind_speed: + p_nom: Pnom + wind_speed: Wind speed + cos_phi: cos phi + extra: + - ID + - Name + Loads: + sym_load: + id: + auto_id: + key: + - Node.Number + - Subnumber + node: + auto_id: + table: Nodes + key: + Number: Node.Number + status: Switch state + type: Behaviour + p_specified: + multiply: + - P + - reference: + query_column: Node.Number + other_table: Nodes + key_column: Number + value_column: Simultaneity + q_specified: + multiply: + - Q + - reference: + query_column: Node.Number + other_table: Nodes + key_column: Number + value_column: Simultaneity + extra: + - ID + - Name + Zigzag transformers: + shunt: + id: + auto_id: + key: + - Node.Number + - Subnumber + node: + auto_id: + table: Nodes + key: + Number: Node.Number + status: Switch state + g1: 0 + b1: 0 + g0: + power_grid_model_io.functions.complex_inverse_real_part: + real: R0 + imag: X0 + b0: + power_grid_model_io.functions.complex_inverse_imaginary_part: + real: R0 + imag: X0 + extra: + - ID + - Name + Capacitors: + shunt: + id: + auto_id: + key: + - Node.Number + - Subnumber + node: + auto_id: + table: Nodes + key: + Number: Node.Number + status: Switch state + g1: 0 + b1: + power_grid_model_io.functions.phase_to_phase.reactive_power_to_susceptance: + q: Q + u_nom: Unom + g0: 0 + b0: 0 + extra: + - ID + - Name + Reactors: + shunt: + id: + auto_id: + key: + - Node.Number + - Subnumber + node: + auto_id: + table: Nodes + key: + Number: Node.Number + status: Switch state + g1: 0 + b1: + multiply: + - power_grid_model_io.functions.phase_to_phase.reactive_power_to_susceptance: + q: Q + u_nom: Unom + - -1 + g0: 0 + b0: 0 + extra: + - ID + - Name + Pvs: + sym_gen: + id: + auto_id: + key: + - Node.Number + - Subnumber + node: + auto_id: + table: Nodes + key: + Number: Node.Number + status: Switch state + type: 0 + p_specified: + power_grid_model_io.functions.phase_to_phase.pvs_power_adjustment: + p: + multiply: + - min: + - Pnom + - Inverter.Pnom | Inverter.Snom + - Scaling + efficiency_type: Inverter.efficiency type + q_specified: + multiply: + - power_grid_model_io.functions.phase_to_phase.reactive_power: + p: + min: + - Pnom + - Inverter.Pnom | Inverter.Snom + cos_phi: Inverter.cos phi + - Scaling + extra: + - ID + - Name + Three winding transformers: + three_winding_transformer: + id: + auto_id: + key: Number + node_1: + auto_id: + table: Nodes + key: + Number: Node1.Number + status_1: Switch state 1 + node_2: + auto_id: + table: Nodes + key: + Number: Node2.Number + status_2: Switch state 2 + node_3: + auto_id: + table: Nodes + key: + Number: Node3.Number + status_3: Switch state 3 + u1: Unom1 + u2: Unom2 + u3: Unom3 + sn_1: Snom1 + sn_2: Snom2 + sn_3: Snom3 + uk_12: + max: + - divide: + - Pk12 + - min: + - Snom1 + - Snom2 + - power_grid_model_io.functions.value_or_default: + value: uk12 + default: 0.001 + - 0.001 + uk_13: + max: + - divide: + - Pk13 + - min: + - Snom1 + - Snom3 + - power_grid_model_io.functions.value_or_default: + value: uk13 + default: 0.001 + - 0.001 + uk_23: + max: + - divide: + - Pk23 + - min: + - Snom2 + - Snom3 + - power_grid_model_io.functions.value_or_default: + value: uk23 + default: 0.001 + - 0.001 + pk_12: Pk12 + pk_13: Pk13 + pk_23: Pk23 + i0: + power_grid_model_io.functions.phase_to_phase.relative_no_load_current: + i_0: Io + p_0: Po + s_nom: Snom3 + u_nom: Unom3 + p0: Po + winding_1: + power_grid_model_io.functions.phase_to_phase.get_winding_1: + conn_str: Connection + neutral_grounding: N1 + winding_2: + power_grid_model_io.functions.phase_to_phase.get_winding_2: + conn_str: Connection + neutral_grounding: N2 + winding_3: + power_grid_model_io.functions.phase_to_phase.get_winding_2: + conn_str: Connection + neutral_grounding: N3 + clock_12: + power_grid_model_io.functions.phase_to_phase.get_clock_12: + conn_str: Connection + clock_13: + power_grid_model_io.functions.phase_to_phase.get_clock_13: + conn_str: Connection + tap_side: Tap side a + tap_pos: Tap a + tap_min: Tap min a + tap_max: Tap max a + tap_nom: Tap nom a + tap_size: Tap size a + r_grounding_1: Re1 + x_grounding_1: Xe1 + r_grounding_2: Re2 + x_grounding_2: Xe2 + r_grounding_3: Re3 + x_grounding_3: Xe3 + extra: + - ID + - Name + +units: + A: null + F: + µF: 0.000_001 + V: + kV: 1_000.0 + VA: + kVA: 1_000.0 + MVA: 1_000_000.0 + VAR: + kvar: 1_000.0 + Mvar: 1_000_000.0 + W: + kW: 1_000.0 + MW: 1_000_000.0 + Wp: + kWp: 1_000.0 + MWp: 1_000_000.0 + m/s: null + ohm: + Ohm: 1.0 + ohm/m: + ohm/km: 0.001 + one: + pu: 1.0 + "%": 0.01 + "‰": 0.001 + +substitutions: + ".*Switch state": + "off": 0 + "in": 1 + "on": 1 + "Switch state .*": + "off": 0 + "in": 1 + "on": 1 + N1: + 0: false + 1: true + none: false + own: true + N2: + 0: false + 1: true + none: false + own: true + N3: + 0: false + 1: true + none: false + own: true + Behaviour: + Constant admittance: 1 + Constant impedance: 1 + ~Constant current: 2 + Constant power: 0 + Default: 0 + Industry: 0 + Business: 0 + Residential: 0 + Living: 0 + Tap side: + 1: 0 + 2: 1 + Synchronous generators.Q: + absorb: -1 + supply: 1 diff --git a/tests/validation/converters/test_vision_excel_converter.py b/tests/validation/converters/test_vision_excel_converter.py index 854af5ba..1c438cef 100644 --- a/tests/validation/converters/test_vision_excel_converter.py +++ b/tests/validation/converters/test_vision_excel_converter.py @@ -28,6 +28,9 @@ LANGUAGES_97 = ["en"] VALIDATION_EN = Path(str(VALIDATION_FILE).format(language="en")) CUSTOM_MAPPING_FILE = DATA_PATH / "vision_9_5_{language:s}.yaml" +VISION_97_MAPPING_FILE = ( + Path(__file__).parent.parent.parent.parent / "src/power_grid_model_io/config/excel" / "vision_en_9_7.yaml" +) terms_changed = {"Grounding1": "N1", "Grounding2": "N2", "Grounding3": "N3", "Load.Behaviour": "Behaviour"} @@ -304,12 +307,36 @@ def test_log_levels(capsys): assert "debug" not in outerr.out -def test_uuid_excel_input(): +def prep_vision_97(language: str) -> VisionExcelConverter: source_file = Path(str(SOURCE_FILE_97).format(language=LANGUAGE_EN)) + return VisionExcelConverter( + source_file, language="en", mapping_file=VISION_97_MAPPING_FILE, terms_changed=terms_changed + ) + + +def test_uuid_excel_input(): ref_file_97 = convert_guid_vision_excel( - excel_file=source_file, number=VISION_EXCEL_LAN_DICT[LANGUAGE_EN][DICT_KEY_NUMBER], terms_changed=terms_changed + excel_file=Path(str(SOURCE_FILE_97).format(language=LANGUAGE_EN)), + number=VISION_EXCEL_LAN_DICT[LANGUAGE_EN][DICT_KEY_NUMBER], + terms_changed=terms_changed, ) - data_native, _ = VisionExcelConverter(source_file, language="en", terms_changed=terms_changed).load_input_data() data_convtd, _ = VisionExcelConverter(source_file=ref_file_97).load_input_data() + vision_cvtr = prep_vision_97(language=LANGUAGE_EN) + data_native, _ = vision_cvtr.load_input_data() assert len(data_native) == len(data_convtd) + + +def test_guid_extra_info(): + print("extra_info") + vision_cvtr = prep_vision_97(language=LANGUAGE_EN) + _, extra_info = vision_cvtr.load_input_data() + + assert extra_info[0]["GUID"] == "{8110C18E-7FDF-4A43-B868-E05E1DD7909F}" + assert extra_info[1]["GUID"] == "{D507B771-AC81-41F5-A292-91CF6119BE70}" + assert extra_info[2]["GUID"] == "{C92556BA-57E6-4780-A69E-DDDCA759BFBE}" + assert extra_info[3]["GUID"] == "{23C17CB9-7C1F-4AD0-B3C9-9B4AB99DB400}" + assert extra_info[4]["GUID"] == "{915748FF-2EE0-4C5E-806A-DBBFC433BEA4}" + + for i in range(5, len(extra_info)): + assert "GUID" not in extra_info[i]