This represents a guide on how to import/export data between TWW and INTERLIS 2 Transfer Format (XTF).
The TWW plugin includes an INTERLIS import/export feature. It is currently capable of importing and exporting the following models:
- SIA405_ABWASSER_2020_1_LV95
- DSS_2020_1_LV95
- VSA_KEK_2020_1_LV95
Note that currently, exports are possible in German only. Translated exports are on the roadmap, let us know if you are interested in this feature.
You need java on your system.
Windows : Java for windows download MacOS : Java for MacOS download Linux : use the package manager
You need sqlalchemy and geoalchemy2.
To install : before launching QGIS, open OSGeo4W Shell
and type:
>>> pip3 install "sqlalchemy~=2.0.38" "geoalchemy2>=0.14.7"
If the command fails, try:
>>> pip3 install --upgrade pip
Note
If pip cannot be found, make sure you’ve installed the python3-pip package using the OSGeo4W network installer (if you’ve installed the standalone version of QGIS, it should be included).
If this command still fails, try:
>>> pip3 install --target=c:\somewhere\other\than\the\default --upgrade pip
where c:\somewhere\other\than\the\default is the path to your current pip package
The same --target flag can be added to install sqlalchemy and geoalchemy2 in another location.
The export only supports up-to-date TWW data model (2020.1 at the time of writing). Ensure your data model is fully updated before trying to import/export.
In the plugin>TWW>settings dialog, under the Developer options tab, ensure Admin mode is enabled. Restart QGIS.
You should now see new import and export buttons in the TWW toolbar.
To export your TWW data, click on the INTERLIS export button.
The following dialog will appear.
At first, select the export model.
If you have an active selection in the nodes and/or reaches layer, you can choose to restrict the export to that selection. This is especially useful in combination with the upstream/downstream selection tools.
The export tools is capable of exporting label positions for different scales. You can choose which scales you are interested in exporting by selected/deselecting them.
Then, confirm the dialog and choose where to save the .xtf file.
Note
Note that windows file pathes with empty strings in the directory path or filename are not supported at the moment.
Note
Note that for large/full exports, the command can take a long time.
Note
Note that QGIS standard behaviour for too long labels is to turn them horizontally (orientation = 0). You can adjust the properties of the vw_tww_reach layer - Symbology - Placement - Label overrun from "no overrun" to e.g. 7 mm (Millimeter)
Note
Note that remark fields are truncated to 80 characters on INTERLIS Export, as the INTERLIS definition is like this. If you have remark fields with more text then consider to move this data to documentation with the classes "file" and "data_media" so it can be exported to INTERLIS completely where you can add any document, photo or video to a class.
The following validations are run as pre-process: * Validate subclasses before export: check_subclass_counts
Note
Currently you will only find an ERROR message in the tww procotol tab if this check fails: 2024-07-26T10:29:03 teksi_wastewater.utils:ERROR: number of subclass elements of structure_part NOT CORRECT in schema tww_od: checksum = 1 (positive number means missing entries, negative means too many subclass entries)
Exports include a validation step at the end using ilivalidator, which will inform you whether the export contains INTERLIS validation errors.
If the export was successful a green success message appears. If the export is not successfull you will get either error messages during the export process or an error warning at the end of the process.
You can check the protocol window and the text logs for further details.
| Heading Export step | Heading Description | Heading Log File |
|---|---|---|
| Export schema | Creation of export schema with ili2db and the selected INTERLIS model file | *.ili2pg-schemaimport.log |
| Data export intermediatary schema | Export of data from tww_od to the new intermediary schema, translation from English to German | *.tww2ili-export.log |
| Data export to xtf | Data export from intermediary schema to INTERLIS Transfer file (xtf) | *.ili2pg-export-MODELNAME.log |
| INTERLIS validation | Validation of the created xtf file(s) with ilivalidator | *.ilivalidator-MODELNAME.log |
To import xtf`files, click on the `INTERLIS import button and navigate to the .xtf file.
Note
Note that windows file pathes with empty strings in the directory path or filename are not supported at the moment.
Organisations before data !
Note
Starting with release 2020 all organisations are in a separated dataset and need to be imported first, else a bunch of errors will be thrown on all references like fk_dataowner, fk_owner, fk_provider, etc. Download the VSA Organisation data set from https://www.vsa.ch/models/organisation/vsa_organisationen_2020_1.xtf and start with importing that data set. This data set is updated regularly - so please come back and check (and maybe re-import) at a later stage to have access to all available organisation.
If you have organisations that are not yet in that data set please inform the VSA and hand in an application to be added.
If you have additional own local organisations that are not (yet) in the VSA organisation data set continue with importing those before you start importing your network data. TV inspection data usually comes last, as it references your network data.
After launching the import process your data set will be validated and imported in a intermediatary schema. Then the following dialog will appear.
The left part of this dialog lists all elements that are going to be imported from the .xtf file, allowing to review what is going to be imported and to deselect elements you may want to skip. It also shows the validation status of each object, showing whether further action is needed (INVALID) or recommended (WARNING) prior to importing.
The right part of this dialog shows a form specific to the type of element selected in the list, allowing to adapt the import.
Note
Currently de-selecting and selecting objects might take a long time depending how many data sets are in that respective class.
Note
Special feature for TV Inspection import: For instance, it allows to attach "examinations" to their pipes.
Once you're happy with the import options, confirm the dialog to persist the changes to your database.
If the import was successful a green success message appears.
For advanced usage, the import/export tool can also be used as a command line tool.
The QGIS python classes must be available in the python interpreter.
- On windows you can use the OSGeo4W environment
- On linux you can extend the PYTHONPATH pointing to you QGIS installation, for example: export PYTHONPATH="$PYTHONPATH:/usr/share/qgis/python"
Import example
$ python3 plugin/tww_cmd.py interlis_import --xtf_file plugin/teksi_wastewater/tests/data/minimal-dataset-SIA405-ABWASSER.xtf --pgservice pg_tww
Full usage
$ python3 plugin/tww_cmd.py interlis_export --help
usage: tww_cmd.py interlis_export [-h] --xtf_file XTF_FILE [--selection SELECTION] [--export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}] [--logs_next_to_file]
[--label_scale_pipeline_registry_1_1000] [--label_scale_network_plan_1_500] [--label_scale_overviewmap_1_10000] [--label_scale_overviewmap_1_5000]
[--label_scale_overviewmap_1_2000] [--selected_ids SELECTED_IDS] [--pgservice PGSERVICE] [--pghost PGHOST] [--pgport PGPORT] [--pgdatabase PGDATABASE]
[--pguser PGUSER] [--pgpass PGPASS]
options:
-h, --help show this help message and exit
--xtf_file XTF_FILE XTF outup file
--selection SELECTION
if provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
--export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}
Model to export (default: DSS_2020_1_LV95)
--logs_next_to_file Put log files next to XTF output file
--label_scale_pipeline_registry_1_1000
Export labels in scale 1:1'000, can be combined with other scales (Leitungskataster/Cadastre des conduites souterraines)
--label_scale_network_plan_1_500
Export labels in scale 1:500, can be combined with other scales (Werkplan/Plan de reseau)
--label_scale_overviewmap_1_10000
Export labels in scale 1:10'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
--label_scale_overviewmap_1_5000
Export labels in scale 1:5'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
--label_scale_overviewmap_1_2000
Export labels in scale 1:2'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
--selected_ids SELECTED_IDS
If provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
--pgservice PGSERVICE
Postgres service name
--pghost PGHOST Postgres host
--pgport PGPORT Postgres port
--pgdatabase PGDATABASE
Postgres database
--pguser PGUSER Postgres user
--pgpass PGPASS Postgres password
Export example
python3 plugin/tww_cmd.py interlis_export --xtf_file "output.xtf" --pgservice pg_tww
Full usage
$ python3 plugin/tww_cmd.py interlis_export --help
usage: tww_cmd.py interlis_export [-h] --xtf_file XTF_FILE [--selection SELECTION] [--export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}] [--logs_next_to_file]
[--label_scale_pipeline_registry_1_1000] [--label_scale_network_plan_1_500] [--label_scale_overviewmap_1_10000] [--label_scale_overviewmap_1_5000]
[--label_scale_overviewmap_1_2000] [--selected_ids SELECTED_IDS] [--pgservice PGSERVICE] [--pghost PGHOST] [--pgport PGPORT] [--pgdatabase PGDATABASE]
[--pguser PGUSER] [--pgpass PGPASS]
options:
-h, --help show this help message and exit
--xtf_file XTF_FILE XTF outup file
--selection SELECTION
if provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
--export_model {SIA405_ABWASSER_2020_1_LV95,DSS_2020_1_LV95,VSA_KEK_2020_1_LV95}
Model to export (default: DSS_2020_1_LV95)
--logs_next_to_file Put log files next to XTF output file
--label_scale_pipeline_registry_1_1000
Export labels in scale 1:1'000, can be combined with other scales (Leitungskataster/Cadastre des conduites souterraines)
--label_scale_network_plan_1_500
Export labels in scale 1:500, can be combined with other scales (Werkplan/Plan de reseau)
--label_scale_overviewmap_1_10000
Export labels in scale 1:10'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
--label_scale_overviewmap_1_5000
Export labels in scale 1:5'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
--label_scale_overviewmap_1_2000
Export labels in scale 1:2'000, can be combined with other scales (Uebersichtsplan/Plan d'ensemble)
--selected_ids SELECTED_IDS
If provided, limits the export to networkelements that are provided in the selection (comma separated list of ids)
--pgservice PGSERVICE
Postgres service name
--pghost PGHOST Postgres host
--pgport PGPORT Postgres port
--pgdatabase PGDATABASE
Postgres database
--pguser PGUSER Postgres user
--pgpass PGPASS Postgres password
