Dependencies

Project Dependencies

To successfully run the code included in this repository requires the preparation of the following items:

Resource Information
Ontology Data
Edge Data
Construction Approach
Mapping and Filtering Data
Relations Data
Metadata

Programmatic Assistance
Users who would like assistance with assembling the required input documents should run the generates_dependency_documents.py script from the command line:

python3 generates_dependency_documents.py

Resource Information

The figure below provides an overview of how the resources/resource_info.txt, resources/ontology_source_list.txt, and resources/edge_source_list.txt data sources are connected as well as how they work together.

GitHub Repository Location: resources/resource_info.txt

Purpose: This file is used as the master organizer for all project resources.

File Format: The program expects the information stored as a "|" delimited file:

Edge Type: an edge label (node1-node2) matching Data Sources in the ontology and non-class edge input files
Source labels: Three ;-separated items (e.g.:;GO_;GO_). The first item (i.e. ':') is a character that is used to split existing labels (e.g. : in GO:1283834), such that all characters that occur before that label are replaced by the content provided after the first and second ';' (i.e. 'GO_' and 'GO_'). If the existing label is correct, type ;;.
Data Type: a label of class or instance provided for each node and separated by - (e.g. class-class)
Edge Relation: a Relation Ontology identifier to be used as an edge between the nodes (e.g. RO_0000056)
Subject URI: a Universal Resource Identifier that will be connected to the subject node in the Edge Type (e.g. http://purl.obolibrary.org/obo/ or http://purl.uniprot.org/geneid/)
Object URI: a Universal Resource Identifier that will be connected to the object node in the Edge Type (e.g. http://purl.obolibrary.org/obo/ or http://purl.uniprot.org/geneid/)
Delimiter: a character used to split input text rows into columns (e.g. t for tab-delimited data or , for comma-delimited data)
Column Indices: two column indices separated by ; (e.g. 0;4 for the first and third columns in the input data source)
Identifier Maps: a string which is used to indicate the index of the edge node column in the input data source that needs to be mapped to a different identifier and a file containing a column that contains the data to map to (e.g. 2:mapping_file_1.txt;4:mapping_file_2.txt - means mapping data from the first node in the edge to the 0th column in mapping_file_1.txt and data from the second node in the edge to the 4th column in mapping_file_2.txt)
Evidence Criteria: sets of 3 ::-separated items, where each set is composed of three pieces of ;-separated information (e.g. 4;!=;IEA::8;<;0.0001):
1. The index of the column to apply the evidence criteria to
  - 4 and 8 from the example above
2. The operator (i.e. ==, !=, <, >, <=, >=, in, .startswith(), .endswith()) to use when filtering
  - != and < from the example above
3. The value to filter on (i.e. int, float, str, [])
  - IEA and 0.0001 from the example above
Filter Criteria: Sets of 3 ::-separated items, where each set is composed of three pieces of ;-separated information (e.g. 5;==;P::7;==;9606):
1. The index of the column to apply the filter criteria to
  - 5 and 7 from the example above
2. The operator to use when filtering (i.e. ==, !=, <, >, <=, >=, in, .startswith(), .endswith()) to use when filtering
- == and == from the example above
1. The value to filter on (i.e. int, float, str, [])
  - P and 9606 from the example above.

NOTE. You can also pass dedup as a Filtering Criteria (e.g. 2-0;dedup;desc):

The column index should be col1-col2:
- col1 is the column you want to filter on
- col2 is the primary identifier to deduplicate
The value should be asc or desc to indicate the direction to sort the pandas.DataFrame prior to deduplicating

TABLE: An example resource_info.txt file is provided in the table below.

Edge Type	Source labels	Data Type	Edge Relation	Subject URI	Object URI	Delimiter	Column Indices	Identifier Maps	Evidence Criteria	Filter Criteria
chemical-gene	;MESH_;	class-entity	RO_0002434	http://purl.obolibrary.org/obo/	http://purl.uniprot.org/geneid/	t	1;4	0:./resources/data_maps/MESH_CHEBI_MAP.txt	None	7;==;9606
gene-gene	.;;	entity-entity	RO_0002434	http://purl.uniprot.org/geneid/	http://purl.uniprot.org/geneid/	' '	0;1	0:./resources/data_maps/STRING_ENTREZ_MAP.txt;1:./resources/data_maps/STRING_ENTREZ_MAP.txt	2;>=;700	None
gene-gobp	;;	entity-class	BFO_0000056	http://purl.uniprot.org/geneid/	http://purl.obolibrary.org/obo/	t	1;4	0:./resources/edge_data/gene-go_goa_class_data.txt	8;==;P	12;==;taxon:9606
pathway-disease	;;	entity-class	RO_0003302	https://reactome.org/content/detail/	http://purl.obolibrary.org/obo/	t	1;0	1:disease-dbxref-map	None	1;.startswith('R-HSA-');

Ontology Data

GitHub Repository Location: resources/ontology_source_list.txt

Purpose: This script is used to identify and download specific ontologies.

File Format: The program expects this information to be stored as a "," delimited file.

TABLE: An example ontology_source_list.txt file is provided in the table below.

Ontology	URL
disease	http://purl.obolibrary.org/obo/doid.owl
go	http://purl.obolibrary.org/obo/go.owl
chemical	ftp://ftp.ebi.ac.uk/pub/databases/chebi/ontology/chebi_lite.owl

Edge Data

GitHub Repository Location: resources/edge_source_list.txt

Purpose: This script is used to identify and download specific publicly available data sources that will be used to derive edges between ontology classes and instances of ontology classes.

File Format: The program expects this information to be stored as a "," delimited file.

TABLE: An example edge_source_list.txt file is provided in the table below.

Data Source	URL
chemical-gene	http://ctdbase.org/reports/CTD_chem_gene_ixns.tsv.gz
gene-gobp	http://geneontology.org/gene-associations/goa_human.gaf.gz
gene-disease	https://www.disgenet.org/static/disgenet_ap1/files/downloads/curated_gene_disease_associations.tsv.gz
gene-gene	https://stringdb-static.org/download/protein.links.v11.0/9606.protein.links.v11.0.txt.gz

Construction Approach

Wiki: KG-Construction
GitHub Repository Location: resources/construction_approach

Purpose: New data can be added to the knowledge graph using 2 different construction approaches: (1) instance-based or (2) subclass-based. Each of these approaches is described further below. For more details, please see the resources/construction_approach/README.md Jupyter Notebook for additional information.

🛑 CONSTRAINTS 🛑
The algorithm makes the following assumptions:

Make sure that you have created the non-ontology node data to ontology class mapping dictionary and added it to the ./resources/construction_approach/ directory.

Construction Approach: Instance-Based

In this approach, each new edge is added as an instance of an existing class (via rdf:Type) in the knowledge graph.

EXAMPLE: Adding the edge: Morphine ➞ isSubstanceThatTreats ➞ Migraine

Would require adding:

isSubstanceThatTreats(Morphine, x1)
Type(x1, Migraine)

In this example, Morphine is a non-ontology data node and Migraine is an HPO ontology term.

Outputs: As mentioned above, a universally unique identifier (UUID) is created for each anonymous node representing an instance of a class. In order to fully utilize the knowledge graph, a .json file containing the mapping from each UUID instance to it's ontology class is output to the ./resources/construction_approach/instance directory. For example,

{
"http://purl.obolibrary.org/obo/CHEBI_24505": "https://github.com/callahantiff/PheKnowLator/obo/ext/c2591241-8952-44ea-a313-e4b3c5fb6d35",
"http://purl.obolibrary.org/obo/PR_000013648": "https://github.com/callahantiff/PheKnowLator/obo/ext/0ea74deb-0002-4f48-b7e4-81a8fd947312",
"http://purl.obolibrary.org/obo/GO_0050031": "https://github.com/callahantiff/PheKnowLator/obo/ext/8f5c81d4-92dd-426e-a2d9-2be87edb1520",
}

Construction Approach: Subclass-Based

In this approach, each new edge is added as a subclass of an existing ontology class (via rdfs:subClassOf) in the knowledge graph.

EXAMPLE: Adding the edge: TGFB1 ➞ participatesIn ➞ Influenza Virus Induced Apoptosis

Would require adding:

participatesIn(TGFB1, Influenza Virus Induced Apoptosis)
subClassOf(Influenza Virus Induced Apoptosis, Influenza A pathway)
Type(Influenza Virus Induced Apoptosis, owl:Class)

Where TGFB1 is an PR ontology term and Influenza Virus Induced Apoptosis is a non-ontology data node. In this example, Influenza A pathway is an existing ontology class.

Outputs: There are no approach-specific output files generated.

Input Requirements for both Approaches: A pickled dictionary where the keys are node identifiers (non-ontology node data) and the values are lists of ontology class identifiers to subclass has been added to the ./resources/construction_approach/ directory. An example of this dictionary is shown below:

{
  'R-HSA-168277'  : ['http://purl.obolibrary.org/obo/PW_0001054',
                     'http://purl.obolibrary.org/obo/GO_0046730'],
  'R-HSA-9026286' : ['http://purl.obolibrary.org/obo/PW_000000001',
                     'http://purl.obolibrary.org/obo/GO_0019372'],
  '100129357'     : ['SO_0000043'],
  '100129358'     : ['SO_0000336'],
}

Please see the Reactome Pathways - Pathway Ontology and Genomic Identifiers - Sequence Ontology sections of the Data_Preparation.ipynb Jupyter Notebook for examples of how to construct this document.

Mapping and Filtering Data

Wiki: v2-Data-Sources

Purpose: Several other files are needed to create data used for the filtering and mapping during the creation of knowledge graph edges. For more details on what these data sources are and how they are created, please see the Data_Preparation.ipynb Jupyter Notebook.

Relations Data

GitHub Repository Location: resources/relations_data

Purpose: PheKnowLator can be built using a single set of provided relations (i.e. the owl:ObjectProperty or edge which is used to connect the nodes in the graph) with or without the inclusion of each relation's inverse.

🛑 CONSTRAINTS 🛑
If you would like the knowledge graph to include relations and their inverse relations, you must add the following to the ./resources/relations_data repository (an example of what should be included in each of these is included below):

A .txt file of all relations and their labels
A .txt file of the relations and their inverse relations

Filename: INVERSE_RELATIONS.txt

The owl:inverseOf property is used to identify each relation's inverse. To make it easier to look up the inverse relations when building the knowledge graph, each relation/inverse relation pair is listed twice, for example:

location of owl:inverseOf located in
located in owl:inverseOf location of

The data in this file should look like:

  RO_0003000  RO_0003001
  RO_0003001  RO_0003000
  RO_0002233  RO_0002352
  RO_0002352  RO_0002233

Filename: RELATIONS_LABELS.txt

Not all relations have an inverse (e.g. interactions). Even though an inverse relation might not exist, we still want to ensure that all interaction relations are symmetrically represented in the graph. To aid in this process, we need to be able to quickly look-up an edge and determine if it is an interaction. To help make this process more efficient, the algorithm expects a list of all relations and their labels in a .txt file.

The data in this file should look like:

  RO_0002285  developmentally replaces
  RO_0002287  part of developmental precursor of
  RO_0002490  existence overlaps
  RO_0002214  has prototype

Please see the Data_Preparation.ipynb Jupyter Notebook for code on how to create these files.

Metadata

GitHub Repository Location: resources/node_data

Purpose: The knowledge graph can be built with or without the inclusion of node and relation metadata (i.e. labels, descriptions or definitions, and synonyms). If you'd like to create and use node metadata, please run the Data_Preparation.ipynb Jupyter Notebook and run the code chunks listed under the INSTANCE AND/OR SUBCLASS (NON-ONTOLOGY CLASS) METADATA section. These code chunks should be run before the knowledge graph is constructed. For more details on what these data sources are and how they are created, please see the node_data README.md.

Example structure of the metadata dictionary is shown below:

{
    'nodes': {
        'http://www.ncbi.nlm.nih.gov/gene/1': {
            'Label': 'A1BG',
            'Description': "A1BG has locus group protein-coding' and is located on chromosome 19 (19q13.43).",
            'Synonym': 'HYST2477alpha-1B-glycoprotein|HEL-S-163pA|ABG|A1B|GAB'} ... },
    'relations': {
        'http://purl.obolibrary.org/obo/RO_0002533': {
            'Label': 'sequence atomic unit',
            'Description': 'Any individual unit of a collection of like units arranged in a linear order',
            'Synonym': 'None'} ... }
}

🛑 CONSTRAINTS 🛑
The algorithm makes the following assumptions:

If metadata is provided, only those edges with nodes that have metadata will be created; valid edges without metadata will be discarded.
Metadata for all non-ontology nodes and all relations for edges added to the core set of ontologies will be saved as a dictionary in the ./resources/node_data/node_metadata_dict.pkl repository.
For each identifier we try to obtain the following metadata: Label, Description, and Synonym. An example of these data types is shown below for a gene identifier 5620:

Metadata Type	Definition	Metadata
ID	Node identifiers for instance data sources	`5620`
Label	The primary label or name for the node	`LANCL2`
Description	A definition or other useful details about the node	`Lanc Like 2` is a `protein-coding` gene that is located on chromosome `7` (map_location: `7p11.2`)
Synonym	Alternative terms used for a node	`GPR69B`, `TASP`, `lanC-like protein 2`, `G protein-coupled receptor 69B`, `LanC (bacterial lantibiotic synthetase component C)-like 2`, `LanC lantibiotic synthetase component C-like 2`, `testis-specific adriamycin sensitivity protein`

Project Information

Tutorials and Use Cases

Tutorials
- Implementing OWL-NETS
- Processing an RDF Graph
Use Cases
- Drug Safety
  - KG-based Investigation of Drug-Outcome Pairs

Releases

Benchmarks and Builds
- Archived

Human Disease KG Builds

Archived Builds
- v1.0.0
  - September 3, 2019
- v2.0.0
- v2.1.0
- v3.0.2
  - October 18, 2021
  - November 01, 2021

FAQs

How to Get Involved

Provide feedback

Saved searches

Use saved searches to filter your results more quickly