Merge from Dev branch (#29)

* update cache builder (#24)

* Note/dev (#25)


* create build_cache.sh

* Update CD.yml (#26)

* Ferdous dev (#28)

* Update help.py

* Update about.py with published paper link

* Update util_help_tables.py with apple tests

* Update util_footer_table.py with better bmws logo

* Add files via upload

* Delete Bundesministerium_für_Wirtschaft_und_Energie_Logo.svg.png

* Update about.py

* Update README.md

* update page title

* fix format and error

* fix format == to is

* Squashed 'pages/libs/mpxsonar/' changes from 6c083cd..91422cf

91422cf fix merge
673fcc9 change == to is
59f49b0 Patch mpoxsonar (#2)
b403feb minor change in dbm
8892a93 Update MpoxSonar 0.5.3 Merge commit 'd20960430509ef4fc45455fdb206e7ef5c8030f5' into note/dev
58c001e fix matching function
668a8a9 qMerge commit '55314730f7bcd1e927c95bb9befc780fdb8ee3c6' into note/dev
ef06eca update NCBI download
09b494e Merge commit '70b50bcac89349125d5f077daa004f97079ac953' into note/dev
c5854d7 fix count issue
bf0fd53 Patch V.0.3.5 (#7)
a0572a0 fix delete function
7618df4 Update Dev branch (#5)
63ee752 Update patch 0.5.1 (#1)
541d76a update 0.5.1, fix bugs
4d9fd3b update 0.5.1
9f71aaf Merge branch 'note/dev' of https://github.com/rki-mf1/MpoxRadar into note/dev
7e9afad Merge commit 'f4e9902d0b5fb8806264882d42b61db87589d3de' as 'pages/libs/mpxsonar'
5568533 remove mpxsonar
245d27d update the site
dbdb006 update mpoxsonar repo.
9c147cb update NT display
570b4ff fix bug
55cc0b4 Update README.md
72e976f Merge pull request #12 from ferbsx/add-license-1
8b5a311 Create LICENSE.md
6c98df9 The first version to the mpoxradar.net (#11)
acb06dd The first version to the mpoxradar.net (#11)
b99fb44 terminology correction
9f4c00e replaced file for the correct name
f427576 correcting name
c14d9aa Update about.py
18333e7 FAQ edit according to alice comment
751c5f7 edit terminology: mpox, MPoxRadar
d015854 edit wrong links with anchor tag
bc7a09c Merge pull request #10 from ferbsx/injun/dev
268e798 Bugremovung
adbb052 Merge pull request #9 from ferbsx/injun/dev
e1cab9c Merge remote-tracking branch 'origin/dev' into injun/dev
5c989dc Merge remote-tracking branch 'origin/dev' into injun/dev
3cbcf5b Merge pull request #8 from ferbsx/ivan/dev
473f845 Merge branch 'injun/dev' into ivan/dev
a8b966f Merge branch 'injun/dev' into ivan/dev
3b32162 garbage cleaning
4cd7dd5 Update help.py
40d86d8 Update about.py
abec293 MPXdatabase dump added
892b0d2 Update util_help_tables.py
2e2d17d Update util_tool_checklists.py
f9159ca Update util_footer_table.py
08cbcd2 Update contact.py
518d27c 1. changed structure 2. !!!callbacks doent trigger
ee77cd4 csv-file in case of no working database - for developemnt only
3594609 data.py connects to a mpox-database and gets a dictionary of dataframes
94225ad stylized names, texts and titles
076fc28 this is Ivan's tool, which is ahead Injun's (--> trying to make one branck/fork from these two, to then put in dev)
7a177bd merge change
829b200 merge change
80833ec map.py is now unncessary
7efebb8 jorge_tool.py is now unnecessary
59e213c pointing at where to replace the map-figures
ca7d045 adding an example to run app.py with different env-paramenters - usefull when developing with accesss to local databases
c43399a fixed zoom in non-jorge's map
5b9ab7b added jorge's code
36d5ff6 updated personal notesvim
599909c updated personal notes
3de147b commented out the whole file to not interfere with pages/tool.py and avoid Werkzeug-Exception
98d2171 location_coordinates.csv file from Ivan
99980ab Data.csv file from Ivan
b36051c added import os in order to make pages/*.csv relative paths, note absolute paths
8fb0984 jorge's personal notes
b9e4854 copy of jorge's tool.py to be merged with dev's tool.py
024927a copy of jorge's data.py for dev-branch
e4bd4a0 rephrasing text. from jorge's branch to dev
364ecb3 general .env.template from ivan's to injun's
47d9421 added one of the two data-files from Ivan
80dd223 resize logo
e6e9b02 Note/dev (#6)
9e2ec74 Note/dev (#6)
4757c44 new design (from Injun)
12c8197 Merge branch 'dev' of https://github.com/ferbsx/MPXRadar-frontend into dev
aac9abb help page added
694caa8 edit Datasource Link
cb54534 add page for table, edit for callback
1f79d23 data for geo
71e1e94 make footer&tool.checkout, edit some pages
ee6e064 callbacks to the separate file
dfed955 added map animation
9026b9f data
f0a8727 Q&A on help.py, add photo, edit footer
9c4c7fa web page ceate and edit
0f50ada web page ceate and edit
dd44c8b edit from evan/dev
dbe41b2 edit from evan/dev
82a9053 map working instruments
96345f4 add style and map.py file
108070d playing with map
02da350 adding dbc components
3f5ef9d update mpxsonar 0.4.3, merge
ec3196b update mpxsonar 0.4.3, merge
5de8641 update mpxsonar 0.4.3, merge
5bace10 update mpxsonar 0.4.3, merge
2744ff3 Squashed 'pages/libs/mpxsonar/' changes from 36581b5..0ffa7eb
8d95d44 note/dev (#5)
159896e Note/dev -Update MPXsonar 0.4.2 (#4)
fd76d1d Note/dev -Update MPXsonar 0.4.2 (#4)
a9c407b map manipulatios
85542e7 pages/home.py
6bea1dd cosmeic changes
3871a65 cosmetic changes
2c3c307 fixed internal address problems
41e3bd3 added an example map
922bcf6 some details
fd9050e some details
e210173 some details
08761c1 new design details
3d00552 Delete .gitkeep
95ba64a Add files via upload
9c303b6 Create .gitkeep
13801c7 Add files via upload
b6fcb62 Merge commit 'f4be82d7efccd85009e4f538b5d52978ba74d4d2' as 'pages/libs/mpxsonar'
f4be82d Squashed 'pages/libs/mpxsonar/' content from commit 36581b5
4ef7779 Merge commit 'f4be82d7efccd85009e4f538b5d52978ba74d4d2' as 'pages/libs/mpxsonar'
3943203 remove subtree bugs
5623f35 Update to latest Note/dev (#3)
b5cca15 Update UI (#2)
be05693 add pages contact, edit app&about pages, add assets folder for imgae files
c82a67e making new repo for mpx ABOUT page
f56f892 making new repo for mpx ABOUT page
7dfdd5f update MPXSonar 0.3.7 (#1)
9112085 update MPXSonar 0.3.7 (#1)
40a838f update config.yml and *.md
db4ba15 setup *.md docs
6de6fe5 setup *.md docs
5a39726 Merge commit '555d93ff42af3d2468eb950ff0d2c8548b8b179e' 0.3.1 into note/dev
555d93f Squashed 'libs/mpxsonar/' changes from 50172ad..ae11fb4
7fc1420 add mpxsonar subtree
b0eddcc Squashed 'libs/mpxsonar/' content from commit 50172ad
511b011 Merge commit 'b0eddcc84014b880deddc0839a64bc4435b8237e' as 'libs/mpxsonar'
3c15519 Initial commit

git-subtree-dir: pages/libs/mpxsonar
git-subtree-split: 91422cfde7b7bdddc4465ec50e52fddd46793a7c

---------

Co-authored-by: K2 <kunaphas.kon@gmail.com>

---------

Co-authored-by: Ferdous Nasri <ferbsx@users.noreply.github.com>

.github/workflows/CD.yml

@@ -22,7 +22,17 @@ jobs:
           cd /opt/MpoxRadar/
           sudo git checkout dev
           sudo git pull
-
+    - name: Create a new cache...
+      uses: appleboy/ssh-action@master
+      with:
+        host: ${{ secrets.SSH_HOST }}
+        username: ${{ secrets.SSH_USERNAME }}
+        key: ${{ secrets.SSH_KEY }}
+        passphrase: ${{ secrets.SSH_PASSPHRASE }}
+        port: 22
+        script: |
+          cd /opt/MpoxRadar/
+          bash build_cache.sh
     - name: Restart the application after deployment
       uses: appleboy/ssh-action@master
       with:

.github/workflows/tests.yml

@@ -62,4 +62,4 @@ jobs:
     - name: Install dependencies
       run: pip install flake8 poetry
     - name: Run flake8 - Code style check
-      run: poetry run flake8 . --config=.flake8 -v
+      run: poetry run flake8 pages/ --config=.flake8 -v

README.md

@@ -1,5 +1,5 @@
 # MpoxRadar-frontend
-MpoxRadar is a worldwide interactive dashboard for genomic surveillance of Mpox (Monkeypox virus).
+MpoxRadar is a worldwide interactive dashboard for genomic surveillance of MPXV (MonkeyPox virus).
 
 
 <img src="docs/MpoxRadar-GraphicalAbstract.drawio.png"

app.py

@@ -35,7 +35,7 @@
                     [
                         html.H1("MpoxRadar", style={"display": "inline-block"}),
                         html.Div(
-                            "An interactive dashboard for genomic surveillance of mpox."
+                            "An interactive dashboard for genomic surveillance of MPXV."
                         ),
                     ]
                 ),

build_cache.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+# eval "$(conda shell.bash hook)"
+# conda activate mpxradar
+source $CONDA_BASE/bin/activate mpxradar
+python data.py

data.py

@@ -12,6 +12,9 @@
 from sqlalchemy import create_engine
 from sqlalchemy import exc
 
+from pages.app_controller import calculate_mutation_sig
+from pages.app_controller import calculate_tri_mutation_sig
+from pages.app_controller import create_snp_table
 from pages.config import CACHE_DIR
 from pages.config import DB_URL
 from pages.config import redis_manager
@@ -206,6 +209,8 @@ def load_db_from_sql(self, table_name: str) -> (str, dict):
         :param table_name: name of the DB table to query
         :return: (path to csv, dtypes dict {column_name: dtype (from column_dtypes)})
         """
+        # cpu_number=  mp.current_process().name.split("-")[-1]  # Get the CPU number
+        # print(cpu_number)
         start = perf_counter()
         db_connection = get_database_connection(self.db_name)
         try:
@@ -509,7 +514,7 @@ def load_all_sql_files(  # noqa: C901
     # check if df_dict is load or not?
     if redis_manager and redis_manager.exists("df_dict") and not test_db:
         # if True:
-        print("Load data from cache")
+        print("Load data from cache...")
         # df_dict = decompress_pickle(os.path.join(CACHE_DIR,"df_dict.pbz2"))
         # df_dict = pickle.loads(redis_manager.get("df_dict"))
         processed_df_dict = load_Cpickle(os.path.join(path_to_cache, "df_dict.pickle"))
@@ -588,3 +593,12 @@ def load_all_sql_files(  # noqa: C901
             # df_dict["referenceView"].to_pickle(".cache/referenceView.pkl")
 
     return processed_df_dict
+
+
+if __name__ == "__main__":
+    print("Build a new cache")
+    load_all_sql_files()
+    create_snp_table()
+    calculate_tri_mutation_sig()
+    calculate_mutation_sig()
+    print("--- Complete ----")

pages/app_controller.py

@@ -230,7 +230,7 @@ def get_value_by_reference(checked_ref):
     for ref in checked_ref:
         print("Query " + ref)
         _df = sonarBasicsChild.match(DB_URL, reference=ref)
-        if type(_df) == str:
+        if type(_df) is str:
             continue
         output_df = pd.concat([output_df, _df], ignore_index=True)
     return output_df
@@ -266,7 +266,7 @@ def get_value_by_filter(checked_ref, mut_checklist, seqtech_checklist):
         _df = sonarBasicsChild.match(
             DB_URL, profiles=mut_profiles, reference=ref, propdict=propdict
         )
-        if type(_df) == str:
+        if type(_df) is str:
             continue
         output_df = pd.concat([output_df, _df], ignore_index=True)
     return output_df

pages/libs/mpxsonar/src/mpxsonar/basics.py

@@ -826,7 +826,7 @@ def exportVCF(cursor, reference, outfile=None, na="*** no match ***"):  # noqa:
     def set_key(dictionary, key, value):
         if key not in dictionary:
             dictionary[key] = value
-        elif type(dictionary[key]) == list:
+        elif type(dictionary[key]) is list:
             dictionary[key].append(value)
         else:
             dictionary[key] = [dictionary[key], value]

pages/tool.py

@@ -609,9 +609,9 @@ def update_output_sonar(n_clicks, commands):  # noqa: C901
             toggle_value = {"display": "block"}
         elif args.tool == "match":
             _tmp_output = match_controller(args)
-            if type(_tmp_output) == int:
+            if type(_tmp_output) is int:
                 output = _tmp_output
-            elif type(_tmp_output) == str:
+            elif type(_tmp_output) is str:
                 output = _tmp_output
             else:
                 df = _tmp_output

pages/util_footer_table.py

@@ -44,32 +44,13 @@
             className="responsive",
         ),
         html.Td(
-            html.Ul(
-                [
-                    html.Li([html.Div(children="Supported by:")]),
-                    html.Li(
-                        [
-                            html.Img(
-                                src=r"assets/Bundesministerium_für_Wirtschaft_und_Energie_Logo.svg.png",
-                                alt="Img_RKI",
-                                style={
-                                    "marginTop": "-10px",
-                                    "height": "auto",
-                                    "minWidth": "100%",
-                                },
-                                className="responsive",
-                            )
-                        ],
-                    ),
-                    html.Li(
-                        [html.Div(children="on the basis of a decision")],
-                        style={"marginTop": "-20px"},
-                    ),
-                    html.Li([html.Div(children="by the German Bundestag")]),
-                ],
-                style={"listStyleType": "none", "textAlign": "center"},
+            html.Img(
+                src=r"assets/BMWK_Logo_en.png",
+                alt="BMWK_logo",
+                style={"height": "auto", "minWidth": "100%", "marginTop": "10px"},
+                className="responsive",
             ),
-            style={"width": "15%"},
+            style={"textAlign": "left", "width": "15%"},
             className="responsive",
         ),
         html.Td(

pages/util_help_tables.py

@@ -32,11 +32,11 @@
 row2 = html.Tr(
     [
         html.Td("MacOS"),
-        html.Td("HighSierra"),
-        html.Td("Not tested"),
-        html.Td("Not tested"),
+        html.Td("Monterey 12.1"),
+        html.Td("115.0.5790.114"),
+        html.Td("115.0.3"),
         html.Td("n/a"),
-        html.Td("Not tested"),
+        html.Td("Version 15.2 (17612.3.6.1.6)"),
     ]
 )
 row3 = html.Tr(

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "mpxradar-frontend"
-version = "0.4.0"
+version = "0.4.2"
 description = "MonkeyPoxRadar"
 authors = ["Kunaphas.Kong <kunaphas.kon@gmail.com>"]