Generation ChemBERT (Chemical Language Model Embedding) Condensation_reactions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8ef8447e-1e4b-4962-8982-7a2102315fd8",
   "metadata": {},
   "source": [
    "# Generation ChemBERT (Chemical Language Model Embedding) Condensation_reactions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22ba55a9-d432-4b10-b8ad-7d2dd5d71ba5",
   "metadata": {},
   "source": [
    "Found this dataset from a recent publication by a D.M. Makarov et al. (Journal of Computational Science 74 (2023) 102173) (https://doi.org/10.1016/j.jocs.2023.102173)\n",
    "\n",
    "\"\"\"We considered pyrrole or dipyrromethane condensation reactions with various aldehydes, resulting in the production of boron(III) dipyrromethene or BODIPY (681 records). These reactions were retrieved from articles (see “Dataset reactions” and Scheme S1). Addi tionally, we used the reactions of the production of dipyrromethane (111 records) and porphyrins (457 records). All condensation reactions for dipyrromethanes and 213 reactions for porphyrins with various al dehydes were obtained in our laboratory. The remaining 244 reactions for the porphyrins synthesis were obtained from articles (see “Dataset reactions”). Our experimental dataset is based on a study of pyrrole condensation processes with aldehydes, using catalytic amounts of organic acids to produce ms-aryl- and ß-alkyl-substituted dipyrro methanes. \"\"\"\n",
    "\n",
    "The objective of this notebook is to introduce a method for predicting yield\n",
    "\n",
    "As a beginner, there may be numerous opportunities for improvement in this notebook. I was largely inspired by the work of D.M. Makarov et al and the STEPHEN LEE's notebook (BELKA: Molecule Representations for ML Tutorial) thanks to them."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "371b98fd-d27f-4e26-bb7a-9efaecc8a32a",
   "metadata": {},
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b142db6d-8e35-4799-adc5-1c563a11d05f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "64d1a451-98a8-4f2a-a1e8-12513aacd312",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.set_option('display.max_row', 211)\n",
    "pd.set_option('display.max_columns',211)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cca43b9b-1699-4134-843a-0738f1af67ca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1249, 4)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SMILES</th>\n",
       "      <th>yield</th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Ind</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CC(C)(C)c1ccc(OCCCCOc2ccc(C(C)(C)C)cc2C=O)c(C=...</td>\n",
       "      <td>0.05</td>\n",
       "      <td>20</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>O=Cc1ccccc1OCCOC(=O)C#CC(=O)OCCOc1ccccc1C=O.c1...</td>\n",
       "      <td>1.00</td>\n",
       "      <td>40</td>\n",
       "      <td>909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c1cc[nH]c1.Cc1cc(C)c(C=O)c(C)c1.COC(=O)c1ccc(C...</td>\n",
       "      <td>2.00</td>\n",
       "      <td>20</td>\n",
       "      <td>747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CC(C)(C)c1ccc(OCCCCCOc2ccc(C(C)(C)C)cc2C=O)c(C...</td>\n",
       "      <td>2.10</td>\n",
       "      <td>20</td>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Cc1ccc(-c2ccccc2C=O)cc1.Fc1c(F)c(F)c(C(c2ccc[n...</td>\n",
       "      <td>2.50</td>\n",
       "      <td>25</td>\n",
       "      <td>266</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              SMILES  yield  Temperature  Ind\n",
       "0  CC(C)(C)c1ccc(OCCCCOc2ccc(C(C)(C)C)cc2C=O)c(C=...   0.05           20   78\n",
       "1  O=Cc1ccccc1OCCOC(=O)C#CC(=O)OCCOc1ccccc1C=O.c1...   1.00           40  909\n",
       "2  c1cc[nH]c1.Cc1cc(C)c(C=O)c(C)c1.COC(=O)c1ccc(C...   2.00           20  747\n",
       "3  CC(C)(C)c1ccc(OCCCCCOc2ccc(C(C)(C)C)cc2C=O)c(C...   2.10           20   77\n",
       "4  Cc1ccc(-c2ccccc2C=O)cc1.Fc1c(F)c(F)c(C(c2ccc[n...   2.50           25  266"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Téléchargement du Dataset \n",
    "DataFrame = r\"C:\\Users\\loris\\Desktop\\IA_chemistry\\Dipyrromethanes_condensation_reactions-main\\Condensation_reactions - Copie.xlsx\"\n",
    "\n",
    "df = pd.read_excel(DataFrame)\n",
    "\n",
    "df= df.copy()\n",
    "\n",
    "print(df.shape)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "52eb7a6f-1430-446a-b9cb-43e77d94449b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'CC(C)(C)c1ccc(OCCCCOc2ccc(C(C)(C)C)cc2C=O)c(C=O)c1.CC(C)(C)c1ccc(OCCCCOc2ccc(C(C)(C)C)cc2C(c2ccc[nH]2)c2ccc[nH]2)c(C(c2ccc[nH]2)c2ccc[nH]2)c1.CCC(=O)O.CC(=O)O.O=[N+]([O-])c1ccccc1>>CC(C)(C)c1ccc2c(c1)-c1c3nc(c4c5ccc([nH]5)c(c5nc(c(c6ccc1[nH]6)-c1cc(C(C)(C)C)ccc1OCCCCOc1ccc(C(C)(C)C)cc1-4)C=C5)-c1cc(C(C)(C)C)ccc1OCCCCO2)C=C3.O.O'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"SMILES\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67ffd110-c6d9-455a-ad96-bb39af5fd3c4",
   "metadata": {},
   "source": [
    "## Missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "5f14037c-4205-451c-960f-ddc6b8af5bad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SMILES         0\n",
       "yield          0\n",
       "Temperature    0\n",
       "Ind            0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Recherche des valeurs manquantes\n",
    "missing_values = df.isnull().sum()\n",
    "missing_values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b0862b97-d681-4219-a66c-36081d285e48",
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'seaborn'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[6], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mseaborn\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01msns\u001b[39;00m\n\u001b[0;32m      2\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mmatplotlib\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mpyplot\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mplt\u001b[39;00m\n\u001b[0;32m      4\u001b[0m \u001b[38;5;66;03m# Création de la figure et des axes pour les subplots\u001b[39;00m\n",
      "\u001b[1;31mModuleNotFoundError\u001b[0m: No module named 'seaborn'"
     ]
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Création de la figure et des axes pour les subplots\n",
    "fig, axs = plt.subplots(2, 1, figsize=(10,8))\n",
    "\n",
    "# Tracé de l'histogramme sur le premier subplot\n",
    "sns.histplot(data=df, x='yield', bins=30, ax=axs[0])\n",
    "axs[0].set_xlabel('Rendement chimique (%)')\n",
    "axs[0].set_ylabel(\"Nombre d'occurences\")\n",
    "axs[0].set_title(\"Distribution des rendements chimiques\")\n",
    "\n",
    "# Tracé du boxplot sur le deuxième subplot\n",
    "sns.boxplot(df[\"yield\"], ax=axs[1])\n",
    "axs[1].set_ylabel(\"Rendement chimique (%)\")\n",
    "axs[1].set_title(\"Distribution des rendements chimiques\")\n",
    "\n",
    "# Ajustement de l'espacement entre les subplots\n",
    "plt.tight_layout()\n",
    "\n",
    "# Affichage des graphiques \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f7dcd422-f8bb-4167-8d5e-35823cc05676",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "389de76c-d44b-4fce-90de-80acf97864eb",
   "metadata": {},
   "source": [
    "## 🤖 Chemical Language Model Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3674f3bb-d80b-4c80-b03f-384f9c58e82c",
   "metadata": {},
   "source": [
    "Just as traditional language models use self-attention mechanisms to compute the representation of each language element (e.g. word in a sentence) to every other element, chemical language models use the same principle in which elements are some chemical unit (e.g. atoms) instead of words.\n",
    "\n",
    "Below is example code to obtain learned transformer-based embeddings from two chemical language models:\n",
    "\n",
    "ChemBERTa: adapted for chemical SMILES from the RoBERTa architecture, trained on a dataset of 77 million molecules\n",
    "\n",
    "MoLFormer: another transformer-based model adapted for SMILES but trained on a larger dataset (1.1 billion molecules!)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6877a21-d421-492e-83c1-5a7c0d7ebef0",
   "metadata": {},
   "source": [
    "\n",
    "References:\n",
    "\n",
    "Chithrananda, S., Grand, G., & Ramsundar, B. (2020). ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction.\n",
    "    \n",
    "Chithrananda, S., Grand, G., & Ramsundar, B. (2022). ChemBERTa-2: Towards Chemical Foundation Models. \n",
    "\n",
    "Ross, J., Belgodere, B., Chenthamarakshan, V., et al. (2022). Large-scale chemical language representations capture molecular structure and properties. \n",
    "Nature Machine Intelligence, 4, 1256-1264. \n",
    "\n",
    "ChemBERTa on HuggingFace Model Repo\n",
    "\n",
    "MolFormer on HuggingFace Model Repo\n",
    "\n",
    "MolFormer GitHub repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "84e180d5-8e82-48cb-ac30-2ba746ae2d6a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\loris\\anaconda3\\envs\\rxnfp\\lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "# for chemical language models\n",
    "from transformers import AutoModel, AutoTokenizer\n",
    "import torch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "65418cc6-adb2-433c-a496-2d891ef8782c",
   "metadata": {},
   "outputs": [],
   "source": [
    "smiles = df[\"SMILES\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ba867e1d-590b-42ce-bcda-7aea52bc05f4",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of RobertaModel were not initialized from the model checkpoint at DeepChem/ChemBERTa-10M-MLM and are newly initialized: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "      <th>20</th>\n",
       "      <th>21</th>\n",
       "      <th>22</th>\n",
       "      <th>23</th>\n",
       "      <th>24</th>\n",
       "      <th>25</th>\n",
       "      <th>26</th>\n",
       "      <th>27</th>\n",
       "      <th>28</th>\n",
       "      <th>29</th>\n",
       "      <th>30</th>\n",
       "      <th>31</th>\n",
       "      <th>32</th>\n",
       "      <th>33</th>\n",
       "      <th>34</th>\n",
       "      <th>35</th>\n",
       "      <th>36</th>\n",
       "      <th>37</th>\n",
       "      <th>38</th>\n",
       "      <th>39</th>\n",
       "      <th>40</th>\n",
       "      <th>41</th>\n",
       "      <th>42</th>\n",
       "      <th>43</th>\n",
       "      <th>44</th>\n",
       "      <th>45</th>\n",
       "      <th>46</th>\n",
       "      <th>47</th>\n",
       "      <th>48</th>\n",
       "      <th>49</th>\n",
       "      <th>50</th>\n",
       "      <th>51</th>\n",
       "      <th>52</th>\n",
       "      <th>53</th>\n",
       "      <th>54</th>\n",
       "      <th>55</th>\n",
       "      <th>56</th>\n",
       "      <th>57</th>\n",
       "      <th>58</th>\n",
       "      <th>59</th>\n",
       "      <th>60</th>\n",
       "      <th>61</th>\n",
       "      <th>62</th>\n",
       "      <th>63</th>\n",
       "      <th>64</th>\n",
       "      <th>65</th>\n",
       "      <th>66</th>\n",
       "      <th>67</th>\n",
       "      <th>68</th>\n",
       "      <th>69</th>\n",
       "      <th>70</th>\n",
       "      <th>71</th>\n",
       "      <th>72</th>\n",
       "      <th>73</th>\n",
       "      <th>74</th>\n",
       "      <th>75</th>\n",
       "      <th>76</th>\n",
       "      <th>77</th>\n",
       "      <th>78</th>\n",
       "      <th>79</th>\n",
       "      <th>80</th>\n",
       "      <th>81</th>\n",
       "      <th>82</th>\n",
       "      <th>83</th>\n",
       "      <th>84</th>\n",
       "      <th>85</th>\n",
       "      <th>86</th>\n",
       "      <th>87</th>\n",
       "      <th>88</th>\n",
       "      <th>89</th>\n",
       "      <th>90</th>\n",
       "      <th>91</th>\n",
       "      <th>92</th>\n",
       "      <th>93</th>\n",
       "      <th>94</th>\n",
       "      <th>95</th>\n",
       "      <th>96</th>\n",
       "      <th>97</th>\n",
       "      <th>98</th>\n",
       "      <th>99</th>\n",
       "      <th>100</th>\n",
       "      <th>101</th>\n",
       "      <th>102</th>\n",
       "      <th>103</th>\n",
       "      <th>104</th>\n",
       "      <th>...</th>\n",
       "      <th>279</th>\n",
       "      <th>280</th>\n",
       "      <th>281</th>\n",
       "      <th>282</th>\n",
       "      <th>283</th>\n",
       "      <th>284</th>\n",
       "      <th>285</th>\n",
       "      <th>286</th>\n",
       "      <th>287</th>\n",
       "      <th>288</th>\n",
       "      <th>289</th>\n",
       "      <th>290</th>\n",
       "      <th>291</th>\n",
       "      <th>292</th>\n",
       "      <th>293</th>\n",
       "      <th>294</th>\n",
       "      <th>295</th>\n",
       "      <th>296</th>\n",
       "      <th>297</th>\n",
       "      <th>298</th>\n",
       "      <th>299</th>\n",
       "      <th>300</th>\n",
       "      <th>301</th>\n",
       "      <th>302</th>\n",
       "      <th>303</th>\n",
       "      <th>304</th>\n",
       "      <th>305</th>\n",
       "      <th>306</th>\n",
       "      <th>307</th>\n",
       "      <th>308</th>\n",
       "      <th>309</th>\n",
       "      <th>310</th>\n",
       "      <th>311</th>\n",
       "      <th>312</th>\n",
       "      <th>313</th>\n",
       "      <th>314</th>\n",
       "      <th>315</th>\n",
       "      <th>316</th>\n",
       "      <th>317</th>\n",
       "      <th>318</th>\n",
       "      <th>319</th>\n",
       "      <th>320</th>\n",
       "      <th>321</th>\n",
       "      <th>322</th>\n",
       "      <th>323</th>\n",
       "      <th>324</th>\n",
       "      <th>325</th>\n",
       "      <th>326</th>\n",
       "      <th>327</th>\n",
       "      <th>328</th>\n",
       "      <th>329</th>\n",
       "      <th>330</th>\n",
       "      <th>331</th>\n",
       "      <th>332</th>\n",
       "      <th>333</th>\n",
       "      <th>334</th>\n",
       "      <th>335</th>\n",
       "      <th>336</th>\n",
       "      <th>337</th>\n",
       "      <th>338</th>\n",
       "      <th>339</th>\n",
       "      <th>340</th>\n",
       "      <th>341</th>\n",
       "      <th>342</th>\n",
       "      <th>343</th>\n",
       "      <th>344</th>\n",
       "      <th>345</th>\n",
       "      <th>346</th>\n",
       "      <th>347</th>\n",
       "      <th>348</th>\n",
       "      <th>349</th>\n",
       "      <th>350</th>\n",
       "      <th>351</th>\n",
       "      <th>352</th>\n",
       "      <th>353</th>\n",
       "      <th>354</th>\n",
       "      <th>355</th>\n",
       "      <th>356</th>\n",
       "      <th>357</th>\n",
       "      <th>358</th>\n",
       "      <th>359</th>\n",
       "      <th>360</th>\n",
       "      <th>361</th>\n",
       "      <th>362</th>\n",
       "      <th>363</th>\n",
       "      <th>364</th>\n",
       "      <th>365</th>\n",
       "      <th>366</th>\n",
       "      <th>367</th>\n",
       "      <th>368</th>\n",
       "      <th>369</th>\n",
       "      <th>370</th>\n",
       "      <th>371</th>\n",
       "      <th>372</th>\n",
       "      <th>373</th>\n",
       "      <th>374</th>\n",
       "      <th>375</th>\n",
       "      <th>376</th>\n",
       "      <th>377</th>\n",
       "      <th>378</th>\n",
       "      <th>379</th>\n",
       "      <th>380</th>\n",
       "      <th>381</th>\n",
       "      <th>382</th>\n",
       "      <th>383</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.162271</td>\n",
       "      <td>0.359961</td>\n",
       "      <td>-0.208631</td>\n",
       "      <td>0.212792</td>\n",
       "      <td>-0.184559</td>\n",
       "      <td>0.023665</td>\n",
       "      <td>0.031174</td>\n",
       "      <td>-0.056128</td>\n",
       "      <td>0.070213</td>\n",
       "      <td>0.246463</td>\n",
       "      <td>0.404678</td>\n",
       "      <td>0.222561</td>\n",
       "      <td>0.246697</td>\n",
       "      <td>0.569248</td>\n",
       "      <td>-0.286271</td>\n",
       "      <td>0.163123</td>\n",
       "      <td>-0.211469</td>\n",
       "      <td>-0.137589</td>\n",
       "      <td>0.333122</td>\n",
       "      <td>0.270672</td>\n",
       "      <td>-0.171357</td>\n",
       "      <td>-0.007442</td>\n",
       "      <td>0.026922</td>\n",
       "      <td>0.093674</td>\n",
       "      <td>0.563860</td>\n",
       "      <td>0.190829</td>\n",
       "      <td>-0.260796</td>\n",
       "      <td>0.335964</td>\n",
       "      <td>0.166725</td>\n",
       "      <td>-0.452981</td>\n",
       "      <td>-0.144069</td>\n",
       "      <td>0.137669</td>\n",
       "      <td>0.074681</td>\n",
       "      <td>-0.076224</td>\n",
       "      <td>0.409972</td>\n",
       "      <td>-0.091544</td>\n",
       "      <td>-0.143137</td>\n",
       "      <td>-0.533438</td>\n",
       "      <td>-0.177174</td>\n",
       "      <td>0.303646</td>\n",
       "      <td>-0.170809</td>\n",
       "      <td>0.113998</td>\n",
       "      <td>-0.180275</td>\n",
       "      <td>-0.298499</td>\n",
       "      <td>0.129155</td>\n",
       "      <td>-0.358678</td>\n",
       "      <td>-0.197952</td>\n",
       "      <td>-0.127823</td>\n",
       "      <td>0.095293</td>\n",
       "      <td>0.146926</td>\n",
       "      <td>0.200765</td>\n",
       "      <td>-0.178596</td>\n",
       "      <td>0.116211</td>\n",
       "      <td>-0.196614</td>\n",
       "      <td>0.198353</td>\n",
       "      <td>0.067136</td>\n",
       "      <td>0.164220</td>\n",
       "      <td>0.255172</td>\n",
       "      <td>-0.430976</td>\n",
       "      <td>-0.052471</td>\n",
       "      <td>0.122035</td>\n",
       "      <td>-0.120684</td>\n",
       "      <td>0.194026</td>\n",
       "      <td>0.059159</td>\n",
       "      <td>-0.242641</td>\n",
       "      <td>0.216599</td>\n",
       "      <td>-0.209428</td>\n",
       "      <td>-0.450314</td>\n",
       "      <td>-0.067776</td>\n",
       "      <td>0.322422</td>\n",
       "      <td>0.050678</td>\n",
       "      <td>-0.037782</td>\n",
       "      <td>0.081106</td>\n",
       "      <td>0.362837</td>\n",
       "      <td>-0.256037</td>\n",
       "      <td>-0.186787</td>\n",
       "      <td>-0.048138</td>\n",
       "      <td>-0.397506</td>\n",
       "      <td>-0.240361</td>\n",
       "      <td>-0.247374</td>\n",
       "      <td>0.147705</td>\n",
       "      <td>0.132116</td>\n",
       "      <td>0.320903</td>\n",
       "      <td>-0.256691</td>\n",
       "      <td>-0.021490</td>\n",
       "      <td>-0.365033</td>\n",
       "      <td>-0.155543</td>\n",
       "      <td>0.344853</td>\n",
       "      <td>0.302176</td>\n",
       "      <td>-0.139125</td>\n",
       "      <td>0.172635</td>\n",
       "      <td>-0.128772</td>\n",
       "      <td>0.321327</td>\n",
       "      <td>0.151302</td>\n",
       "      <td>-0.063909</td>\n",
       "      <td>-0.135832</td>\n",
       "      <td>0.332110</td>\n",
       "      <td>-0.003239</td>\n",
       "      <td>0.258752</td>\n",
       "      <td>0.012680</td>\n",
       "      <td>-0.174956</td>\n",
       "      <td>0.260750</td>\n",
       "      <td>0.035437</td>\n",
       "      <td>0.454084</td>\n",
       "      <td>0.084206</td>\n",
       "      <td>...</td>\n",
       "      <td>0.325775</td>\n",
       "      <td>0.087837</td>\n",
       "      <td>-0.047203</td>\n",
       "      <td>-0.375325</td>\n",
       "      <td>0.240884</td>\n",
       "      <td>0.096204</td>\n",
       "      <td>0.210107</td>\n",
       "      <td>0.152762</td>\n",
       "      <td>-0.285373</td>\n",
       "      <td>-0.281662</td>\n",
       "      <td>0.243171</td>\n",
       "      <td>0.335456</td>\n",
       "      <td>0.193422</td>\n",
       "      <td>-0.344888</td>\n",
       "      <td>-0.180593</td>\n",
       "      <td>0.024781</td>\n",
       "      <td>-0.214996</td>\n",
       "      <td>-0.305409</td>\n",
       "      <td>-0.109843</td>\n",
       "      <td>-0.137799</td>\n",
       "      <td>-0.499643</td>\n",
       "      <td>0.295993</td>\n",
       "      <td>0.223503</td>\n",
       "      <td>0.012261</td>\n",
       "      <td>-0.147910</td>\n",
       "      <td>0.103415</td>\n",
       "      <td>-0.091400</td>\n",
       "      <td>0.090393</td>\n",
       "      <td>0.178400</td>\n",
       "      <td>-0.128588</td>\n",
       "      <td>0.159457</td>\n",
       "      <td>0.131086</td>\n",
       "      <td>-0.192747</td>\n",
       "      <td>0.246661</td>\n",
       "      <td>-0.018331</td>\n",
       "      <td>-0.004980</td>\n",
       "      <td>0.066220</td>\n",
       "      <td>0.229715</td>\n",
       "      <td>0.022504</td>\n",
       "      <td>-0.275646</td>\n",
       "      <td>0.057845</td>\n",
       "      <td>0.330876</td>\n",
       "      <td>0.000617</td>\n",
       "      <td>-0.515452</td>\n",
       "      <td>0.235576</td>\n",
       "      <td>0.299304</td>\n",
       "      <td>0.124795</td>\n",
       "      <td>0.086530</td>\n",
       "      <td>0.011721</td>\n",
       "      <td>-0.124608</td>\n",
       "      <td>0.353909</td>\n",
       "      <td>0.176914</td>\n",
       "      <td>0.021789</td>\n",
       "      <td>-0.560504</td>\n",
       "      <td>-0.019877</td>\n",
       "      <td>0.027930</td>\n",
       "      <td>-0.026669</td>\n",
       "      <td>-0.124045</td>\n",
       "      <td>0.082781</td>\n",
       "      <td>-0.416438</td>\n",
       "      <td>0.100424</td>\n",
       "      <td>0.171194</td>\n",
       "      <td>-0.240402</td>\n",
       "      <td>-0.120383</td>\n",
       "      <td>-0.101247</td>\n",
       "      <td>-0.219663</td>\n",
       "      <td>-0.180439</td>\n",
       "      <td>0.171812</td>\n",
       "      <td>-0.378690</td>\n",
       "      <td>-0.122113</td>\n",
       "      <td>0.160866</td>\n",
       "      <td>0.131128</td>\n",
       "      <td>0.284670</td>\n",
       "      <td>0.330584</td>\n",
       "      <td>-0.425320</td>\n",
       "      <td>0.112695</td>\n",
       "      <td>0.506408</td>\n",
       "      <td>0.361750</td>\n",
       "      <td>-0.046845</td>\n",
       "      <td>0.007392</td>\n",
       "      <td>-0.070656</td>\n",
       "      <td>-0.199145</td>\n",
       "      <td>0.104572</td>\n",
       "      <td>-0.008363</td>\n",
       "      <td>-0.072674</td>\n",
       "      <td>-0.080588</td>\n",
       "      <td>-0.084459</td>\n",
       "      <td>0.144243</td>\n",
       "      <td>0.227459</td>\n",
       "      <td>-0.067456</td>\n",
       "      <td>-0.541151</td>\n",
       "      <td>0.081943</td>\n",
       "      <td>-0.373041</td>\n",
       "      <td>-0.345800</td>\n",
       "      <td>0.036262</td>\n",
       "      <td>0.095997</td>\n",
       "      <td>-0.096873</td>\n",
       "      <td>-0.063860</td>\n",
       "      <td>-0.114892</td>\n",
       "      <td>0.092941</td>\n",
       "      <td>-0.445563</td>\n",
       "      <td>-0.034386</td>\n",
       "      <td>-0.321318</td>\n",
       "      <td>-0.407256</td>\n",
       "      <td>0.045911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.164998</td>\n",
       "      <td>0.200312</td>\n",
       "      <td>-0.291708</td>\n",
       "      <td>0.385741</td>\n",
       "      <td>-0.113083</td>\n",
       "      <td>0.090308</td>\n",
       "      <td>-0.015352</td>\n",
       "      <td>0.234842</td>\n",
       "      <td>0.140149</td>\n",
       "      <td>0.200789</td>\n",
       "      <td>0.300456</td>\n",
       "      <td>0.268847</td>\n",
       "      <td>0.327487</td>\n",
       "      <td>0.552415</td>\n",
       "      <td>-0.058277</td>\n",
       "      <td>0.069845</td>\n",
       "      <td>-0.283775</td>\n",
       "      <td>0.059433</td>\n",
       "      <td>0.132053</td>\n",
       "      <td>0.206883</td>\n",
       "      <td>0.020623</td>\n",
       "      <td>-0.173855</td>\n",
       "      <td>0.225282</td>\n",
       "      <td>-0.017904</td>\n",
       "      <td>0.497691</td>\n",
       "      <td>0.216123</td>\n",
       "      <td>-0.304758</td>\n",
       "      <td>0.297272</td>\n",
       "      <td>0.063490</td>\n",
       "      <td>-0.155111</td>\n",
       "      <td>0.192790</td>\n",
       "      <td>0.090803</td>\n",
       "      <td>0.076976</td>\n",
       "      <td>0.226654</td>\n",
       "      <td>0.310238</td>\n",
       "      <td>-0.049298</td>\n",
       "      <td>0.095382</td>\n",
       "      <td>-0.430092</td>\n",
       "      <td>-0.348674</td>\n",
       "      <td>-0.033861</td>\n",
       "      <td>0.297530</td>\n",
       "      <td>0.052408</td>\n",
       "      <td>-0.273249</td>\n",
       "      <td>-0.018303</td>\n",
       "      <td>0.000791</td>\n",
       "      <td>-0.184625</td>\n",
       "      <td>-0.371987</td>\n",
       "      <td>-0.196453</td>\n",
       "      <td>0.265759</td>\n",
       "      <td>-0.089040</td>\n",
       "      <td>0.163914</td>\n",
       "      <td>-0.434184</td>\n",
       "      <td>0.054147</td>\n",
       "      <td>-0.180023</td>\n",
       "      <td>0.367596</td>\n",
       "      <td>0.134108</td>\n",
       "      <td>0.269924</td>\n",
       "      <td>0.013862</td>\n",
       "      <td>-0.086283</td>\n",
       "      <td>0.153577</td>\n",
       "      <td>-0.441175</td>\n",
       "      <td>-0.102963</td>\n",
       "      <td>0.169320</td>\n",
       "      <td>0.168424</td>\n",
       "      <td>-0.019549</td>\n",
       "      <td>0.066295</td>\n",
       "      <td>0.027417</td>\n",
       "      <td>-0.022250</td>\n",
       "      <td>0.115971</td>\n",
       "      <td>-0.026206</td>\n",
       "      <td>0.111903</td>\n",
       "      <td>0.134881</td>\n",
       "      <td>0.163826</td>\n",
       "      <td>0.415292</td>\n",
       "      <td>-0.045816</td>\n",
       "      <td>0.029169</td>\n",
       "      <td>-0.002534</td>\n",
       "      <td>-0.437910</td>\n",
       "      <td>-0.069225</td>\n",
       "      <td>0.127752</td>\n",
       "      <td>-0.087582</td>\n",
       "      <td>0.209030</td>\n",
       "      <td>-0.017341</td>\n",
       "      <td>-0.325876</td>\n",
       "      <td>0.031164</td>\n",
       "      <td>-0.381508</td>\n",
       "      <td>-0.209271</td>\n",
       "      <td>0.060853</td>\n",
       "      <td>-0.095384</td>\n",
       "      <td>-0.291828</td>\n",
       "      <td>-0.112984</td>\n",
       "      <td>-0.085400</td>\n",
       "      <td>-0.271536</td>\n",
       "      <td>-0.287510</td>\n",
       "      <td>-0.201039</td>\n",
       "      <td>-0.028334</td>\n",
       "      <td>0.257096</td>\n",
       "      <td>0.126726</td>\n",
       "      <td>-0.001215</td>\n",
       "      <td>0.055023</td>\n",
       "      <td>0.019923</td>\n",
       "      <td>0.145393</td>\n",
       "      <td>0.170775</td>\n",
       "      <td>0.345924</td>\n",
       "      <td>-0.001274</td>\n",
       "      <td>...</td>\n",
       "      <td>0.059508</td>\n",
       "      <td>0.037455</td>\n",
       "      <td>-0.226644</td>\n",
       "      <td>-0.200301</td>\n",
       "      <td>0.135569</td>\n",
       "      <td>-0.035710</td>\n",
       "      <td>0.130619</td>\n",
       "      <td>0.011335</td>\n",
       "      <td>-0.017847</td>\n",
       "      <td>-0.136202</td>\n",
       "      <td>0.259121</td>\n",
       "      <td>-0.040067</td>\n",
       "      <td>0.328404</td>\n",
       "      <td>-0.241552</td>\n",
       "      <td>0.123093</td>\n",
       "      <td>-0.204784</td>\n",
       "      <td>-0.206881</td>\n",
       "      <td>-0.301866</td>\n",
       "      <td>-0.098708</td>\n",
       "      <td>-0.209272</td>\n",
       "      <td>-0.543192</td>\n",
       "      <td>0.190023</td>\n",
       "      <td>0.235180</td>\n",
       "      <td>-0.119917</td>\n",
       "      <td>0.086322</td>\n",
       "      <td>0.212423</td>\n",
       "      <td>-0.129963</td>\n",
       "      <td>0.174801</td>\n",
       "      <td>0.025072</td>\n",
       "      <td>-0.205548</td>\n",
       "      <td>0.252248</td>\n",
       "      <td>0.091975</td>\n",
       "      <td>0.292690</td>\n",
       "      <td>0.279312</td>\n",
       "      <td>0.064004</td>\n",
       "      <td>0.029540</td>\n",
       "      <td>-0.140886</td>\n",
       "      <td>0.274939</td>\n",
       "      <td>-0.214205</td>\n",
       "      <td>0.232435</td>\n",
       "      <td>-0.021775</td>\n",
       "      <td>0.326521</td>\n",
       "      <td>-0.173589</td>\n",
       "      <td>-0.327937</td>\n",
       "      <td>0.211141</td>\n",
       "      <td>0.306258</td>\n",
       "      <td>-0.026692</td>\n",
       "      <td>0.191214</td>\n",
       "      <td>-0.011201</td>\n",
       "      <td>-0.158324</td>\n",
       "      <td>0.485520</td>\n",
       "      <td>0.042530</td>\n",
       "      <td>0.131386</td>\n",
       "      <td>-0.392145</td>\n",
       "      <td>-0.045617</td>\n",
       "      <td>0.089598</td>\n",
       "      <td>0.010491</td>\n",
       "      <td>-0.237086</td>\n",
       "      <td>0.047300</td>\n",
       "      <td>-0.110376</td>\n",
       "      <td>-0.070836</td>\n",
       "      <td>-0.116565</td>\n",
       "      <td>0.031171</td>\n",
       "      <td>-0.102659</td>\n",
       "      <td>0.007115</td>\n",
       "      <td>-0.085476</td>\n",
       "      <td>-0.344428</td>\n",
       "      <td>-0.038877</td>\n",
       "      <td>-0.376358</td>\n",
       "      <td>0.223386</td>\n",
       "      <td>0.124038</td>\n",
       "      <td>0.318288</td>\n",
       "      <td>0.010998</td>\n",
       "      <td>0.290550</td>\n",
       "      <td>-0.214707</td>\n",
       "      <td>0.292548</td>\n",
       "      <td>0.314998</td>\n",
       "      <td>0.234466</td>\n",
       "      <td>0.075865</td>\n",
       "      <td>0.162426</td>\n",
       "      <td>0.147017</td>\n",
       "      <td>-0.085037</td>\n",
       "      <td>-0.049484</td>\n",
       "      <td>-0.181991</td>\n",
       "      <td>0.153568</td>\n",
       "      <td>-0.151253</td>\n",
       "      <td>0.183836</td>\n",
       "      <td>0.133308</td>\n",
       "      <td>0.088930</td>\n",
       "      <td>-0.268663</td>\n",
       "      <td>-0.258682</td>\n",
       "      <td>0.156034</td>\n",
       "      <td>-0.331153</td>\n",
       "      <td>-0.366319</td>\n",
       "      <td>-0.134505</td>\n",
       "      <td>-0.029372</td>\n",
       "      <td>0.222066</td>\n",
       "      <td>0.003363</td>\n",
       "      <td>-0.056301</td>\n",
       "      <td>-0.256319</td>\n",
       "      <td>-0.349865</td>\n",
       "      <td>0.015541</td>\n",
       "      <td>-0.251664</td>\n",
       "      <td>-0.024831</td>\n",
       "      <td>-0.041127</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.087443</td>\n",
       "      <td>0.348354</td>\n",
       "      <td>0.075141</td>\n",
       "      <td>0.265771</td>\n",
       "      <td>-0.317597</td>\n",
       "      <td>-0.080730</td>\n",
       "      <td>0.031378</td>\n",
       "      <td>-0.159422</td>\n",
       "      <td>0.076782</td>\n",
       "      <td>0.450581</td>\n",
       "      <td>0.020486</td>\n",
       "      <td>0.412668</td>\n",
       "      <td>0.452576</td>\n",
       "      <td>0.266731</td>\n",
       "      <td>-0.088604</td>\n",
       "      <td>0.035567</td>\n",
       "      <td>-0.181696</td>\n",
       "      <td>-0.197046</td>\n",
       "      <td>0.319921</td>\n",
       "      <td>0.329228</td>\n",
       "      <td>-0.230394</td>\n",
       "      <td>-0.054677</td>\n",
       "      <td>0.264726</td>\n",
       "      <td>0.104968</td>\n",
       "      <td>0.342561</td>\n",
       "      <td>0.441863</td>\n",
       "      <td>-0.341266</td>\n",
       "      <td>0.059278</td>\n",
       "      <td>0.336389</td>\n",
       "      <td>-0.076413</td>\n",
       "      <td>0.208437</td>\n",
       "      <td>0.191560</td>\n",
       "      <td>0.204068</td>\n",
       "      <td>0.083635</td>\n",
       "      <td>0.235593</td>\n",
       "      <td>-0.106136</td>\n",
       "      <td>0.289786</td>\n",
       "      <td>-0.252061</td>\n",
       "      <td>-0.266143</td>\n",
       "      <td>0.307279</td>\n",
       "      <td>-0.119827</td>\n",
       "      <td>-0.172364</td>\n",
       "      <td>-0.145971</td>\n",
       "      <td>-0.299094</td>\n",
       "      <td>0.357934</td>\n",
       "      <td>-0.059064</td>\n",
       "      <td>-0.443192</td>\n",
       "      <td>-0.211800</td>\n",
       "      <td>0.157538</td>\n",
       "      <td>0.082950</td>\n",
       "      <td>-0.139568</td>\n",
       "      <td>-0.335272</td>\n",
       "      <td>0.136966</td>\n",
       "      <td>-0.431957</td>\n",
       "      <td>0.064600</td>\n",
       "      <td>-0.038943</td>\n",
       "      <td>0.230240</td>\n",
       "      <td>0.004617</td>\n",
       "      <td>-0.389729</td>\n",
       "      <td>-0.042800</td>\n",
       "      <td>-0.323701</td>\n",
       "      <td>0.019915</td>\n",
       "      <td>0.090443</td>\n",
       "      <td>-0.200131</td>\n",
       "      <td>-0.156367</td>\n",
       "      <td>-0.051447</td>\n",
       "      <td>-0.080336</td>\n",
       "      <td>-0.053585</td>\n",
       "      <td>0.072576</td>\n",
       "      <td>0.029933</td>\n",
       "      <td>0.189949</td>\n",
       "      <td>0.046997</td>\n",
       "      <td>0.251135</td>\n",
       "      <td>0.505619</td>\n",
       "      <td>-0.113831</td>\n",
       "      <td>-0.042439</td>\n",
       "      <td>0.008957</td>\n",
       "      <td>-0.528493</td>\n",
       "      <td>-0.237793</td>\n",
       "      <td>-0.154873</td>\n",
       "      <td>0.127994</td>\n",
       "      <td>0.068931</td>\n",
       "      <td>0.257621</td>\n",
       "      <td>-0.324349</td>\n",
       "      <td>-0.068819</td>\n",
       "      <td>-0.075986</td>\n",
       "      <td>-0.260203</td>\n",
       "      <td>0.257970</td>\n",
       "      <td>-0.067435</td>\n",
       "      <td>-0.132135</td>\n",
       "      <td>0.114913</td>\n",
       "      <td>-0.042286</td>\n",
       "      <td>-0.296356</td>\n",
       "      <td>-0.137618</td>\n",
       "      <td>0.058961</td>\n",
       "      <td>-0.165032</td>\n",
       "      <td>0.498271</td>\n",
       "      <td>-0.165034</td>\n",
       "      <td>0.341459</td>\n",
       "      <td>0.013973</td>\n",
       "      <td>-0.128801</td>\n",
       "      <td>0.129680</td>\n",
       "      <td>-0.112283</td>\n",
       "      <td>0.303148</td>\n",
       "      <td>-0.040948</td>\n",
       "      <td>...</td>\n",
       "      <td>0.332492</td>\n",
       "      <td>0.025870</td>\n",
       "      <td>-0.151847</td>\n",
       "      <td>-0.217514</td>\n",
       "      <td>0.112499</td>\n",
       "      <td>-0.370836</td>\n",
       "      <td>0.152899</td>\n",
       "      <td>0.015132</td>\n",
       "      <td>-0.226122</td>\n",
       "      <td>-0.105108</td>\n",
       "      <td>0.320797</td>\n",
       "      <td>0.272942</td>\n",
       "      <td>0.320707</td>\n",
       "      <td>-0.060986</td>\n",
       "      <td>-0.301983</td>\n",
       "      <td>-0.103061</td>\n",
       "      <td>-0.183105</td>\n",
       "      <td>-0.022568</td>\n",
       "      <td>-0.245966</td>\n",
       "      <td>-0.115399</td>\n",
       "      <td>-0.429196</td>\n",
       "      <td>0.369226</td>\n",
       "      <td>0.339843</td>\n",
       "      <td>0.007097</td>\n",
       "      <td>0.079089</td>\n",
       "      <td>-0.031242</td>\n",
       "      <td>-0.007945</td>\n",
       "      <td>-0.031585</td>\n",
       "      <td>0.197342</td>\n",
       "      <td>-0.037872</td>\n",
       "      <td>0.163866</td>\n",
       "      <td>-0.137193</td>\n",
       "      <td>-0.108474</td>\n",
       "      <td>0.155755</td>\n",
       "      <td>-0.125552</td>\n",
       "      <td>-0.024259</td>\n",
       "      <td>-0.114572</td>\n",
       "      <td>0.037851</td>\n",
       "      <td>-0.132192</td>\n",
       "      <td>-0.036920</td>\n",
       "      <td>0.108299</td>\n",
       "      <td>0.408869</td>\n",
       "      <td>0.094320</td>\n",
       "      <td>-0.339686</td>\n",
       "      <td>0.086932</td>\n",
       "      <td>0.372600</td>\n",
       "      <td>0.080849</td>\n",
       "      <td>0.171356</td>\n",
       "      <td>-0.155160</td>\n",
       "      <td>-0.173940</td>\n",
       "      <td>0.399333</td>\n",
       "      <td>0.123328</td>\n",
       "      <td>-0.080917</td>\n",
       "      <td>-0.506595</td>\n",
       "      <td>-0.073971</td>\n",
       "      <td>0.098760</td>\n",
       "      <td>0.042386</td>\n",
       "      <td>-0.131056</td>\n",
       "      <td>0.021697</td>\n",
       "      <td>-0.016402</td>\n",
       "      <td>-0.091808</td>\n",
       "      <td>0.026021</td>\n",
       "      <td>-0.084562</td>\n",
       "      <td>0.027770</td>\n",
       "      <td>-0.086904</td>\n",
       "      <td>-0.062518</td>\n",
       "      <td>-0.211319</td>\n",
       "      <td>0.055575</td>\n",
       "      <td>-0.173359</td>\n",
       "      <td>-0.016561</td>\n",
       "      <td>-0.078423</td>\n",
       "      <td>0.076999</td>\n",
       "      <td>0.092031</td>\n",
       "      <td>0.185906</td>\n",
       "      <td>-0.532436</td>\n",
       "      <td>0.376591</td>\n",
       "      <td>0.106628</td>\n",
       "      <td>0.114349</td>\n",
       "      <td>0.102644</td>\n",
       "      <td>0.191851</td>\n",
       "      <td>0.037620</td>\n",
       "      <td>-0.343103</td>\n",
       "      <td>-0.184611</td>\n",
       "      <td>-0.083883</td>\n",
       "      <td>0.058932</td>\n",
       "      <td>0.040169</td>\n",
       "      <td>-0.105352</td>\n",
       "      <td>0.038794</td>\n",
       "      <td>-0.052485</td>\n",
       "      <td>0.007382</td>\n",
       "      <td>-0.301429</td>\n",
       "      <td>0.043780</td>\n",
       "      <td>-0.478187</td>\n",
       "      <td>-0.103193</td>\n",
       "      <td>-0.006233</td>\n",
       "      <td>0.230008</td>\n",
       "      <td>0.009344</td>\n",
       "      <td>-0.184263</td>\n",
       "      <td>0.031506</td>\n",
       "      <td>0.062032</td>\n",
       "      <td>-0.151290</td>\n",
       "      <td>0.162426</td>\n",
       "      <td>-0.353901</td>\n",
       "      <td>-0.378658</td>\n",
       "      <td>-0.271250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.176260</td>\n",
       "      <td>0.331500</td>\n",
       "      <td>-0.217019</td>\n",
       "      <td>0.218216</td>\n",
       "      <td>-0.184135</td>\n",
       "      <td>0.013105</td>\n",
       "      <td>0.025132</td>\n",
       "      <td>-0.038927</td>\n",
       "      <td>0.075716</td>\n",
       "      <td>0.217835</td>\n",
       "      <td>0.394953</td>\n",
       "      <td>0.209932</td>\n",
       "      <td>0.223927</td>\n",
       "      <td>0.594058</td>\n",
       "      <td>-0.284427</td>\n",
       "      <td>0.171727</td>\n",
       "      <td>-0.221581</td>\n",
       "      <td>-0.123168</td>\n",
       "      <td>0.306397</td>\n",
       "      <td>0.265301</td>\n",
       "      <td>-0.183306</td>\n",
       "      <td>0.007575</td>\n",
       "      <td>0.060609</td>\n",
       "      <td>0.096721</td>\n",
       "      <td>0.556450</td>\n",
       "      <td>0.177043</td>\n",
       "      <td>-0.261085</td>\n",
       "      <td>0.320190</td>\n",
       "      <td>0.155211</td>\n",
       "      <td>-0.436180</td>\n",
       "      <td>-0.188859</td>\n",
       "      <td>0.121432</td>\n",
       "      <td>0.058139</td>\n",
       "      <td>-0.102365</td>\n",
       "      <td>0.389624</td>\n",
       "      <td>-0.113831</td>\n",
       "      <td>-0.169072</td>\n",
       "      <td>-0.526837</td>\n",
       "      <td>-0.186467</td>\n",
       "      <td>0.306183</td>\n",
       "      <td>-0.118291</td>\n",
       "      <td>0.102877</td>\n",
       "      <td>-0.162771</td>\n",
       "      <td>-0.291369</td>\n",
       "      <td>0.117419</td>\n",
       "      <td>-0.337459</td>\n",
       "      <td>-0.190894</td>\n",
       "      <td>-0.094893</td>\n",
       "      <td>0.122876</td>\n",
       "      <td>0.110975</td>\n",
       "      <td>0.203009</td>\n",
       "      <td>-0.161310</td>\n",
       "      <td>0.103420</td>\n",
       "      <td>-0.196771</td>\n",
       "      <td>0.209090</td>\n",
       "      <td>0.066207</td>\n",
       "      <td>0.156469</td>\n",
       "      <td>0.282545</td>\n",
       "      <td>-0.436368</td>\n",
       "      <td>-0.041194</td>\n",
       "      <td>0.152029</td>\n",
       "      <td>-0.163704</td>\n",
       "      <td>0.208622</td>\n",
       "      <td>0.081991</td>\n",
       "      <td>-0.222724</td>\n",
       "      <td>0.200585</td>\n",
       "      <td>-0.189137</td>\n",
       "      <td>-0.445165</td>\n",
       "      <td>-0.061106</td>\n",
       "      <td>0.329794</td>\n",
       "      <td>0.043275</td>\n",
       "      <td>-0.031708</td>\n",
       "      <td>0.044975</td>\n",
       "      <td>0.330188</td>\n",
       "      <td>-0.257523</td>\n",
       "      <td>-0.191809</td>\n",
       "      <td>-0.026744</td>\n",
       "      <td>-0.394451</td>\n",
       "      <td>-0.216947</td>\n",
       "      <td>-0.258086</td>\n",
       "      <td>0.136926</td>\n",
       "      <td>0.127646</td>\n",
       "      <td>0.341987</td>\n",
       "      <td>-0.242775</td>\n",
       "      <td>-0.020420</td>\n",
       "      <td>-0.342146</td>\n",
       "      <td>-0.134044</td>\n",
       "      <td>0.349911</td>\n",
       "      <td>0.297647</td>\n",
       "      <td>-0.128103</td>\n",
       "      <td>0.177103</td>\n",
       "      <td>-0.146584</td>\n",
       "      <td>0.326321</td>\n",
       "      <td>0.154350</td>\n",
       "      <td>-0.100294</td>\n",
       "      <td>-0.146707</td>\n",
       "      <td>0.318168</td>\n",
       "      <td>0.012583</td>\n",
       "      <td>0.245729</td>\n",
       "      <td>-0.015691</td>\n",
       "      <td>-0.147501</td>\n",
       "      <td>0.265394</td>\n",
       "      <td>0.018757</td>\n",
       "      <td>0.452250</td>\n",
       "      <td>0.058254</td>\n",
       "      <td>...</td>\n",
       "      <td>0.286873</td>\n",
       "      <td>0.075731</td>\n",
       "      <td>-0.060459</td>\n",
       "      <td>-0.369617</td>\n",
       "      <td>0.230923</td>\n",
       "      <td>0.091402</td>\n",
       "      <td>0.222454</td>\n",
       "      <td>0.119655</td>\n",
       "      <td>-0.288558</td>\n",
       "      <td>-0.329720</td>\n",
       "      <td>0.243055</td>\n",
       "      <td>0.333877</td>\n",
       "      <td>0.196843</td>\n",
       "      <td>-0.336662</td>\n",
       "      <td>-0.154158</td>\n",
       "      <td>0.026134</td>\n",
       "      <td>-0.223284</td>\n",
       "      <td>-0.306917</td>\n",
       "      <td>-0.098450</td>\n",
       "      <td>-0.144181</td>\n",
       "      <td>-0.496922</td>\n",
       "      <td>0.273367</td>\n",
       "      <td>0.190322</td>\n",
       "      <td>0.032943</td>\n",
       "      <td>-0.149621</td>\n",
       "      <td>0.097822</td>\n",
       "      <td>-0.119656</td>\n",
       "      <td>0.101924</td>\n",
       "      <td>0.198588</td>\n",
       "      <td>-0.158726</td>\n",
       "      <td>0.159052</td>\n",
       "      <td>0.113750</td>\n",
       "      <td>-0.158031</td>\n",
       "      <td>0.269808</td>\n",
       "      <td>-0.020800</td>\n",
       "      <td>0.012123</td>\n",
       "      <td>0.047193</td>\n",
       "      <td>0.227319</td>\n",
       "      <td>0.013312</td>\n",
       "      <td>-0.267428</td>\n",
       "      <td>0.066360</td>\n",
       "      <td>0.320278</td>\n",
       "      <td>-0.023354</td>\n",
       "      <td>-0.514239</td>\n",
       "      <td>0.228328</td>\n",
       "      <td>0.292068</td>\n",
       "      <td>0.109983</td>\n",
       "      <td>0.072455</td>\n",
       "      <td>-0.001843</td>\n",
       "      <td>-0.089084</td>\n",
       "      <td>0.361358</td>\n",
       "      <td>0.171718</td>\n",
       "      <td>0.038441</td>\n",
       "      <td>-0.574028</td>\n",
       "      <td>-0.042479</td>\n",
       "      <td>0.027775</td>\n",
       "      <td>0.003253</td>\n",
       "      <td>-0.101972</td>\n",
       "      <td>0.103554</td>\n",
       "      <td>-0.439865</td>\n",
       "      <td>0.107998</td>\n",
       "      <td>0.138701</td>\n",
       "      <td>-0.236071</td>\n",
       "      <td>-0.147181</td>\n",
       "      <td>-0.083717</td>\n",
       "      <td>-0.198127</td>\n",
       "      <td>-0.157046</td>\n",
       "      <td>0.170630</td>\n",
       "      <td>-0.361139</td>\n",
       "      <td>-0.091939</td>\n",
       "      <td>0.160136</td>\n",
       "      <td>0.125527</td>\n",
       "      <td>0.263363</td>\n",
       "      <td>0.338415</td>\n",
       "      <td>-0.404516</td>\n",
       "      <td>0.117673</td>\n",
       "      <td>0.499794</td>\n",
       "      <td>0.387065</td>\n",
       "      <td>-0.062616</td>\n",
       "      <td>-0.006799</td>\n",
       "      <td>-0.106931</td>\n",
       "      <td>-0.180950</td>\n",
       "      <td>0.097889</td>\n",
       "      <td>0.019412</td>\n",
       "      <td>-0.092774</td>\n",
       "      <td>-0.104903</td>\n",
       "      <td>-0.104122</td>\n",
       "      <td>0.167840</td>\n",
       "      <td>0.250242</td>\n",
       "      <td>-0.074696</td>\n",
       "      <td>-0.541867</td>\n",
       "      <td>0.063428</td>\n",
       "      <td>-0.365987</td>\n",
       "      <td>-0.374295</td>\n",
       "      <td>0.042924</td>\n",
       "      <td>0.080642</td>\n",
       "      <td>-0.067237</td>\n",
       "      <td>-0.074299</td>\n",
       "      <td>-0.152272</td>\n",
       "      <td>0.088243</td>\n",
       "      <td>-0.449713</td>\n",
       "      <td>-0.005897</td>\n",
       "      <td>-0.332854</td>\n",
       "      <td>-0.401828</td>\n",
       "      <td>0.084737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.137525</td>\n",
       "      <td>0.265025</td>\n",
       "      <td>-0.010123</td>\n",
       "      <td>0.407960</td>\n",
       "      <td>-0.002231</td>\n",
       "      <td>0.017766</td>\n",
       "      <td>0.034262</td>\n",
       "      <td>-0.041863</td>\n",
       "      <td>0.038054</td>\n",
       "      <td>0.227417</td>\n",
       "      <td>0.051316</td>\n",
       "      <td>0.236605</td>\n",
       "      <td>0.482495</td>\n",
       "      <td>0.572863</td>\n",
       "      <td>-0.236792</td>\n",
       "      <td>0.023759</td>\n",
       "      <td>-0.301567</td>\n",
       "      <td>-0.119358</td>\n",
       "      <td>0.053671</td>\n",
       "      <td>0.260520</td>\n",
       "      <td>-0.101332</td>\n",
       "      <td>0.046685</td>\n",
       "      <td>0.147247</td>\n",
       "      <td>-0.146464</td>\n",
       "      <td>0.226873</td>\n",
       "      <td>0.238391</td>\n",
       "      <td>-0.332869</td>\n",
       "      <td>0.323694</td>\n",
       "      <td>0.209686</td>\n",
       "      <td>-0.173994</td>\n",
       "      <td>0.087302</td>\n",
       "      <td>-0.117985</td>\n",
       "      <td>0.112142</td>\n",
       "      <td>0.146261</td>\n",
       "      <td>0.414851</td>\n",
       "      <td>-0.071862</td>\n",
       "      <td>-0.006431</td>\n",
       "      <td>-0.225612</td>\n",
       "      <td>-0.289347</td>\n",
       "      <td>0.048542</td>\n",
       "      <td>0.198003</td>\n",
       "      <td>-0.126273</td>\n",
       "      <td>-0.273958</td>\n",
       "      <td>0.148164</td>\n",
       "      <td>0.377191</td>\n",
       "      <td>-0.131095</td>\n",
       "      <td>-0.500679</td>\n",
       "      <td>-0.194147</td>\n",
       "      <td>0.210539</td>\n",
       "      <td>-0.103456</td>\n",
       "      <td>-0.074326</td>\n",
       "      <td>-0.274666</td>\n",
       "      <td>0.255624</td>\n",
       "      <td>-0.269893</td>\n",
       "      <td>-0.066593</td>\n",
       "      <td>-0.059447</td>\n",
       "      <td>0.350845</td>\n",
       "      <td>0.024596</td>\n",
       "      <td>-0.328549</td>\n",
       "      <td>0.117672</td>\n",
       "      <td>-0.323744</td>\n",
       "      <td>-0.052897</td>\n",
       "      <td>0.043723</td>\n",
       "      <td>0.088231</td>\n",
       "      <td>0.086536</td>\n",
       "      <td>0.076209</td>\n",
       "      <td>0.017663</td>\n",
       "      <td>-0.033744</td>\n",
       "      <td>-0.072464</td>\n",
       "      <td>-0.117434</td>\n",
       "      <td>0.319621</td>\n",
       "      <td>0.174219</td>\n",
       "      <td>0.200891</td>\n",
       "      <td>0.483039</td>\n",
       "      <td>0.131650</td>\n",
       "      <td>-0.053119</td>\n",
       "      <td>0.151730</td>\n",
       "      <td>-0.206849</td>\n",
       "      <td>-0.073102</td>\n",
       "      <td>0.159188</td>\n",
       "      <td>-0.000124</td>\n",
       "      <td>0.230478</td>\n",
       "      <td>0.299481</td>\n",
       "      <td>-0.321978</td>\n",
       "      <td>-0.113721</td>\n",
       "      <td>-0.273351</td>\n",
       "      <td>-0.057563</td>\n",
       "      <td>0.163494</td>\n",
       "      <td>0.025687</td>\n",
       "      <td>-0.194245</td>\n",
       "      <td>-0.078695</td>\n",
       "      <td>-0.113460</td>\n",
       "      <td>-0.400821</td>\n",
       "      <td>-0.138526</td>\n",
       "      <td>-0.270769</td>\n",
       "      <td>-0.235079</td>\n",
       "      <td>0.165924</td>\n",
       "      <td>0.054616</td>\n",
       "      <td>0.059850</td>\n",
       "      <td>0.018769</td>\n",
       "      <td>-0.122229</td>\n",
       "      <td>-0.015627</td>\n",
       "      <td>0.184568</td>\n",
       "      <td>0.245649</td>\n",
       "      <td>-0.183787</td>\n",
       "      <td>...</td>\n",
       "      <td>0.092792</td>\n",
       "      <td>0.097946</td>\n",
       "      <td>-0.038889</td>\n",
       "      <td>-0.039618</td>\n",
       "      <td>0.280452</td>\n",
       "      <td>-0.132929</td>\n",
       "      <td>0.143625</td>\n",
       "      <td>0.038979</td>\n",
       "      <td>0.037168</td>\n",
       "      <td>-0.363583</td>\n",
       "      <td>0.308713</td>\n",
       "      <td>0.042017</td>\n",
       "      <td>0.307992</td>\n",
       "      <td>-0.325368</td>\n",
       "      <td>-0.254257</td>\n",
       "      <td>-0.111948</td>\n",
       "      <td>0.026055</td>\n",
       "      <td>-0.164983</td>\n",
       "      <td>-0.260877</td>\n",
       "      <td>0.108569</td>\n",
       "      <td>-0.452726</td>\n",
       "      <td>0.237875</td>\n",
       "      <td>0.161968</td>\n",
       "      <td>-0.092039</td>\n",
       "      <td>0.181199</td>\n",
       "      <td>0.112137</td>\n",
       "      <td>-0.039931</td>\n",
       "      <td>0.002873</td>\n",
       "      <td>0.318678</td>\n",
       "      <td>-0.083134</td>\n",
       "      <td>0.093927</td>\n",
       "      <td>-0.057268</td>\n",
       "      <td>0.006241</td>\n",
       "      <td>0.388679</td>\n",
       "      <td>-0.371659</td>\n",
       "      <td>0.035060</td>\n",
       "      <td>-0.178270</td>\n",
       "      <td>0.105218</td>\n",
       "      <td>-0.184556</td>\n",
       "      <td>0.161999</td>\n",
       "      <td>0.172466</td>\n",
       "      <td>0.317760</td>\n",
       "      <td>-0.047731</td>\n",
       "      <td>-0.385257</td>\n",
       "      <td>0.129601</td>\n",
       "      <td>0.266923</td>\n",
       "      <td>-0.046403</td>\n",
       "      <td>0.292638</td>\n",
       "      <td>-0.080534</td>\n",
       "      <td>0.247024</td>\n",
       "      <td>0.574939</td>\n",
       "      <td>-0.089122</td>\n",
       "      <td>-0.027367</td>\n",
       "      <td>-0.344002</td>\n",
       "      <td>-0.015118</td>\n",
       "      <td>0.071419</td>\n",
       "      <td>0.230483</td>\n",
       "      <td>-0.172330</td>\n",
       "      <td>-0.065445</td>\n",
       "      <td>-0.251528</td>\n",
       "      <td>-0.018486</td>\n",
       "      <td>0.147374</td>\n",
       "      <td>0.036158</td>\n",
       "      <td>-0.306249</td>\n",
       "      <td>0.034631</td>\n",
       "      <td>0.169968</td>\n",
       "      <td>-0.115037</td>\n",
       "      <td>-0.008717</td>\n",
       "      <td>-0.322422</td>\n",
       "      <td>0.329700</td>\n",
       "      <td>0.164823</td>\n",
       "      <td>0.152219</td>\n",
       "      <td>-0.002751</td>\n",
       "      <td>0.218693</td>\n",
       "      <td>-0.033536</td>\n",
       "      <td>0.090824</td>\n",
       "      <td>0.218572</td>\n",
       "      <td>0.335920</td>\n",
       "      <td>0.058331</td>\n",
       "      <td>0.111005</td>\n",
       "      <td>0.151942</td>\n",
       "      <td>-0.218390</td>\n",
       "      <td>-0.239677</td>\n",
       "      <td>0.230013</td>\n",
       "      <td>-0.029526</td>\n",
       "      <td>-0.029864</td>\n",
       "      <td>0.108796</td>\n",
       "      <td>0.082263</td>\n",
       "      <td>0.230274</td>\n",
       "      <td>-0.396407</td>\n",
       "      <td>-0.154371</td>\n",
       "      <td>0.200644</td>\n",
       "      <td>-0.391888</td>\n",
       "      <td>-0.319362</td>\n",
       "      <td>-0.082632</td>\n",
       "      <td>0.050960</td>\n",
       "      <td>0.341008</td>\n",
       "      <td>0.046780</td>\n",
       "      <td>-0.221067</td>\n",
       "      <td>-0.215818</td>\n",
       "      <td>-0.455212</td>\n",
       "      <td>0.067931</td>\n",
       "      <td>-0.197712</td>\n",
       "      <td>-0.222433</td>\n",
       "      <td>-0.051372</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 384 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        0         1         2         3         4         5         6    \\\n",
       "0 -0.162271  0.359961 -0.208631  0.212792 -0.184559  0.023665  0.031174   \n",
       "1 -0.164998  0.200312 -0.291708  0.385741 -0.113083  0.090308 -0.015352   \n",
       "2  0.087443  0.348354  0.075141  0.265771 -0.317597 -0.080730  0.031378   \n",
       "3 -0.176260  0.331500 -0.217019  0.218216 -0.184135  0.013105  0.025132   \n",
       "4 -0.137525  0.265025 -0.010123  0.407960 -0.002231  0.017766  0.034262   \n",
       "\n",
       "        7         8         9         10        11        12        13   \\\n",
       "0 -0.056128  0.070213  0.246463  0.404678  0.222561  0.246697  0.569248   \n",
       "1  0.234842  0.140149  0.200789  0.300456  0.268847  0.327487  0.552415   \n",
       "2 -0.159422  0.076782  0.450581  0.020486  0.412668  0.452576  0.266731   \n",
       "3 -0.038927  0.075716  0.217835  0.394953  0.209932  0.223927  0.594058   \n",
       "4 -0.041863  0.038054  0.227417  0.051316  0.236605  0.482495  0.572863   \n",
       "\n",
       "        14        15        16        17        18        19        20   \\\n",
       "0 -0.286271  0.163123 -0.211469 -0.137589  0.333122  0.270672 -0.171357   \n",
       "1 -0.058277  0.069845 -0.283775  0.059433  0.132053  0.206883  0.020623   \n",
       "2 -0.088604  0.035567 -0.181696 -0.197046  0.319921  0.329228 -0.230394   \n",
       "3 -0.284427  0.171727 -0.221581 -0.123168  0.306397  0.265301 -0.183306   \n",
       "4 -0.236792  0.023759 -0.301567 -0.119358  0.053671  0.260520 -0.101332   \n",
       "\n",
       "        21        22        23        24        25        26        27   \\\n",
       "0 -0.007442  0.026922  0.093674  0.563860  0.190829 -0.260796  0.335964   \n",
       "1 -0.173855  0.225282 -0.017904  0.497691  0.216123 -0.304758  0.297272   \n",
       "2 -0.054677  0.264726  0.104968  0.342561  0.441863 -0.341266  0.059278   \n",
       "3  0.007575  0.060609  0.096721  0.556450  0.177043 -0.261085  0.320190   \n",
       "4  0.046685  0.147247 -0.146464  0.226873  0.238391 -0.332869  0.323694   \n",
       "\n",
       "        28        29        30        31        32        33        34   \\\n",
       "0  0.166725 -0.452981 -0.144069  0.137669  0.074681 -0.076224  0.409972   \n",
       "1  0.063490 -0.155111  0.192790  0.090803  0.076976  0.226654  0.310238   \n",
       "2  0.336389 -0.076413  0.208437  0.191560  0.204068  0.083635  0.235593   \n",
       "3  0.155211 -0.436180 -0.188859  0.121432  0.058139 -0.102365  0.389624   \n",
       "4  0.209686 -0.173994  0.087302 -0.117985  0.112142  0.146261  0.414851   \n",
       "\n",
       "        35        36        37        38        39        40        41   \\\n",
       "0 -0.091544 -0.143137 -0.533438 -0.177174  0.303646 -0.170809  0.113998   \n",
       "1 -0.049298  0.095382 -0.430092 -0.348674 -0.033861  0.297530  0.052408   \n",
       "2 -0.106136  0.289786 -0.252061 -0.266143  0.307279 -0.119827 -0.172364   \n",
       "3 -0.113831 -0.169072 -0.526837 -0.186467  0.306183 -0.118291  0.102877   \n",
       "4 -0.071862 -0.006431 -0.225612 -0.289347  0.048542  0.198003 -0.126273   \n",
       "\n",
       "        42        43        44        45        46        47        48   \\\n",
       "0 -0.180275 -0.298499  0.129155 -0.358678 -0.197952 -0.127823  0.095293   \n",
       "1 -0.273249 -0.018303  0.000791 -0.184625 -0.371987 -0.196453  0.265759   \n",
       "2 -0.145971 -0.299094  0.357934 -0.059064 -0.443192 -0.211800  0.157538   \n",
       "3 -0.162771 -0.291369  0.117419 -0.337459 -0.190894 -0.094893  0.122876   \n",
       "4 -0.273958  0.148164  0.377191 -0.131095 -0.500679 -0.194147  0.210539   \n",
       "\n",
       "        49        50        51        52        53        54        55   \\\n",
       "0  0.146926  0.200765 -0.178596  0.116211 -0.196614  0.198353  0.067136   \n",
       "1 -0.089040  0.163914 -0.434184  0.054147 -0.180023  0.367596  0.134108   \n",
       "2  0.082950 -0.139568 -0.335272  0.136966 -0.431957  0.064600 -0.038943   \n",
       "3  0.110975  0.203009 -0.161310  0.103420 -0.196771  0.209090  0.066207   \n",
       "4 -0.103456 -0.074326 -0.274666  0.255624 -0.269893 -0.066593 -0.059447   \n",
       "\n",
       "        56        57        58        59        60        61        62   \\\n",
       "0  0.164220  0.255172 -0.430976 -0.052471  0.122035 -0.120684  0.194026   \n",
       "1  0.269924  0.013862 -0.086283  0.153577 -0.441175 -0.102963  0.169320   \n",
       "2  0.230240  0.004617 -0.389729 -0.042800 -0.323701  0.019915  0.090443   \n",
       "3  0.156469  0.282545 -0.436368 -0.041194  0.152029 -0.163704  0.208622   \n",
       "4  0.350845  0.024596 -0.328549  0.117672 -0.323744 -0.052897  0.043723   \n",
       "\n",
       "        63        64        65        66        67        68        69   \\\n",
       "0  0.059159 -0.242641  0.216599 -0.209428 -0.450314 -0.067776  0.322422   \n",
       "1  0.168424 -0.019549  0.066295  0.027417 -0.022250  0.115971 -0.026206   \n",
       "2 -0.200131 -0.156367 -0.051447 -0.080336 -0.053585  0.072576  0.029933   \n",
       "3  0.081991 -0.222724  0.200585 -0.189137 -0.445165 -0.061106  0.329794   \n",
       "4  0.088231  0.086536  0.076209  0.017663 -0.033744 -0.072464 -0.117434   \n",
       "\n",
       "        70        71        72        73        74        75        76   \\\n",
       "0  0.050678 -0.037782  0.081106  0.362837 -0.256037 -0.186787 -0.048138   \n",
       "1  0.111903  0.134881  0.163826  0.415292 -0.045816  0.029169 -0.002534   \n",
       "2  0.189949  0.046997  0.251135  0.505619 -0.113831 -0.042439  0.008957   \n",
       "3  0.043275 -0.031708  0.044975  0.330188 -0.257523 -0.191809 -0.026744   \n",
       "4  0.319621  0.174219  0.200891  0.483039  0.131650 -0.053119  0.151730   \n",
       "\n",
       "        77        78        79        80        81        82        83   \\\n",
       "0 -0.397506 -0.240361 -0.247374  0.147705  0.132116  0.320903 -0.256691   \n",
       "1 -0.437910 -0.069225  0.127752 -0.087582  0.209030 -0.017341 -0.325876   \n",
       "2 -0.528493 -0.237793 -0.154873  0.127994  0.068931  0.257621 -0.324349   \n",
       "3 -0.394451 -0.216947 -0.258086  0.136926  0.127646  0.341987 -0.242775   \n",
       "4 -0.206849 -0.073102  0.159188 -0.000124  0.230478  0.299481 -0.321978   \n",
       "\n",
       "        84        85        86        87        88        89        90   \\\n",
       "0 -0.021490 -0.365033 -0.155543  0.344853  0.302176 -0.139125  0.172635   \n",
       "1  0.031164 -0.381508 -0.209271  0.060853 -0.095384 -0.291828 -0.112984   \n",
       "2 -0.068819 -0.075986 -0.260203  0.257970 -0.067435 -0.132135  0.114913   \n",
       "3 -0.020420 -0.342146 -0.134044  0.349911  0.297647 -0.128103  0.177103   \n",
       "4 -0.113721 -0.273351 -0.057563  0.163494  0.025687 -0.194245 -0.078695   \n",
       "\n",
       "        91        92        93        94        95        96        97   \\\n",
       "0 -0.128772  0.321327  0.151302 -0.063909 -0.135832  0.332110 -0.003239   \n",
       "1 -0.085400 -0.271536 -0.287510 -0.201039 -0.028334  0.257096  0.126726   \n",
       "2 -0.042286 -0.296356 -0.137618  0.058961 -0.165032  0.498271 -0.165034   \n",
       "3 -0.146584  0.326321  0.154350 -0.100294 -0.146707  0.318168  0.012583   \n",
       "4 -0.113460 -0.400821 -0.138526 -0.270769 -0.235079  0.165924  0.054616   \n",
       "\n",
       "        98        99        100       101       102       103       104  ...  \\\n",
       "0  0.258752  0.012680 -0.174956  0.260750  0.035437  0.454084  0.084206  ...   \n",
       "1 -0.001215  0.055023  0.019923  0.145393  0.170775  0.345924 -0.001274  ...   \n",
       "2  0.341459  0.013973 -0.128801  0.129680 -0.112283  0.303148 -0.040948  ...   \n",
       "3  0.245729 -0.015691 -0.147501  0.265394  0.018757  0.452250  0.058254  ...   \n",
       "4  0.059850  0.018769 -0.122229 -0.015627  0.184568  0.245649 -0.183787  ...   \n",
       "\n",
       "        279       280       281       282       283       284       285  \\\n",
       "0  0.325775  0.087837 -0.047203 -0.375325  0.240884  0.096204  0.210107   \n",
       "1  0.059508  0.037455 -0.226644 -0.200301  0.135569 -0.035710  0.130619   \n",
       "2  0.332492  0.025870 -0.151847 -0.217514  0.112499 -0.370836  0.152899   \n",
       "3  0.286873  0.075731 -0.060459 -0.369617  0.230923  0.091402  0.222454   \n",
       "4  0.092792  0.097946 -0.038889 -0.039618  0.280452 -0.132929  0.143625   \n",
       "\n",
       "        286       287       288       289       290       291       292  \\\n",
       "0  0.152762 -0.285373 -0.281662  0.243171  0.335456  0.193422 -0.344888   \n",
       "1  0.011335 -0.017847 -0.136202  0.259121 -0.040067  0.328404 -0.241552   \n",
       "2  0.015132 -0.226122 -0.105108  0.320797  0.272942  0.320707 -0.060986   \n",
       "3  0.119655 -0.288558 -0.329720  0.243055  0.333877  0.196843 -0.336662   \n",
       "4  0.038979  0.037168 -0.363583  0.308713  0.042017  0.307992 -0.325368   \n",
       "\n",
       "        293       294       295       296       297       298       299  \\\n",
       "0 -0.180593  0.024781 -0.214996 -0.305409 -0.109843 -0.137799 -0.499643   \n",
       "1  0.123093 -0.204784 -0.206881 -0.301866 -0.098708 -0.209272 -0.543192   \n",
       "2 -0.301983 -0.103061 -0.183105 -0.022568 -0.245966 -0.115399 -0.429196   \n",
       "3 -0.154158  0.026134 -0.223284 -0.306917 -0.098450 -0.144181 -0.496922   \n",
       "4 -0.254257 -0.111948  0.026055 -0.164983 -0.260877  0.108569 -0.452726   \n",
       "\n",
       "        300       301       302       303       304       305       306  \\\n",
       "0  0.295993  0.223503  0.012261 -0.147910  0.103415 -0.091400  0.090393   \n",
       "1  0.190023  0.235180 -0.119917  0.086322  0.212423 -0.129963  0.174801   \n",
       "2  0.369226  0.339843  0.007097  0.079089 -0.031242 -0.007945 -0.031585   \n",
       "3  0.273367  0.190322  0.032943 -0.149621  0.097822 -0.119656  0.101924   \n",
       "4  0.237875  0.161968 -0.092039  0.181199  0.112137 -0.039931  0.002873   \n",
       "\n",
       "        307       308       309       310       311       312       313  \\\n",
       "0  0.178400 -0.128588  0.159457  0.131086 -0.192747  0.246661 -0.018331   \n",
       "1  0.025072 -0.205548  0.252248  0.091975  0.292690  0.279312  0.064004   \n",
       "2  0.197342 -0.037872  0.163866 -0.137193 -0.108474  0.155755 -0.125552   \n",
       "3  0.198588 -0.158726  0.159052  0.113750 -0.158031  0.269808 -0.020800   \n",
       "4  0.318678 -0.083134  0.093927 -0.057268  0.006241  0.388679 -0.371659   \n",
       "\n",
       "        314       315       316       317       318       319       320  \\\n",
       "0 -0.004980  0.066220  0.229715  0.022504 -0.275646  0.057845  0.330876   \n",
       "1  0.029540 -0.140886  0.274939 -0.214205  0.232435 -0.021775  0.326521   \n",
       "2 -0.024259 -0.114572  0.037851 -0.132192 -0.036920  0.108299  0.408869   \n",
       "3  0.012123  0.047193  0.227319  0.013312 -0.267428  0.066360  0.320278   \n",
       "4  0.035060 -0.178270  0.105218 -0.184556  0.161999  0.172466  0.317760   \n",
       "\n",
       "        321       322       323       324       325       326       327  \\\n",
       "0  0.000617 -0.515452  0.235576  0.299304  0.124795  0.086530  0.011721   \n",
       "1 -0.173589 -0.327937  0.211141  0.306258 -0.026692  0.191214 -0.011201   \n",
       "2  0.094320 -0.339686  0.086932  0.372600  0.080849  0.171356 -0.155160   \n",
       "3 -0.023354 -0.514239  0.228328  0.292068  0.109983  0.072455 -0.001843   \n",
       "4 -0.047731 -0.385257  0.129601  0.266923 -0.046403  0.292638 -0.080534   \n",
       "\n",
       "        328       329       330       331       332       333       334  \\\n",
       "0 -0.124608  0.353909  0.176914  0.021789 -0.560504 -0.019877  0.027930   \n",
       "1 -0.158324  0.485520  0.042530  0.131386 -0.392145 -0.045617  0.089598   \n",
       "2 -0.173940  0.399333  0.123328 -0.080917 -0.506595 -0.073971  0.098760   \n",
       "3 -0.089084  0.361358  0.171718  0.038441 -0.574028 -0.042479  0.027775   \n",
       "4  0.247024  0.574939 -0.089122 -0.027367 -0.344002 -0.015118  0.071419   \n",
       "\n",
       "        335       336       337       338       339       340       341  \\\n",
       "0 -0.026669 -0.124045  0.082781 -0.416438  0.100424  0.171194 -0.240402   \n",
       "1  0.010491 -0.237086  0.047300 -0.110376 -0.070836 -0.116565  0.031171   \n",
       "2  0.042386 -0.131056  0.021697 -0.016402 -0.091808  0.026021 -0.084562   \n",
       "3  0.003253 -0.101972  0.103554 -0.439865  0.107998  0.138701 -0.236071   \n",
       "4  0.230483 -0.172330 -0.065445 -0.251528 -0.018486  0.147374  0.036158   \n",
       "\n",
       "        342       343       344       345       346       347       348  \\\n",
       "0 -0.120383 -0.101247 -0.219663 -0.180439  0.171812 -0.378690 -0.122113   \n",
       "1 -0.102659  0.007115 -0.085476 -0.344428 -0.038877 -0.376358  0.223386   \n",
       "2  0.027770 -0.086904 -0.062518 -0.211319  0.055575 -0.173359 -0.016561   \n",
       "3 -0.147181 -0.083717 -0.198127 -0.157046  0.170630 -0.361139 -0.091939   \n",
       "4 -0.306249  0.034631  0.169968 -0.115037 -0.008717 -0.322422  0.329700   \n",
       "\n",
       "        349       350       351       352       353       354       355  \\\n",
       "0  0.160866  0.131128  0.284670  0.330584 -0.425320  0.112695  0.506408   \n",
       "1  0.124038  0.318288  0.010998  0.290550 -0.214707  0.292548  0.314998   \n",
       "2 -0.078423  0.076999  0.092031  0.185906 -0.532436  0.376591  0.106628   \n",
       "3  0.160136  0.125527  0.263363  0.338415 -0.404516  0.117673  0.499794   \n",
       "4  0.164823  0.152219 -0.002751  0.218693 -0.033536  0.090824  0.218572   \n",
       "\n",
       "        356       357       358       359       360       361       362  \\\n",
       "0  0.361750 -0.046845  0.007392 -0.070656 -0.199145  0.104572 -0.008363   \n",
       "1  0.234466  0.075865  0.162426  0.147017 -0.085037 -0.049484 -0.181991   \n",
       "2  0.114349  0.102644  0.191851  0.037620 -0.343103 -0.184611 -0.083883   \n",
       "3  0.387065 -0.062616 -0.006799 -0.106931 -0.180950  0.097889  0.019412   \n",
       "4  0.335920  0.058331  0.111005  0.151942 -0.218390 -0.239677  0.230013   \n",
       "\n",
       "        363       364       365       366       367       368       369  \\\n",
       "0 -0.072674 -0.080588 -0.084459  0.144243  0.227459 -0.067456 -0.541151   \n",
       "1  0.153568 -0.151253  0.183836  0.133308  0.088930 -0.268663 -0.258682   \n",
       "2  0.058932  0.040169 -0.105352  0.038794 -0.052485  0.007382 -0.301429   \n",
       "3 -0.092774 -0.104903 -0.104122  0.167840  0.250242 -0.074696 -0.541867   \n",
       "4 -0.029526 -0.029864  0.108796  0.082263  0.230274 -0.396407 -0.154371   \n",
       "\n",
       "        370       371       372       373       374       375       376  \\\n",
       "0  0.081943 -0.373041 -0.345800  0.036262  0.095997 -0.096873 -0.063860   \n",
       "1  0.156034 -0.331153 -0.366319 -0.134505 -0.029372  0.222066  0.003363   \n",
       "2  0.043780 -0.478187 -0.103193 -0.006233  0.230008  0.009344 -0.184263   \n",
       "3  0.063428 -0.365987 -0.374295  0.042924  0.080642 -0.067237 -0.074299   \n",
       "4  0.200644 -0.391888 -0.319362 -0.082632  0.050960  0.341008  0.046780   \n",
       "\n",
       "        377       378       379       380       381       382       383  \n",
       "0 -0.114892  0.092941 -0.445563 -0.034386 -0.321318 -0.407256  0.045911  \n",
       "1 -0.056301 -0.256319 -0.349865  0.015541 -0.251664 -0.024831 -0.041127  \n",
       "2  0.031506  0.062032 -0.151290  0.162426 -0.353901 -0.378658 -0.271250  \n",
       "3 -0.152272  0.088243 -0.449713 -0.005897 -0.332854 -0.401828  0.084737  \n",
       "4 -0.221067 -0.215818 -0.455212  0.067931 -0.197712 -0.222433 -0.051372  \n",
       "\n",
       "[5 rows x 384 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load pre-trained ChemBERTa model checkpoint and tokenizer\n",
    "cb_tokenizer = AutoTokenizer.from_pretrained('DeepChem/ChemBERTa-10M-MLM')\n",
    "cb_model = AutoModel.from_pretrained('DeepChem/ChemBERTa-10M-MLM')\n",
    "cb_model.eval()\n",
    "\n",
    "# tokenize SMILES\n",
    "cb_encoded_inputs = cb_tokenizer(list(smiles), padding=True, truncation=True, return_tensors=\"pt\")\n",
    "\n",
    "# calculate embeddings\n",
    "with torch.no_grad():\n",
    "    outputs = cb_model(**cb_encoded_inputs)\n",
    "\n",
    "# extract pooled output\n",
    "cb_embeddings = outputs.pooler_output\n",
    "\n",
    "cb_embeddings_df = pd.DataFrame(cb_embeddings.numpy())\n",
    "cb_embeddings_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c770107-d8e8-4fda-b166-ce95b7e21578",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "9ebc553f-ae00-4289-ba6b-cf67ef67e61c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1249, 384)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = cb_embeddings_df\n",
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "de43362f-0b96-41b5-b37f-9b47be6c6374",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1249,)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df[\"yield\"]\n",
    "y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "c29a1385-60f1-493b-bffe-b271032e1527",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sauvegarde du DataFrame dans un fichier CSV\n",
    "X.to_csv(\"cb_embeddings_df.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d479abe4-96a6-47c9-bd3a-54a2f1f75150",
   "metadata": {},
   "outputs": [],
   "source": [
    "def filter_descriptors_low(data, threshold):\n",
    "    # Drop colonne inutile\n",
    "    data = data.drop([\"SMILES\", \"Ind\", \"SMILES_2\", \"mols\"], axis=1)\n",
    "    # Copie des descripteurs\n",
    "    descriptors = data.copy()\n",
    "    \n",
    "    # Calcul de la matrice de corrélation et mise à zéro de la diagonale\n",
    "    corr = descriptors.corr()\n",
    "    for index in corr.index:\n",
    "        corr.loc[index, index] = 0\n",
    "    \n",
    "    # Préparation d'une liste pour collecter les noms des descripteurs non fortement corrélés\n",
    "    descriptors_not_correlated = []\n",
    "    \n",
    "    # Vérifier chaque colonne individuellement\n",
    "    for col in corr.columns:\n",
    "        if all((corr[col][corr[col].notna()] <= threshold)):\n",
    "            descriptors_not_correlated.append(col)\n",
    "    \n",
    "    print('Number of descriptors:', len(descriptors_not_correlated))\n",
    "    \n",
    "    # Retourner les données filtrées\n",
    "    return data[descriptors_not_correlated]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b3e8869b-a93c-4741-8669-04c64df29b79",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cafb74ea-3eb3-471e-9c73-756f6dc8d930",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_3_desc = filter_descriptors_low(df_2, 0.9)\n",
    "df_3_desc.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "607bd0f5-a751-4b90-9050-7e0012c1d97c",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_3_desc.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32065c85-1601-487d-86b8-282a897e2945",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "def filter_descriptors_hard(data, threshold):\n",
    "    # Drop colonne inutile\n",
    "    data = data.drop([\"SMILES\", \"Ind\", \"SMILES_2\", \"mols\"], axis=1)\n",
    "    # Copie des descripteurs\n",
    "    descriptors = data.copy()\n",
    "    \n",
    "    # Calcul de la matrice de corrélation et mise à zéro de la diagonale\n",
    "    corr = descriptors.corr()\n",
    "    for index in corr.index:\n",
    "        corr.loc[index, index] = 0\n",
    "    \n",
    "    # Identifie les groupes de descripteurs fortement corrélés\n",
    "    corr_mask = corr.abs() > threshold\n",
    "    groups = []\n",
    "    for col in corr.columns:\n",
    "        correlated_with = corr.index[corr_mask[col]].tolist()\n",
    "        if correlated_with and col not in sum(groups, []):\n",
    "            groups.append(correlated_with)\n",
    "    \n",
    "    # Choix d'un descripteur par groupe\n",
    "    selected_descriptors = []\n",
    "    for group in groups:\n",
    "        # Choix basé sur le minimum de corrélations maximales avec les autres\n",
    "        min_correlation_sum = np.inf\n",
    "        selected = None\n",
    "        for item in group:\n",
    "            # Assurez-vous de ne considérer que les valeurs non-NaN pour la somme des corrélations\n",
    "            corr_sum = corr[item][corr_mask[item] & corr[item].notna()].sum()\n",
    "            if corr_sum < min_correlation_sum:\n",
    "                min_correlation_sum = corr_sum\n",
    "                selected = item\n",
    "        selected_descriptors.append(selected)\n",
    "    \n",
    "    print('Descripteurs sélectionnés:', selected_descriptors)\n",
    "        \n",
    "    # Retourner les données filtrées\n",
    "    return data[selected_descriptors], data['yield']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6a59e5e-0650-4478-9fe3-0008bdb304d1",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df = calculate_descriptors(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "93bb56ab-8a47-44be-94ab-4601df486eec",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7296712-502d-4e88-a6ef-a392858c8962",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "rxnfp",
   "language": "python",
   "name": "rxnfp"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}