Skip to content

Commit

Permalink
removed notebook output
Browse files Browse the repository at this point in the history
  • Loading branch information
@loekadam
loekadam committed Sep 13, 2024
1 parent 4c834ac commit 2bf5bb1
Showing 1 changed file with 15 additions and 233 deletions.
248 changes: 15 additions & 233 deletions notebooks/nb_example_carbon_tracking.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,35 +24,9 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[codecarbon INFO @ 13:23:02] [setup] RAM Tracking...\n",
"[codecarbon INFO @ 13:23:02] [setup] GPU Tracking...\n",
"[codecarbon INFO @ 13:23:04] No GPU found.\n",
"[codecarbon INFO @ 13:23:04] [setup] CPU Tracking...\n",
"[codecarbon WARNING @ 13:23:04] No CPU tracking mode found. Falling back on CPU constant mode. \n",
" Linux OS detected: Please ensure RAPL files exist at \\sys\\class\\powercap\\intel-rapl to measure CPU\n",
"\n",
"[codecarbon INFO @ 13:23:05] CPU Model on constant consumption mode: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 13:23:05] >>> Tracker's metadata:\n",
"[codecarbon INFO @ 13:23:05] Platform system: Linux-5.15.0-1064-azure-x86_64-with-glibc2.31\n",
"[codecarbon INFO @ 13:23:05] Python version: 3.10.14\n",
"[codecarbon INFO @ 13:23:05] CodeCarbon version: 2.7.1\n",
"[codecarbon INFO @ 13:23:05] Available RAM : 13.648 GB\n",
"[codecarbon INFO @ 13:23:05] CPU count: 2\n",
"[codecarbon INFO @ 13:23:05] CPU model: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 13:23:05] GPU count: None\n",
"[codecarbon INFO @ 13:23:05] GPU model: None\n",
"[codecarbon WARNING @ 13:23:05] Cloud provider 'azure' do not publish electricity carbon intensity. Using country value instead.\n",
"[codecarbon INFO @ 13:23:05] Saving emissions data to file /mnt/batch/tasks/shared/LS_root/mounts/clusters/loektest310/code/emissions.csv\n"
]
}
],
"outputs": [],
"source": [
"# Codeblock where we import the EmissionsTracker and start the tracker.\n",
"# This should be placed at the beginning of the notebook if we want to track the whole notebook.\n",
Expand All @@ -65,27 +39,7 @@
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[codecarbon INFO @ 13:24:50] Energy consumed for RAM : 0.000149 kWh. RAM Power : 5.1181769371032715 W\n",
"[codecarbon INFO @ 13:24:50] Energy consumed for all CPUs : 0.002403 kWh. Total CPU Power : 82.5 W\n",
"[codecarbon INFO @ 13:24:50] 0.002552 kWh of electricity used since the beginning.\n"
]
},
{
"data": {
"text/plain": [
"0.0006828843305487836"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"# Codeblock where we stop the tracker.\n",
"# This should be placed at the end of the notebook if we want to track the whole notebook.\n",
Expand All @@ -111,54 +65,7 @@
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n",
"[codecarbon INFO @ 13:25:00] [setup] RAM Tracking...\n",
"[codecarbon INFO @ 13:25:00] [setup] GPU Tracking...\n",
"[codecarbon INFO @ 13:25:02] No GPU found.\n",
"[codecarbon INFO @ 13:25:02] [setup] CPU Tracking...\n",
"[codecarbon WARNING @ 13:25:02] No CPU tracking mode found. Falling back on CPU constant mode. \n",
" Linux OS detected: Please ensure RAPL files exist at \\sys\\class\\powercap\\intel-rapl to measure CPU\n",
"\n",
"[codecarbon INFO @ 13:25:03] CPU Model on constant consumption mode: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 13:25:03] >>> Tracker's metadata:\n",
"[codecarbon INFO @ 13:25:03] Platform system: Linux-5.15.0-1064-azure-x86_64-with-glibc2.31\n",
"[codecarbon INFO @ 13:25:03] Python version: 3.10.14\n",
"[codecarbon INFO @ 13:25:03] CodeCarbon version: 2.7.1\n",
"[codecarbon INFO @ 13:25:03] Available RAM : 13.648 GB\n",
"[codecarbon INFO @ 13:25:03] CPU count: 2\n",
"[codecarbon INFO @ 13:25:03] CPU model: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 13:25:03] GPU count: None\n",
"[codecarbon INFO @ 13:25:03] GPU model: None\n",
"[codecarbon WARNING @ 13:25:03] Cloud provider 'azure' do not publish electricity carbon intensity. Using country value instead.\n",
"[codecarbon INFO @ 13:25:03] Saving emissions data to file /mnt/batch/tasks/shared/LS_root/mounts/clusters/loektest310/code/emissions.csv\n",
"[codecarbon INFO @ 13:25:03] Energy consumed for RAM : 0.000000 kWh. RAM Power : 5.1181769371032715 W\n",
"[codecarbon INFO @ 13:25:03] Energy consumed for all CPUs : 0.000000 kWh. Total CPU Power : 82.5 W\n",
"[codecarbon INFO @ 13:25:03] 0.000000 kWh of electricity used since the beginning.\n",
"[codecarbon INFO @ 13:25:03] Energy consumed for RAM : 0.000000 kWh. RAM Power : 5.1181769371032715 W\n",
"[codecarbon INFO @ 13:25:03] Energy consumed for all CPUs : 0.000000 kWh. Total CPU Power : 82.5 W\n",
"[codecarbon INFO @ 13:25:03] 0.000000 kWh of electricity used since the beginning.\n",
"/anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/codecarbon/output_methods/file.py:81: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.\n",
" df = pd.concat(\n"
]
},
{
"ename": "NameError",
"evalue": "name 'load_dataset' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[3], line 8\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[38;5;66;03m# We start a specific task using the tracker\u001b[39;00m\n\u001b[1;32m 7\u001b[0m tracker\u001b[38;5;241m.\u001b[39mstart_task(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mload dataset\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m----> 8\u001b[0m dataset \u001b[38;5;241m=\u001b[39m \u001b[43mload_dataset\u001b[49m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mimdb\u001b[39m\u001b[38;5;124m\"\u001b[39m, split\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtest\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m 9\u001b[0m \u001b[38;5;66;03m# We have to stop individual tasks\u001b[39;00m\n\u001b[1;32m 10\u001b[0m imdb_emissions \u001b[38;5;241m=\u001b[39m tracker\u001b[38;5;241m.\u001b[39mstop_task()\n",
"\u001b[0;31mNameError\u001b[0m: name 'load_dataset' is not defined"
]
}
],
"outputs": [],
"source": [
"from codecarbon import EmissionsTracker\n",
"import datasets\n",
Expand All @@ -167,11 +74,11 @@
" tracker = EmissionsTracker(project_name=\"small_pieces\", measure_power_secs=10)\n",
" # We start a specific task using the tracker\n",
" tracker.start_task(\"load dataset\")\n",
" ## Some code to load dataset ##\n",
" dataset = load_dataset(\"imdb\", split=\"test\")\n",
" # We have to stop individual tasks\n",
" imdb_emissions = tracker.stop_task()\n",
" tracker.start_task(\"build model\")\n",
" ## Some code to build the model ##\n",
" model = build_model()\n",
" model_emissions = tracker.stop_task()\n",
"finally:\n",
" # We \n",
Expand Down Expand Up @@ -205,7 +112,8 @@
"from codecarbon import EmissionsTracker\n",
"\n",
"with EmissionsTracker() as tracker:\n",
" # Compute intensive training code goes here"
" # Compute intensive training code goes here\n",
" print()"
]
},
{
Expand All @@ -218,15 +126,16 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"from codecarbon import track_emissions\n",
"\n",
"@track_emissions\n",
"def training_loop():\n",
" # Compute intensive training code goes here"
" # Compute intensive training code goes here\n",
" print()"
]
},
{
Expand All @@ -249,43 +158,7 @@
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[codecarbon INFO @ 12:59:13] [setup] RAM Tracking...\n",
"[codecarbon INFO @ 12:59:13] [setup] GPU Tracking...\n",
"[codecarbon INFO @ 12:59:15] No GPU found.\n",
"[codecarbon INFO @ 12:59:15] [setup] CPU Tracking...\n",
"[codecarbon WARNING @ 12:59:15] No CPU tracking mode found. Falling back on CPU constant mode. \n",
" Linux OS detected: Please ensure RAPL files exist at \\sys\\class\\powercap\\intel-rapl to measure CPU\n",
"\n",
"[codecarbon INFO @ 12:59:16] CPU Model on constant consumption mode: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 12:59:16] >>> Tracker's metadata:\n",
"[codecarbon INFO @ 12:59:16] Platform system: Linux-5.15.0-1064-azure-x86_64-with-glibc2.31\n",
"[codecarbon INFO @ 12:59:16] Python version: 3.10.14\n",
"[codecarbon INFO @ 12:59:16] CodeCarbon version: 2.7.1\n",
"[codecarbon INFO @ 12:59:16] Available RAM : 13.648 GB\n",
"[codecarbon INFO @ 12:59:16] CPU count: 2\n",
"[codecarbon INFO @ 12:59:16] CPU model: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 12:59:16] GPU count: None\n",
"[codecarbon INFO @ 12:59:16] GPU model: None\n",
"[codecarbon WARNING @ 12:59:16] Cloud provider 'azure' do not publish electricity carbon intensity. Using country value instead.\n",
"[codecarbon INFO @ 12:59:16] Saving emissions data to file /mnt/batch/tasks/shared/LS_root/mounts/clusters/loektest310/code/emissions.csv\n",
"[codecarbon INFO @ 12:59:21] Energy consumed for RAM : 0.000007 kWh. RAM Power : 5.1181769371032715 W\n",
"[codecarbon INFO @ 12:59:21] Energy consumed for all CPUs : 0.000115 kWh. Total CPU Power : 82.5 W\n",
"[codecarbon INFO @ 12:59:21] 0.000122 kWh of electricity used since the beginning.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Emissions during training: 0.00003267 kg CO2eq\n"
]
}
],
"outputs": [],
"source": [
"import numpy as np\n",
"from sklearn.model_selection import train_test_split\n",
Expand Down Expand Up @@ -332,36 +205,7 @@
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[codecarbon INFO @ 12:59:25] [setup] RAM Tracking...\n",
"[codecarbon INFO @ 12:59:25] [setup] GPU Tracking...\n",
"[codecarbon INFO @ 12:59:26] No GPU found.\n",
"[codecarbon INFO @ 12:59:26] [setup] CPU Tracking...\n",
"[codecarbon WARNING @ 12:59:26] No CPU tracking mode found. Falling back on CPU constant mode. \n",
" Linux OS detected: Please ensure RAPL files exist at \\sys\\class\\powercap\\intel-rapl to measure CPU\n",
"\n",
"[codecarbon INFO @ 12:59:28] CPU Model on constant consumption mode: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 12:59:28] >>> Tracker's metadata:\n",
"[codecarbon INFO @ 12:59:28] Platform system: Linux-5.15.0-1064-azure-x86_64-with-glibc2.31\n",
"[codecarbon INFO @ 12:59:28] Python version: 3.10.14\n",
"[codecarbon INFO @ 12:59:28] CodeCarbon version: 2.7.1\n",
"[codecarbon INFO @ 12:59:28] Available RAM : 13.648 GB\n",
"[codecarbon INFO @ 12:59:28] CPU count: 2\n",
"[codecarbon INFO @ 12:59:28] CPU model: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 12:59:28] GPU count: None\n",
"[codecarbon INFO @ 12:59:28] GPU model: None\n",
"[codecarbon WARNING @ 12:59:28] Cloud provider 'azure' do not publish electricity carbon intensity. Using country value instead.\n",
"[codecarbon INFO @ 12:59:28] Saving emissions data to file /mnt/batch/tasks/shared/LS_root/mounts/clusters/loektest310/code/emissions.csv\n",
"[codecarbon INFO @ 12:59:33] Energy consumed for RAM : 0.000007 kWh. RAM Power : 5.1181769371032715 W\n",
"[codecarbon INFO @ 12:59:33] Energy consumed for all CPUs : 0.000115 kWh. Total CPU Power : 82.5 W\n",
"[codecarbon INFO @ 12:59:33] 0.000122 kWh of electricity used since the beginning.\n"
]
}
],
"outputs": [],
"source": [
"import numpy as np\n",
"from sklearn.model_selection import train_test_split\n",
Expand Down Expand Up @@ -402,41 +246,7 @@
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"[codecarbon INFO @ 12:59:44] [setup] RAM Tracking...\n",
"[codecarbon INFO @ 12:59:44] [setup] GPU Tracking...\n",
"[codecarbon INFO @ 12:59:45] No GPU found.\n",
"[codecarbon INFO @ 12:59:45] [setup] CPU Tracking...\n",
"[codecarbon WARNING @ 12:59:45] No CPU tracking mode found. Falling back on CPU constant mode. \n",
" Linux OS detected: Please ensure RAPL files exist at \\sys\\class\\powercap\\intel-rapl to measure CPU\n",
"\n",
"[codecarbon INFO @ 12:59:46] CPU Model on constant consumption mode: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 12:59:46] >>> Tracker's metadata:\n",
"[codecarbon INFO @ 12:59:46] Platform system: Linux-5.15.0-1064-azure-x86_64-with-glibc2.31\n",
"[codecarbon INFO @ 12:59:46] Python version: 3.10.14\n",
"[codecarbon INFO @ 12:59:46] CodeCarbon version: 2.7.1\n",
"[codecarbon INFO @ 12:59:46] Available RAM : 13.648 GB\n",
"[codecarbon INFO @ 12:59:46] CPU count: 2\n",
"[codecarbon INFO @ 12:59:46] CPU model: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz\n",
"[codecarbon INFO @ 12:59:46] GPU count: None\n",
"[codecarbon INFO @ 12:59:46] GPU model: None\n",
"[codecarbon WARNING @ 12:59:46] Cloud provider 'azure' do not publish electricity carbon intensity. Using country value instead.\n",
"[codecarbon INFO @ 12:59:47] Saving emissions data to file /mnt/batch/tasks/shared/LS_root/mounts/clusters/loektest310/code/emissions.csv\n",
"[codecarbon INFO @ 12:59:52] \n",
"Graceful stopping: collecting and writing information.\n",
"Please wait a few seconds...\n",
"[codecarbon INFO @ 12:59:52] Energy consumed for RAM : 0.000007 kWh. RAM Power : 5.1181769371032715 W\n",
"[codecarbon INFO @ 12:59:52] Energy consumed for all CPUs : 0.000115 kWh. Total CPU Power : 82.5 W\n",
"[codecarbon INFO @ 12:59:52] 0.000122 kWh of electricity used since the beginning.\n",
"[codecarbon INFO @ 12:59:52] Done!\n",
"\n"
]
}
],
"outputs": [],
"source": [
"import numpy as np\n",
"from sklearn.model_selection import train_test_split\n",
Expand Down Expand Up @@ -471,45 +281,17 @@
"## Visualization\n",
"CodeCarbon comes with a `Dash App` where the emissions are visualized.\n",
"\n",
"To run it, we execute the CLI command below:\n",
"To host it locally it, we execute the CLI command below:\n",
"\n",
"`carbonboard --filepath=\"emissions.csv\" --port=3333`"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"import os"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.10 - SDK v2",
"language": "python",
"name": "python310-sdkv2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
}
},
"nbformat": 4,
Expand Down

0 comments on commit 2bf5bb1

Please sign in to comment.