environment_set_up/Help_AWS_Credentials.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AWS Credentials\n",
    "\n",
    "If a workflow needs to access S3, it will be important to establish AWS\n",
    "credentials before first access to the object storage. If you are working in a workflow using a dask cluster, you will also need to establish AWS credentials before the cluster starts up.\n",
    "\n",
    "The following code block will handle **general** configuration for establishing AWS credentials for the notebook and any spawned cluster workers. \n",
    "\n",
    "You will need to make sure your `AWS_PROFILE` or `AWS_S3_ENDPOINT` environmental variables are established before this is run. If they are not, the defaults will be set to the access the OSN storage pod (with a profile named `osn-renci` and an endpoint of `https://renc.osn.xsede.org`). These default settings will only work if you have this AWS profile cofigured."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import logging\n",
    "import configparser\n",
    "\n",
    "awsconfig = configparser.ConfigParser()\n",
    "awsconfig.read(\n",
    "    os.path.expanduser('~/.aws/credentials') # default location... if yours is elsewhere, change this.\n",
    ")\n",
    "## NOTE:  The default will be for the OSN / RENCI profile and endpoint. Override this\n",
    "## by setting environment variables before executing this cell/notebook.\n",
    "_profile_nm  = os.environ.get('AWS_PROFILE', 'osn-renci')\n",
    "_endpoint = os.environ.get('AWS_S3_ENDPOINT', 'https://renc.osn.xsede.org')\n",
    "# Set environment vars based on parsed awsconfig\n",
    "try:\n",
    "    os.environ['AWS_ACCESS_KEY_ID']     = awsconfig[_profile_nm]['aws_access_key_id']\n",
    "    os.environ['AWS_SECRET_ACCESS_KEY'] = awsconfig[_profile_nm]['aws_secret_access_key']\n",
    "    os.environ['AWS_S3_ENDPOINT']       = _endpoint\n",
    "    os.environ['AWS_PROFILE'] = _profile_nm\n",
    "    os.environ['AWS_DEFAULT_PROFILE'] = _profile_nm\n",
    "    os.environ['AWS_S3_REGION'] = _profile_nm\n",
    "except KeyError:\n",
    "    logging.error(\"Problem parsing the AWS credentials file. \")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "vscode": {
   "interpreter": {
    "hash": "d7ebce313f85fb1ac8949e834c83f371584cb2422d845bf1570c1220fdedc716"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}