Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/35 implement regex matching #44

Merged
merged 20 commits into from
Jan 29, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
4d14a5e
Implemented regex matching, initial commit
PiaSchroeder Nov 28, 2023
8575422
Added credentials check for cubes and removed all references to set_db()
PiaSchroeder Dec 4, 2023
a47f437
Merge branch 'dev' into feat/35-implement-regex-matching
PiaSchroeder Dec 5, 2023
dd92bf4
Implemented regex matching, initial commit
PiaSchroeder Nov 28, 2023
d171ff8
Added credentials check for cubes and removed all references to set_db()
PiaSchroeder Dec 4, 2023
37c3366
fix tests
pmayd Dec 8, 2023
263da5f
refactoring Find and Result class to work with new database detection…
pmayd Dec 8, 2023
f897c77
fix find notebook and some errors related to db_name
pmayd Dec 8, 2023
07cca97
remove deprecated function
pmayd Dec 8, 2023
bea7e13
Renamed some functions and variables for consistency.
PiaSchroeder Dec 12, 2023
5d878b3
Merge branch 'dev' into feat/35-implement-regex-matching
PiaSchroeder Dec 12, 2023
76f60c5
Merged changes allowing to pass a db_name argument with find into loc…
PiaSchroeder Dec 12, 2023
380e078
Stored regex as dictionary and updated credential checks.
PiaSchroeder Dec 12, 2023
ae58504
fixed pre-commit issues
PiaSchroeder Dec 19, 2023
8d67630
fixed tests
PiaSchroeder Jan 7, 2024
0c0af52
add mocker for check_db_credentials so tests can run without proper c…
pmayd Jan 29, 2024
b856c48
fix pylint issues
pmayd Jan 29, 2024
3081fbc
properly fix pylint issues without breaking tests
pmayd Jan 29, 2024
eea0607
fix mypy issues, correct type hints
pmayd Jan 29, 2024
613a437
fix pylint issues
pmayd Jan 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 29 additions & 11 deletions nb/00_Setup.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
"source": [
"# Setup `pystatis`\n",
"\n",
"You don't need to do much to use `pystatis`. Basically, the first time you import the package, it will create a `config.ini` file under `~/.pystatis`. This file is used for storing settings, for example your credentials fpr the supported databases.\n",
"You don't need to do much to use `pystatis`. Basically, the first time you import the package, it will create a `config.ini` file under `~/.pystatis`. This file is used for storing settings, for example your credentials for the supported databases.\n",
"\n",
"To set up your credentials, we need to ask you for your username and password. This is done by the `setup_credentials()` function. It will ask you interactivly for the credentials, or you can set the following environmental variables:\n",
"To set up your credentials, we need to ask you for your username and password. This is done by the `setup_credentials()` function. It will ask you interactively for the credentials, or you can set the following environment variables:\n",
"- `PYSTATIS_GENESIS_API_USERNAME`\n",
"- `PYSTATIS_GENESIS_API_PASSWORD`\n",
"- `PYSTATIS_ZENSUS_API_USERNAME`\n",
Expand All @@ -21,15 +21,33 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"`dotenv` is uses here to load a local `.env` file that contains the above mentioned environmental variables so we don't have to input them."
"`dotenv` is used here to load a local `.env` file that contains the above mentioned environment variables so we don't have to input them."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 5,
"id": "888706f5-3a9e-4e0a-9ca6-fa430280bc03",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"pystatis version: 0.1.5\n"
]
},
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from pathlib import Path\n",
"\n",
Expand All @@ -43,13 +61,13 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 6,
"id": "ee7969b6",
"metadata": {},
"outputs": [],
"source": [
"# only execute if you want to delete your config file for test purposes\n",
"# config.delete_config()"
"pystatis.config.delete_config()"
]
},
{
Expand All @@ -67,16 +85,16 @@
"source": [
"The only thing you have to do is to set up your user credentials.\n",
"\n",
"You can do so either by:\n",
"1. specifying the 4 environment variables `PYSTATIS_GENESIS_API_USERNAME|PASSWORD`, and `PYSTATIS_ZENSUS_API_USERNAME|PASSWORD`\n",
"You can do so by following either of these steps:\n",
"1. specifying the 6 environment variables `PYSTATIS_GENESIS_API_USERNAME|PASSWORD`, `PYSTATIS_ZENSUS_API_USERNAME|PASSWORD`, `PYSTATIS_REGIO_API_USERNAME|PASSWORD`\n",
"2. calling the function `setup_credentials()` which will guide you through the process\n",
"\n",
"Even if you do 1. please call `setup_credentials()` once as it will read out the environment variables and write the credentials to the `config.ini` in your config dir."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 7,
"id": "6c301da5",
"metadata": {},
"outputs": [],
Expand Down Expand Up @@ -127,7 +145,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.6"
"version": "3.11.5"
}
},
"nbformat": 4,
Expand Down
70 changes: 21 additions & 49 deletions nb/01_Databases.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -8,67 +8,48 @@
"\n",
"Once you have at least once called `setup_credentials()` you are good to use any method of this package with any of the supported databases.\n",
"\n",
"However, before you can do so, you have to tell `pystatis` which database you want to work with. \n",
"The respective database (GENESIS-online, Zensusdatenbank, or Regionaldatenbank) will be determined automatically, all you need to provide the item code.\n",
PiaSchroeder marked this conversation as resolved.
Show resolved Hide resolved
"\n",
"This can be done with the `set_db()` function that is used to lock in the database for all subsequent called functions.\n",
"\n",
"If you don't know the names of the supported databases, ask `pystatis.config.get_supported_db()`."
"There are a few functions in pystatis that cannot determine the database automatically, such as checking your login credentials or setting a new password. In these cases you need to explicitly specify the database you want to work with. If you don't know the names of the supported databases, ask `pystatis.config.get_supported_db()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from pystatis import set_db, logincheck, db\n",
"from pystatis import logincheck\n",
"from pystatis import config"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"['genesis', 'zensus', 'regio']"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"config.get_supported_db()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Whenever `pystatis` needs to fetch data from a database, it calls `get_db()` internally to get the database set is currently set as active database. If `set_db()` was not called before, `get_db()` will throw an error and inform you about this."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# expected to fail so you know you have to call set_db() first\n",
"db.get_db()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# expected to fail!\n",
"set_db(\"test\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"set_db(\"genesis\")\n",
"logincheck()"
"logincheck('genesis')"
]
},
{
Expand All @@ -77,8 +58,7 @@
"metadata": {},
"outputs": [],
"source": [
"set_db(\"zensus\")\n",
"logincheck()"
"logincheck(\"zensus\")"
]
},
{
Expand All @@ -87,16 +67,8 @@
"metadata": {},
"outputs": [],
"source": [
"set_db(\"regio\")\n",
"logincheck()"
"logincheck(\"regio\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand All @@ -115,7 +87,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.6"
"version": "3.11.5"
}
},
"nbformat": 4,
Expand Down
13 changes: 1 addition & 12 deletions nb/cache.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,7 @@
"import logging\n",
"logging.basicConfig(level=logging.INFO)\n",
"\n",
"from pystatis import Cube, Table, clear_cache\n",
"from pystatis.db import set_db"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5c944d20",
"metadata": {},
"outputs": [],
"source": [
"set_db(\"genesis\")"
"from pystatis import Cube, Table, clear_cache"
]
},
{
Expand Down
13 changes: 1 addition & 12 deletions nb/cube.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,7 @@
"import logging\n",
"logging.basicConfig(level=logging.INFO)\n",
"\n",
"from pystatis import Cube\n",
"from pystatis.db import set_db"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "812c0e39",
"metadata": {},
"outputs": [],
"source": [
"set_db(\"genesis\")"
"from pystatis import Cube"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion nb/find.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -423,7 +423,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.7"
"version": "3.11.5"
},
"vscode": {
"interpreter": {
Expand Down
Loading
Loading