Skip to content

Commit

Permalink
'final_update'
Browse files Browse the repository at this point in the history
  • Loading branch information
yuvaneshkm committed Jun 16, 2024
1 parent 4fde08e commit d0af6c5
Show file tree
Hide file tree
Showing 3 changed files with 240 additions and 0 deletions.
32 changes: 32 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Python application

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

permissions:
contents: read

jobs:
build:

runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
python-version: ["3.10", "3.11"]

steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements_dev.txt
- name: Test with tox
run: tox
49 changes: 49 additions & 0 deletions .github/workflows/python-publish.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.

name: Upload Python Package

on:
release:
types: [published]

permissions:
contents: read

jobs:
deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements_dev.txt
pip install build
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytest -v
- name: Build package
run: python -m build
- name: Publish package
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
159 changes: 159 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
# Database Connector Package

# Overview
**dbsconnector** is a Python package designed to simplify the process of connecting to different data sources like csv files, excel sheets, google sheets,MySQL database,MongoDB database etc. This package provides a streamlined API for loading the data from different sources and return as Pandas DataFrame for any kind of DataScience, DataAnalysis and MachineLearning purpose.

# Features
* Easy connection to multiple data sources
* Return Pandas DataFrame as the Output

# Project Structure
```plaintext
dbsconnector/
├── .github/
│ └── workflows/
│ └── ci.yaml
│ └── python-publish.yaml
├── src/
│ └── dbsconnector/
│ └── databases.py
├── tests/
│ ├── unit/
│ │ └── test_unit.py
│ └── integration/
│ └── test_integration.py
├── .gitignore
├── LICENSE
├── pyproject.toml
├── README.md
├── requirements_dev.txt
├── requirements.txt
├── setup.cfg
├── setup.py
├── template.py
└── tox.ini
```

## requirements_dev.txt we use for the testing
It makes it easier to install and manage dependencies for development and testing, separate from the dependencies required for production.

## difference between requirements_dev.txt and requirements.txt
requirements.txt is used to specify the dependencies required to run the production code of a Python project, while requirements_dev.txt is used to specify the dependencies required for development and testing purposes.

## tox.ini
We use if for the testing in the python package testing against different version of the python

### how tox works tox enviornment creation
1. Install depedencies and packages
2. Run commands
3. Its a combination of the (virtualenvwrapper and makefile)
4. It creates a .tox

## pyproject.toml
it is being used for configuration the python project it is a alternative of the setup.cfg file. its containts configuration related to the build system
such as the build tool used package name version author license and dependencies.

## setup.cfg
In summary, setup.cfg is used by setuptools to configure the packaging and installation of a Python projec

## Testing python application
*types of testing*
1. Automated testing
2. Manual testing

*Mode of testing*
1. Unit testing
2. Integration tests

*Testing frameworks*
1. pytest
2. unittest
3. robotframework
4. selenium
5. behave
6. doctest

## check with the code style formatting and syntax(coding standard)
1. pylint
2. flake8(it is best because it containt 3 library pylint pycodestyle mccabe)
3. pycodestyle

## CI/CD
Implemented a robust CI/CD pipeline using GitHub Actions to automate testing, building, and deployment of this package to the PyPI repository. This ensures that every change is thoroughly tested and seamlessly deployed, maintaining the highest quality standards.

# How to use this package?

## Installation
To install the package, use pip:
```bash
pip install dbsconnector==0.1
```

## Usage

### Connecting to csv
```py
# import the module:
from dbsconnector import databases
# load the data:
df = databases.load_csv('sample.csv', ',')
# display the data:
df
```

### Connecting to Excel
```py
# import the module:
from dbsconnector import databases
# load the data:
df = databases.load_excelsheet('sample.xlsx', 'sample_sheet')
# display the data:
df
```

### Connecting to gsheet
```py
# import the module:
from dbsconnector import databases
# load the data:
df = databases.load_gsheet('17r9f4BL7sjmdLBnt92OdQP3CHK5bdT3hozg6DUJXGqU', 'sample_sheet')
# display the data:
df
```

### Connecting to MySQL
```py
# import the module:
from dbsconnector import databases
# load the data:
# You can store your database password in a different text file and add the text file to .gitignore file
with open('mysql_password.txt', 'r') as f:
password = f.read()
f.close()
df = databases.load_mysqldata('localhost', 'root', password, 'sample_database', 'sample_table')
# display the data:
df
```

### Connecting to MongoDB
```py
# import the module:
from dbsconnector import databases
# load the data:
df = databases.load_mongodbdata('localhost', 'sample_database', 'sample_collection')
# display the data:
df
```

# Contributing
* Several other data sources are to be included in upcomming versions.
* Contributions are welcome! Please open an issue or submit a pull request on GitHub.

# License
This project is licensed under the MIT License.

# Contact
For any questions or suggestions, please contact [yuvaneshkm05@gmail.com](yuvaneshkm05@gmail.com)

# Connect
Connect with me on [LinkedIn](https://www.linkedin.com/in/yuvaneshkm)

0 comments on commit d0af6c5

Please sign in to comment.