-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[IV-22-23] Objetivo 4 PMP_IV #23
Merged
+779
−114
Merged
Changes from 25 commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
24594f9
test(.toml) && docs(Assertion && Test .md): Elección de cada uno y añ…
MauronMP b1385fe
test(.toml && tasks.py) Incluido orden jupyter notebook y añadido lib…
MauronMP 4d4faf1
build(data/ && notebooks/) directorios para una buenas estructura de …
MauronMP 3a9433b
:construction_worker: ✨ feat(notebooks/): Clase Datos, lee csv y ejec…
MauronMP 27a4617
:construction_worker: ✨ feat(notebooks/): Clase Datos, lee csv y ejec…
MauronMP 9611d1a
:white_check_mark: :heavy_check_mark: :rotating_light: (tests/*) Crea…
MauronMP ac73844
Bugfix (iv.yaml): Salto linea
MauronMP a47aa13
:rotating_light: (tasks.py) Añadido check
MauronMP e04a592
:books: Docs(README.md) Añadido invoke check y test hechos
MauronMP ab588f7
:wrench: :white_check_mark: (tests/) Ejecutan .py, notebooks eliminad…
MauronMP a579ed6
:art: :books: Reestructuración ficheros proyectos #21, #22
MauronMP 653b0f6
:art: :books: Reestructuración ficheros proyectos #21, #22
MauronMP 959e196
:art: style(pmp_iv/) Quitado comentarios y generación imagenes correcta
MauronMP e338bda
:books: (docs/img/) Imagenes repetidas
MauronMP 11b5aaf
:books: :heavy_minus_sign: :rotating_light: (tasks.py) Remove --tap-s…
MauronMP 49817e4
:rotating_light: refactor(Datos.py && test_Datos.py) removed pandas, …
MauronMP dd29313
:art: style(Invoke_test.png) Updated test image demostration for #24
MauronMP 83b4e8a
✨ feat(regression_algorithm.py) Tipo enumerado, nombre algoritmos usa…
MauronMP dbdba2a
✨ feat(eda.py) Sesgado datos csv #24, #25
MauronMP 7329635
✨ feat(model_building.py) Elección mejor algoritmo regresion #26, #27…
MauronMP 0dfc4fc
👷 build(pyproject.toml): Añadido librerías para #27
MauronMP 8721910
✅ test(test_pmp_iv.py): Update to #25, #26, #29
MauronMP 11ed26f
📝 docs(docs/): Eliminado información no necesaria
MauronMP 4867518
✅ test(Invoke_test.png): Demostración funcionando #31
MauronMP 88385c7
✅ test(test_pmp_iv.py): Uso de constantes en los tests #31
MauronMP aa1d0a6
✅ test(Invoke_test.png): Tests passed #31
MauronMP 894b416
🎨 style (eda.py): Typo error #25
MauronMP e22797d
✅ test(test_pmp_iv.py): Comrpueba correlacion #26 #31
MauronMP 0efb5eb
👷 build(regression && model.py): Delete, no son necesarios para #26
MauronMP 3330d68
✨ feat(correlacion_area.py): Obtención correlación dato-area #26
MauronMP 5527a51
:rotating_light: refactor(pyproject.toml) removed scikit-learn #31
MauronMP c868000
💚 ci(pmp_iv/) Reestructuración directorio #25, #26
MauronMP 54e9cd6
✅ test(test_pmp_iv.py): Using fixtures, arrange, act
MauronMP f0894d7
💚 ci(pmp_iv/): Update imports #25, #26 and #31
MauronMP 92e90f2
♻️ refactor(eda.py): Set to float values #25
MauronMP 38206e8
♻️ refactor(correlacion_area.py): Constructor receive eda values
MauronMP 06a1081
💚 ci(pmp_iv/models/*) Delete folder #25, #26
MauronMP e2423ff
✅ test(tasks.py): Check syntax pmp_iv/ #32
MauronMP 7b6fa23
✅ test(tests_pmp:iv.py): Removed syntax test #31
MauronMP 4a389b9
💚 ci(pmp_iv/): Restructuring files #25, #26
MauronMP 21e0bf5
♻️ refactor(eda.py) Using map instead of loop for #25
MauronMP 7e330c6
♻️ refactor(correlacion_area.py): Constructor makes operations
MauronMP 8d31ded
📝 docs(README.md) Update test image and docs links.
MauronMP File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,6 @@ | ||
__pycache__ | ||
testmain_* | ||
*.pyc | ||
*.pytest_cache | ||
*.pytest_cache | ||
.ipynb_checkpoints | ||
notebooks/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# Elección de una biblioteca de aserción. | ||
|
||
Vamos a partir de las valoraciones de [esta página](https://snyk.io/advisor/python). Que hace una comparativa de cada librería para un proyecto para distintos lenguajes, en este caso python. | ||
|
||
## **Opciones**. | ||
|
||
|
||
- **grappa** Tiene como finalidad hacer que las pruebas sean más sencillas. Tiene dos tipos de asserciones: "expect y should". Tiene un sistema detallado de informes de errores "amigable". Pero tiene una mala valoración y apenas hay documentación o ejemplos además de la oficial. | ||
|
||
- **assertpy** Tiene un buen soporte para poder trabajar con estructuras nativas de python como list, set o dict entre otros. Tiene personalización de los mensajes de aserciones, aunque la valoración que tiene sigue siendo mala, aunque mejor que la anterior. | ||
|
||
|
||
- **:checkered_flag: pyhamcrest** Permite definir de manera clara las reglas, tiene un buen manejo de los mensajes de errores y ayuda así a una mejor comprensión. Tiene una amplia documentación y cuenta con muchos "[matchers](https://pyhamcrest.readthedocs.io/en/release-1.8/tutorial/#predefined-matchers)" predefinidos. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# Elección de un test runner. | ||
|
||
Vamos a partir de las valoraciones de [esta página](https://snyk.io/advisor/python). Que hace una comparativa de cada librería para un proyecto para distintos lenguajes, en este caso python. | ||
|
||
## **Opciones**. | ||
|
||
- **:checkered_flag: PyTest** Partiendo que tiene [la mejor valoración](https://snyk.io/advisor/python/pytest). Sigue el estandar TAP. Tiene como características la búsqueda automática de ficheros con nombre "test" o sufijo "_test.py", la creación de los test es bastante sencilla, permite parametrización, tiene Hooks que son útiles en las fases de setup y teardown. | ||
|
||
- **Nose2** EL sucesor de nose, [tiene buena valoración, pero es menor](https://snyk.io/advisor/python/nose2) que la de pytest. Basado en unittest2 con mejores opciones de testeo, en este caso solo ejecutará los ficheros que empiecen por "test_". Tiene menos popularidad, soporte y es como una "extensión mejorada" de unittest. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,3 +9,5 @@ automatizar: | |
fichero: tasks.py | ||
orden: invoke | ||
|
||
test: | ||
- tests/test_pmp_iv.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
from enum import Enum | ||
|
||
class Regression_algorithm(Enum): | ||
Lasso = 'Lasso' | ||
Ridge = 'Ridge' | ||
RandomForestRegressor = 'RandomForestRegressor' | ||
KNeighborsRegressor = 'KNeighborsRegressor' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
import csv | ||
|
||
class EDA(): | ||
|
||
def __init__(self): | ||
csvfile = open('data/incendiosForestales.csv', 'r', newline='') | ||
self.weather_data = csv.DictReader(csvfile) | ||
|
||
def by_date_property(self, mes, dia, propiedad=None): | ||
mes_dia = [] | ||
for row in self.weather_data: | ||
if(row['month'] == mes and row['day'] == dia and propiedad is not None): | ||
mes_dia.append( [row['X'], row['Y'],row['month'], row['day'],row[propiedad]]) | ||
elif(propiedad is None and row['month'] == mes and row['day'] == dia): | ||
mes_dia.append( [row['X'], row['Y'],row['month'], row['day']]) | ||
return mes_dia | ||
|
||
def by_property(self,propiedad): | ||
list_property = [] | ||
for row in self.weather_data: | ||
list_property.append(row[propiedad]) | ||
list_property = [float(i) for i in list_property] | ||
return list_property | ||
|
||
def weather(self): | ||
all_propierties = [] | ||
for row in self.weather_data: | ||
all_propierties.append([row['FFMC'], row['DMC'], row['DC'], row['ISI'], | ||
row['temp'], row['RH'],row['wind'], row['area']]) | ||
all_propierties = [list(map(float, sublist)) for sublist in all_propierties] | ||
return all_propierties |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
from sklearn.preprocessing import StandardScaler | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ¿Qué parte de "sólo puedes usar código propio" no has entendido? |
||
from sklearn.metrics import mean_absolute_error, r2_score | ||
from sklearn.linear_model import Lasso | ||
from sklearn.linear_model import Ridge | ||
from sklearn.ensemble import RandomForestRegressor | ||
from sklearn.neighbors import KNeighborsRegressor | ||
from pmp_iv.forest_prediction.eda import * | ||
from sklearn.model_selection import train_test_split | ||
from pmp_iv.enums.regression_algorithm import * | ||
|
||
class model_building(): | ||
|
||
def __init__(self): | ||
self.data_csv = EDA() | ||
X = EDA().weather() | ||
Y = EDA().by_property('area') | ||
self.X_train, self.X_test, self.y_train, self.y_test = train_test_split(X,Y,test_size=0.20,shuffle=True,random_state=1) | ||
|
||
def scaler_standard(self, X_train, X_test): | ||
scaler = StandardScaler() | ||
X_train_scaled = scaler.fit_transform(X_train) | ||
X_test_scaled = scaler.transform(X_test) | ||
|
||
return X_train_scaled, X_test_scaled | ||
|
||
def regresion_todos(self): | ||
X_train_scaled, X_test_scaled = self.scaler_standard(self.X_train, self.X_test) | ||
rSquare_mae = [] | ||
|
||
for i_regresion in (Lasso(), Ridge(), RandomForestRegressor(), KNeighborsRegressor()): | ||
i_regresion.fit(X_train_scaled, self.y_train) | ||
i_regresion_prediccion = i_regresion.predict(X_test_scaled) | ||
mae = mean_absolute_error(self.y_test, i_regresion_prediccion) | ||
r2 = r2_score(self.y_test, i_regresion_prediccion) | ||
rSquare_mae.append([r2,mae]) | ||
|
||
j = 0 | ||
for i in Regression_algorithm: | ||
rSquare_mae[j].append(i.value) | ||
j+=1 | ||
|
||
return rSquare_mae | ||
|
||
def get_best_results(self): | ||
listado_resultados = self.regresion_todos() | ||
listado_resultados = sorted(listado_resultados, key = lambda x: (-x[0])) | ||
return listado_resultados[0][2] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo: propierties