analyse_data.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8f707849",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import scipy as sc\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "#%matplotlib inline\n",
    "from IPython.display import Image"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7ca05dc",
   "metadata": {},
   "source": [
    "This code is a series of instructions using the Pandas library to load multiple CSV files containing data related to an Airbnb project. Each CSV file contains specific information that will be used for analysis or modeling."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64754249",
   "metadata": {},
   "source": [
    "# pyspark"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b73ae68f",
   "metadata": {},
   "source": [
    "users = pd.read_csv('/opt/bitnami/spark/airbnb_data/train_users_2.csv')\n",
    "countries = pd.read_csv('/opt/bitnami/spark/airbnb_data/countries.csv')\n",
    "sessions = pd.read_csv('/opt/bitnami/spark/airbnb_data/sessions2.csv')\n",
    "age_gender_bkts = pd.read_csv('/opt/bitnami/spark/airbnb_data/age_gender_bkts.csv')\n",
    "\n",
    "sample_submission_NDF = pd.read_csv('/opt/bitnami/spark/airbnb_data/sample_submission_NDF.csv')\n",
    "test_users = pd.read_csv('/opt/bitnami/spark/airbnb_data/test_users.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2758c741",
   "metadata": {},
   "outputs": [],
   "source": [
    "# country = pd.read_csv('data/countries.csv')\n",
    "# age_gender = pd.read_csv('data/age_gender_bkts.csv')\n",
    "# session = pd.read_csv('data/sessions.csv')\n",
    "train_user= pd.read_csv('data/train_user.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "def644c3",
   "metadata": {},
   "source": [
    "this code displays the shape of each dataframe loaded from the CSV files, allowing you to understand the size of each dataset and the amount of information contained in each dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "641f715d",
   "metadata": {},
   "outputs": [],
   "source": [
    "print('country' ,country.shape)\n",
    "print('age_gender',age_gender.shape)\n",
    "print('session',session.shape)\n",
    "print('train_user' ,train_user.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e17bfdaa",
   "metadata": {},
   "source": [
    "this code generates a summary dataframe called info_dataset that contains information such as the number of columns, column names and number of rows for each dataset loaded from the CSV files. The color gradient style is applied to facilitate the visualization of information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e2f34278",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = [country , age_gender ,session , train_user]\n",
    "titles = [\"countries\",\"age_genders\",\"sessions\",\"train_users\"]\n",
    "info_dataset = pd.DataFrame({},)\n",
    "info_dataset['dataset']= titles\n",
    "\n",
    "\n",
    "\n",
    "info_dataset = pd.DataFrame({},)\n",
    "info_dataset['dataset']= titles\n",
    "\n",
    "info_dataset['no_of_columns']= [len(df.columns) for df in dataset ]\n",
    "info_dataset['columns_name']= [', '.join(list(df.columns)) for df in dataset] \n",
    "info_dataset['no_of_rows'] = [len(df) for df in dataset]\n",
    "info_dataset.style.background_gradient(cmap ='Greys')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a04cddd4",
   "metadata": {},
   "source": [
    "this code is intended to show you how many null values are present in each column of each dataframe.\n",
    "This can help identify areas where data is missing and make management decisions \n",
    "of these values missing in the subsequent analysis or modelling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "884ce240",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(country.isnull().sum())\n",
    "print(age_gender.isnull().sum())\n",
    "print(session.isnull().sum())\n",
    "print(train_user.isnull().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b634ba2",
   "metadata": {},
   "source": [
    "This code is intended to show you how many missing values (or null values) are present in each column \n",
    "of each dataframe. This can help you identify areas where data is missing and decide how to\n",
    "manage these missing values in subsequent analysis or modelling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4511a36e",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(country.isna().sum())\n",
    "print(age_gender.isna().sum())\n",
    "print(session.isna().sum())\n",
    "print(train_user.isna().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65dd3cb4",
   "metadata": {},
   "source": [
    " this code generates a destination count graph per enrollment flow for training users. Each bar represents the number of users who have chosen a certain signup stream, colored according to their country destination. This chart allows you to visualize destination trends based on enrollment flow choices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f48d53ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "signup_flow = train_user[\"signup_flow\"]\n",
    "country_destination = train_user[\"country_destination\"]\n",
    "\n",
    "# Créer un graphique de comptage des destinations par signup flow\n",
    "plt.figure(figsize=(12, 6))\n",
    "sns.countplot(x=signup_flow, hue=country_destination, data=train_user)\n",
    "plt.title('Count of Country Destinations by Signup flow', fontsize=16)\n",
    "plt.xlabel('Signup flow', fontsize=14)\n",
    "plt.ylabel('Count', fontsize=14)\n",
    "plt.legend(title='Country Destination')\n",
    "plt.xticks(rotation=90)\n",
    "sns.despine()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5688b30a",
   "metadata": {},
   "source": [
    "ce code génère un graphique de comptage des flux d’enregistrement par pays de destination pour les utilisateurs de formation.\n",
    "Chaque barre représente le nombre d’utilisateurs qui ont choisi un certain flux d’enregistrement, coloré selon\n",
    "pays de destination. Le graphique est agrandi pour tenir compte\n",
    "résultats plus détaillés."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fb4e7da2",
   "metadata": {},
   "outputs": [],
   "source": [
    "filtered_data = train_user[train_user['signup_flow'] < 25]  # Vous pouvez ajuster la condition ici\n",
    "plt.figure(figsize=(24, 16))\n",
    "sns.countplot(data=filtered_data, x='country_destination', hue='signup_flow')\n",
    "plt.title('Signup Flow Count by Country Destination', fontsize=16)\n",
    "plt.xlabel('Country Destination', fontsize=14)\n",
    "plt.ylabel('Count', fontsize=14)\n",
    "plt.legend(title='Signup Flow')\n",
    "plt.xticks(rotation=90)\n",
    "sns.despine()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9406668b",
   "metadata": {},
   "source": [
    "This code generates a graph of age ranges by country destination for training users.\n",
    "Each bar represents the number of users of a certain age group, colored according to their country destination.\n",
    "The graph is enlarged to allow a more detailed visualization of the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c2b9a08",
   "metadata": {},
   "outputs": [],
   "source": [
    "filtered_data = train_user[(train_user['age'] > 18) & (train_user['age'] < 100)]  # Vous pouvez ajuster la condition ici\n",
    "plt.figure(figsize=(24, 16))\n",
    "sns.countplot(data=filtered_data, x='country_destination', hue='age')\n",
    "plt.title('Age Count by Country Destination', fontsize=16)\n",
    "plt.xlabel('Country Destination', fontsize=14)\n",
    "plt.ylabel('Count', fontsize=14)\n",
    "plt.legend(title=' Age')\n",
    "plt.xticks(rotation=90)\n",
    "sns.despine()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4b09e1d",
   "metadata": {},
   "source": [
    "It appears that you are creating a pie chart using the “matplotlib” and “seaborn” libraries to visualize the distribution of enrollment methods in the user DataFrame. The code you provided will generate a pie chart with labels indicating the registration methods and their respective percentages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8d89a96",
   "metadata": {},
   "outputs": [],
   "source": [
    "score_signup_method = train_user['signup_method'].value_counts()\n",
    "\n",
    "# Set the color palette for the pie chart\n",
    "colors = sns.color_palette('pastel')[0:len(score_signup_method)]\n",
    "\n",
    "# Create a pie chart\n",
    "plt.pie(score_signup_method, labels=score_signup_method.index, autopct='%1.1f%%', colors=colors)\n",
    "plt.title(\"Signup Method\")\n",
    "plt.show()\n",
    "\n",
    "#pyspark\n",
    "plt.savefig('res/pie_chart.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe0cca98",
   "metadata": {},
   "source": [
    "It looks like you’re trying to create a countplot using Seaborn to visualize the distribution of user enrollment methods based on the signup_flow column of a DataFrame named train_user. In addition, you save the trace as an image file using Matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1701db6",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.countplot(x='signup_flow', data=train_user , order=[5,4,3,2,1])\n",
    "plt.title(\"Signup Method\", size=12);\n",
    "\n",
    "#pyspark\n",
    "plt.savefig('res/signup_method.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91708da1",
   "metadata": {},
   "source": [
    "It looks like you’re trying to create a countplot using Seaborn to visualize the destination language distribution based on the destination_languagecolumn of a country named DataFrame. You also want to save the trace as an image file using Matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b2f572bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.countplot(x='destination_language ', data=country)\n",
    "plt.title(\"Destination Language\", size=13);\n",
    "\n",
    "#pyspark\n",
    "plt.savefig('res/destination_language.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b118015b",
   "metadata": {},
   "source": [
    "Your code snippet is on track to adjust the plot size before creating a countplot using Seaborn. You also save the plot as an image using Matplotlib. However, there is a minor fix needed: you use plt.subplots(figsize=(12,8)) to create the sublot, but not explicitly using the sublot created when you draw the countplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f3d71eb9",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.subplots(figsize=(12,8 )) #is used to adjust the size of the plot before creating any visualizations. It specifies the dimensions of the plot in inches (width, height). \n",
    "sns.countplot(y='country_destination', data=train_user ,color = 'blue')\n",
    "plt.title(\"Country Destination\", size=9);\n",
    "\n",
    "#pyspark\n",
    "plt.savefig('res/country_destination2.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ef8b5cd",
   "metadata": {},
   "source": [
    "It looks like you're creating a new DataFrame `new_df` by selecting specific columns from the `users` and `sessions` DataFrames and then saving this new DataFrame to a CSV file named 'users_sessions.csv'. This process involves creating a DataFrame with selected columns and saving it to a file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd16700f",
   "metadata": {},
   "outputs": [],
   "source": [
    "new_df = pd.DataFrame({\n",
    "    'country_destination':train_user['country_destination'],\n",
    "    'device_type':session['device_type'],})\n",
    "\n",
    "#jupyter\n",
    "new_df.to_csv('users_sessions.csv', index=False)\n",
    "\n",
    "#pyspark\n",
    "# new_df.to_csv('/opt/bitnami/spark/airbnb_data/users_sessions.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97514de3",
   "metadata": {},
   "source": [
    "It appears that you're performing some data analysis using the `new_df` DataFrame to find the most common country destinations based on the count of different device types used"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdc8d6c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "most_country_destination = new_df.groupby(['country_destination'], as_index=False)['device_type'].count().rename( columns = {'device_type':'total'})\n",
    "\n",
    "most_country_destination['%'] = round((most_country_destination['total'] / most_country_destination['total'].sum()) * 100, 2)\n",
    "most_country_destination.nlargest(15, 'total')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02d82469",
   "metadata": {},
   "source": [
    "The code you've provided is creating a bar plot using the `seaborn` library to visualize the top 15 country destinations based on their percentages in the `most_country_destination` DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "de0ec736",
   "metadata": {},
   "outputs": [],
   "source": [
    "most = sns.barplot(y='country_destination', x='%', palette='Reds_r', data=most_country_destination.nlargest(15, '%'))\n",
    "\n",
    "plt.title('TOTAL country_destination BY device_type', size=15)\n",
    "plt.yticks(fontsize=10, color='black')\n",
    "plt.xticks([0, 10, 20, 30, 40, 50, 60], ['0%', '10%', '20%', '30%', '40%', '50%', '60%'], fontsize=10)\n",
    "plt.xlabel(\"Percentage of country_destination\")\n",
    "\n",
    "plt.show()\n",
    "\n",
    "#pyspark\n",
    "plt.savefig('res/Percentage_of_country_destination.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18b45c20",
   "metadata": {},
   "source": [
    "The code you've provided is creating a heatmap to visualize the relationship between 'country_destination' and 'device_type' using the `seaborn` and `matplotlib` libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "697994d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "cont_table = pd.crosstab(train_user['country_destination'], session['device_type'])\n",
    "plt.figure(figsize=(8, 6))\n",
    "plt.title('country_destination and device_type')\n",
    "sns.heatmap(cont_table, cmap='Reds', annot=True, fmt='d')\n",
    "plt.tick_params(axis='both', which='major', labelsize=12)\n",
    "sns.despine()\n",
    "plt.show()\n",
    "\n",
    "#pyspark\n",
    "plt.savefig('res/country_destination_and_device_type.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f00152c",
   "metadata": {},
   "source": [
    "The code you've provided is creating a scatter plot using the `seaborn` and `matplotlib` libraries to visualize the relationship between 'age' from the `users` DataFrame and 'secs_elapsed' from the `sessions` DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3000b9ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(8, 6))\n",
    "sns.scatterplot(x=train_user['age'], y=train_user['gender'])\n",
    "plt.title('age and secs_elapsed', fontsize=16)\n",
    "plt.xlabel('age', fontsize=14)\n",
    "plt.ylabel('secs_elapsed', fontsize=14)\n",
    "plt.tick_params(axis='both', which='major', labelsize=12)\n",
    "sns.despine()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "305b2c35",
   "metadata": {},
   "outputs": [],
   "source": [
    "max_value = train_user['age'].max()\n",
    "print(\"Maximum value:\", max_value)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21bdcf3c",
   "metadata": {},
   "source": [
    "The code you've provided is creating a scatter plot that focuses on the relationship between 'age' from the `users` DataFrame (filtered to include only ages below 100) and 'secs_elapsed' from the `sessions` DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e09ceb7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "filtered_data = train_user[train_user['age'] < 100]\n",
    "filtered_data = filtered_data[18 < train_user['age']]\n",
    "plt.figure(figsize=(10,12))\n",
    "sns.scatterplot(x=filtered_data['age'], y=session['secs_elapsed'])\n",
    "plt.title('age and secs_elapsed', fontsize=16)\n",
    "sns.despine()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4eff937",
   "metadata": {},
   "source": [
    "The code you've provided is creating a histogram using the `seaborn` library to visualize the distribution of ages from the `users` DataFrame (filtered to include only ages below 100)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d44722b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "filtered_data = train_user[train_user['age'] <100]\n",
    "plt.figure(figsize=(8, 6))\n",
    "sns.histplot(filtered_data['age'], color='blue')\n",
    "plt.title('age and secs_elapsed', fontsize=16)\n",
    "plt.xlabel('age', fontsize=14)\n",
    "plt.ylabel('secs_elapsed', fontsize=14)\n",
    "sns.despine()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e494285e",
   "metadata": {},
   "source": [
    "The code you've provided is merging the `users`, `countries`, and `age_gender_bkts` DataFrames based on the 'country_destination' column and then creating a heatmap to visualize the correlation matrix between selected columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00139814",
   "metadata": {},
   "outputs": [],
   "source": [
    "merged_data = pd.merge(train_user, country, on='country_destination')\n",
    "merged_data = pd.merge(merged_data, age_gender, on='country_destination')\n",
    "corr_matrix = merged_data[['age','distance_km' ,'population_in_thousands']].corr()\n",
    "plt.figure(figsize=(8, 6))\n",
    "sns.heatmap(corr_matrix,annot=True, cmap='coolwarm', annot_kws={'size': 14})\n",
    "plt.title('Correlation Matrix', fontsize=16)\n",
    "plt.tick_params(axis='both', which='major', labelsize=12)\n",
    "sns.despine()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83606802",
   "metadata": {},
   "source": [
    "The code you've provided is using the `plotly.express` library to create a line plot to visualize the relationship between 'country_destination' and 'device_type' from the `new_df` DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e3e8685",
   "metadata": {},
   "outputs": [],
   "source": [
    "import plotly.express as px\n",
    "\n",
    "fig = px.line(new_df, x=\"country_destination\", y=\"device_type\", color='device_type', title='country_destination & device_type')\n",
    "\n",
    "fig.update_layout(\n",
    "    title=\"country_destination & device_type\",\n",
    "    xaxis_title=\"country_destination\",\n",
    "    yaxis_title=\"device_type\",\n",
    "    font=dict(\n",
    "        family=\"Courier New, monospace\",\n",
    "        size=15,\n",
    "        color=\"#7f7f7f\"\n",
    "    )\n",
    ")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a469354",
   "metadata": {},
   "source": [
    "It looks like you're performing some data analysis using the `users` and `countries` DataFrames, calculating the mean values of certain columns based on age and signup flow, and then sorting and selecting the top 15 rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "75d43902",
   "metadata": {},
   "outputs": [],
   "source": [
    "merged_data= pd.merge(train_user, country , on= 'country_destination')\n",
    "\n",
    "MeanSales_cat = merged_data.groupby(['age']).agg({'signup_flow':'mean', 'timestamp_first_active':'mean'})\n",
    "MeanSales_cat = MeanSales_cat.sort_values(ascending=False, by='signup_flow')[:15].reset_index()\n",
    "MeanSales_cat"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e22329c6",
   "metadata": {},
   "source": [
    "The code you've provided is creating a bar plot using the `seaborn` library to visualize the relationship between 'age' and 'signup_flow' in the `users` DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0172448",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(8,5))\n",
    "\n",
    "sns.set(style='white')\n",
    "sns.set_palette(\"PuRd\")\n",
    "\n",
    "sns.barplot(x=\"age\", y='signup_flow', data=train_user)\n",
    "plt.xticks(rotation=90)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b05832e",
   "metadata": {},
   "source": [
    "The code you've provided is performing some date-related operations on the `users` DataFrame and then creating a heatmap using the `seaborn` library to visualize the count of account creations based on the day of the week and month."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16506189",
   "metadata": {},
   "outputs": [],
   "source": [
    "date_columns = ['date_account_created', 'date_first_booking']\n",
    "for col in date_columns:\n",
    "    train_user[col] = pd.to_datetime(train_user[col], format='%Y-%m-%d')\n",
    "    \n",
    "    \n",
    "train_user['order_purchase_year'] = train_user.date_account_created.apply(lambda x: x.year)\n",
    "train_user['order_purchase_month'] = train_user.date_account_created.apply(lambda x: x.month)\n",
    "train_user['order_purchase_dayofweek'] = train_user.date_account_created.apply(lambda x: x.dayofweek)\n",
    "train_user['order_purchase_day'] = train_user['order_purchase_dayofweek'].map({0:'Mon',1:'Tue',2:'Wed',3:'Thu',4:'Fri',5:'Sat',6:'Sun'})\n",
    "train_user['order_purchase_mon'] = train_user.date_account_created.apply(lambda x: x.month).map({1:'Jan',2:'Feb',3:'Mar',4:'Apr',5:'May',6:'Jun',7:'Jul',8:'Aug',9:'Sep',10:'Oct',11:'Nov',12:'Dec'})\n",
    "\n",
    "purchase_count = train_user.groupby(['order_purchase_day', 'order_purchase_month']).nunique()['id'].unstack()\n",
    "plt.figure(figsize=(20,8))\n",
    "sns.heatmap(purchase_count.reindex(index = ['Mon','Tue','Wed','Thu','Fri','Sat','Sun']), cmap=\"coolwarm\", annot=True, fmt=\"d\", linewidths=0.2)\n",
    "plt.xlabel('Purchase Hour')\n",
    "plt.ylabel('Day of Week')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43973de6",
   "metadata": {},
   "source": [
    "This code generates a \"countplot\" chart using the Seaborn library (sns). The graph shows the number of occurrences of each destination country in the \"train_user\" dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e2cefc78",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.countplot(x='country_destination', data=train_user, order=[\"AU\",\"CA\",\"DE\",\"ES\",\"FR\",\"GB\",\"IT\",\"NL\",\"PT\",\"US\"])\n",
    "plt.title(\"country destination\", size=6)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39e70622",
   "metadata": {},
   "source": [
    "this code performs data preprocessing by filtering users according to certain criteria and transforming categories into numeric values for later analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1080791",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "\n",
    "selected_countries=[\"AU\",\"CA\",\"DE\",\"ES\",\"FR\",\"GB\",\"IT\",\"NL\",\"PT\",\"US\"]\n",
    "filtered_train_users = train_user[(train_user['gender'] != '-unknown-') &(train_user['age'] < 100) & (~train_user['age'].isnull()) & (train_user['country_destination'].isin(selected_countries))]\n",
    "\n",
    "filtered_train_users_copy = filtered_train_users.copy()\n",
    "\n",
    "label_encoder = LabelEncoder()\n",
    "\n",
    "\n",
    "filtered_train_users_copy['gender'] = label_encoder.fit_transform(filtered_train_users_copy['gender'])\n",
    "filtered_train_users_copy['country_destination'] = label_encoder.fit_transform(filtered_train_users_copy['country_destination'])\n",
    "filtered_train_users_copy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d1af099",
   "metadata": {},
   "source": [
    "This code creates a correlation matrix for selected variables in a filtered data set. Using a heat map (heatmap), it visually displays the correlation relationships between destination countries, age and gender of these people."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd067250",
   "metadata": {},
   "outputs": [],
   "source": [
    "columns_to_include = ['country_destination', 'age', 'gender']\n",
    "corr_matrix = filtered_train_users_copy[columns_to_include].corr()\n",
    "\n",
    "# Create the heatmap\n",
    "plt.figure(figsize=(8, 6))\n",
    "sns.heatmap(corr_matrix, annot=True, cmap='coolwarm', annot_kws={'size': 14})\n",
    "plt.title('Correlation Matrix', fontsize=16)\n",
    "plt.tick_params(axis='both', which='major', labelsize=12)\n",
    "sns.despine()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "852f17b5",
   "metadata": {},
   "source": [
    "This bar plot illustrates the distribution of Airbnb user destinations, namely: 'US', 'FR', 'CA', 'GB', 'ES', 'IT', 'PT', 'NL','DE', 'AU', and 'NDF' (no destination found).\n",
    "\n",
    "The analysis of the bar plot indicates that a significant portion of users have not yet made any bookings. Among the users who have made bookings, the 'US' (United States) emerges as the most popular destination. This observation can be attributed to the fact that the United States has the highest number of listings available on Airbnb compared to other destinations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59778f50",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.countplot(x='country_destination', data=train_user,\n",
    "              order=train_user['country_destination'].value_counts().index, color = 'teal' )\n",
    "plt.title('country_destination', size=15)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62da4839",
   "metadata": {},
   "source": [
    "This graph represents the gender distribution of users. It is evident from the plot that a lot of users have not specified their gender during the signup process.\n",
    "This absence of gender specification may be attributed to various factors, such as user privacy preferences or a desire to keep personal information undisclosed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2359fa8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.countplot(x='gender', data=train_user,\n",
    "order=train_user['gender'].value_counts().index, color = 'teal' )\n",
    "plt.title('gender', size=15)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91f62569",
   "metadata": {},
   "source": [
    "This graph illustrates the distribution of devices that users have used for signing up. It is notable that the majority of them have used Mac the Windows Desktop. Following closely in popularity are iPhone and iPad devices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "774ffb8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.countplot(y='first_device_type', data=train_user,\n",
    "order=train_user['first_device_type'].value_counts().index, color = 'teal' )\n",
    "plt.title('first device type', size=15)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "baaaa368",
   "metadata": {},
   "source": [
    "The graph represents the age distribution of users, giving a normal result where the majority of users fall within the age range of 18 to 50 years.\n",
    "This age distribution aligns with general trends in online services, as individuals within this age range are more likely to be active and engaged in digital platforms such as airbnb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e1359a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(10, 10))\n",
    "fig, ax = plt.subplots() \n",
    "\n",
    "filtered_age = train_user[train_user['age'] <100]\n",
    "\n",
    "sns.distplot(filtered_age['age'], color='teal')\n",
    "plt.title('Distribution of Age', fontsize=16)\n",
    "plt.xlabel('Age', fontsize=14)\n",
    "plt.ylabel('Density', fontsize=14)\n",
    "sns.despine()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "330cb1c0",
   "metadata": {},
   "source": [
    "The histogram represents the browser preferences of users during the signup process, revealing the prevalence the well-known browser such as Chrome, Safari, and Firefox."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "566c84ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(10, 10))\n",
    "\n",
    "sns.countplot(y='first_browser', data=train_user,\n",
    "              order=train_user['first_browser'].value_counts().index, color = 'teal' )\n",
    "plt.title('first_browser', size=15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c0b8167",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}