Learning Web Scraping using BeautifulSoup Python.
BeautifulSoup is a third party Python library from Crummy. The library is designed for quick turnaround projects like screen-scraping
- Beautiful Soup parses anything you give it and does the tree traversal stuff for you.
- You can use it to find all the links of a website
- Find all the links whose urls match "foo.com"
- Find the table heading that’s got bold text, then give me that text.
- Find every "a" element that has an href attribute etc.
Beautifulsoup, Pandas and Matplotlib
- Collect the name of all the countries per continent from English Wikipedia.
- Create countries-continents dataframe, this dataframe should have two columns: country , continent.
- Collect the happiness score, GDP per capital, social support, healthy life expectancy, freedom to make life choices, generosity, and perceptions of corruption per country in 2019 from English Wikipedia and put all collected information in a dataframe .
- Create a new dataframe with all the information that you collectedand save it in a CSV file.
- Plot the distribution of happiness score per country.
- Find the top 10 happiest countries.
- What is the average of happiness.
- What are the countries below the median.
- Plot the correlation between different variables and give an interpretation. for each continent, give the happiest country.
https://en.wikipedia.org/wiki/World_Happiness_Report#2019_report https://simple.wikipedia.org/wiki/List_of_countries_by_continents