Releases: orangecoding/fredy
11.0.0
The most significant update to Fredy yet 🎉
Until now, Fredy used an XHR-based approach to extract information from various providers. For each run, it sent a GET request to gather the required data. However, this approach had two major drawbacks:
- It relied on a library that is no longer maintained.
- With the growing prevalence of SPAs, XHR requests could no longer retrieve the necessary source code, as it is loaded dynamically.
To address these issues, I decided to rewrite Fredy using Puppeteer (https://pptr.dev/). With Puppeteer, Fredy can control an actual browser and behave like a real user, allowing it to wait for dynamically loaded content.
At the same time, I removed ScrapingAnt. While ScrapingAnt was incredibly useful in the early stages, it has become more of a burden lately. It was originally intended to assist with pages like Immoscout that have advanced bot protection, but even ScrapingAnt is now frequently detected as a bot. Additionally, contacting support at ScrapingAnt has proven nearly impossible.
10.4.4.
- Fixing immowelt provider
10.4.3
- improve tracking
- upgrade dependencies
10.4.1
- Fixing typo
10.4.0
- Adding a Demo Mode for Fredy
- improving Tracking
10.3.0
- Fixing ui-freeze when scrapingAnt Api key was wrong
- Fixing ui-freeze when config is not available
- Adding ability to track usage of Fredy is user allows so
10.2.0
- upgrading dependencies
- adding sqlite as adapter to store listings for later analysis
10.1.1
- leaving out immowelt search results if they do not match the exact search params
10.1.0
- regex for einsAImmobilien price normalization
- filter listings that does not have all required keys
10.0.0
- Upgrading dependencies
- Fixing Immowelt Scraper
[BREAKING CHANGE]
A listing may be edited, resulting in a price change. Previously, Fredy only stored the listing ID, so even if the price changed, Fredy wouldn’t alert on that listing again because it had already been sent. To address this issue, I modified the ID to be a hash of the ID and price. This way, if the price changes, the ID also changes, ensuring that the listing is sent again.