"--disable-blink-features=AutomationControlled" Really?

[RonaldRuckus]

I can understand wanting to use an LLM to browse the web.

Purposely obfuscating the fact that you're an automation tool is gross and supports people using more intrusive DRM to prevent bots from accessing their sites - at the expense of real visitors.

By all means this is automation software and should respect anti-bot protections.

For context:

"--disable-blink-features=AutomationControlled" is a command line argument that prevents Chromium from indicating that it's currently being controlled by automation software. This is a typical method used by data scrapers to obfuscate their program, bypassing anti-scraping/anti-bot protection.

For the good health of the WWW this service should be more respectful. Otherwise more intrusive methods will be introduced/implemented such as the intrusive Web Environment Integrity

[suchintan]

Thanks for opening this issue! We just added it because a lot of websites just dont work without it enabled

[suchintan]

Does anyone else have a strong opinion either way? We will let the community decide whether they want to remove it or keep it. Please 👍 or 👎 !

Keep it = 👍
Remove it = 👎

[RonaldRuckus]

I feel like this is like asking car drivers if they want to add or remove pedestrian walking strips.

Implementing this is a push towards more intrusive anti-bot security that will ultimately be the demise of this project and closed-off WWW. If a website doesn't want automated sessions it should be respected.

[Ou7law007]

You know most car owners also walk and most pedestrians also own cars. These 2 are not mutually exclusive.

[RonaldRuckus]

You missed the point.

Here they are asking people who want to use this service, hence, "only asking the car drivers". It's a bias audience.

[suchintan]

The community is pretty split here. I'll leave this issue open in case anyone else has an opinion

FWIW -- the reason we left this in was because this is a pretty well understood chrome functionality -- it just goes one step further than the stuff mentioned here (https://stackoverflow.com/questions/43143014/chrome-is-being-controlled-by-automated-test-software)

[andr0s]

There's already millions of bots on the internet. One less, one more. Doesn't change anything. At the end of the day, it doesn't matter how we automate some tasks. You can spend your money on doing things using artificial networks via Skyvern, or pay your money to a cheap virtual assistant from Upwork, using human brain networks.

[piranhap]

I agree with this. There are so many bots and scrapers already that I don't think if one decides to be different it will make that much of a difference, heck, probably someone already grabbed the source code and made their own scraper already.

[RonaldRuckus]

There's already millions of bots on the internet. One less, one more. Doesn't change anything. At the end of the day, it doesn't matter how we automate some tasks. You can spend your money on doing things using artificial networks via Skyvern, or pay your money to a cheap virtual assistant from Upwork, using human brain networks.

This is such a classic bandwagon fallacy.

If everyone does it and blatantly abuses its purpose then eventually it won't be a command line feature. Similar to Geckodriver.

it doesn't matter how we automate some tasks

Yes, it does matter. If people starting ripping automation tools off the internet that misuse features to remain undetected it will become an eventual board-room discussion once the digital footprint becomes noticeable enough.

Seriously. You're all depending on a driver maintained by Google. Using these tools for scraping/automation is already on thin ice.

[andr0s]

@RonaldRuckus since you mentioned the Web environment integrity project, let me actually quote what Mozilla replied: "Any browser, server, or publisher that implements common standards is automatically part of the Web". I, as a user of the Internet, do not have any moral obligations to disclose how exactly I use the Web, what my browser agent is, what my screen resolution is, and other fingerprint features. Including the way I actually browse the web. Is apple automation script "automation"? If I have JS widget that integrates the weather forecast iframe into another website, an "automation"? it might be initiated by a real user (me) but it actually pulls the data from an unrelated source automatically (via the iframe and javascript).
Btw, selenium and similar tools are really easy to detect, without that flag.
As for the moral standpoint, the whole LLM world exists today "thanks" to the content big companies parsed (sometimes borderline stole) from authors, developers, etc., not caring about the license it has been posted with. If you're using any of those, well, congrats - it's immoral to its core. Open source does not equal to "free to be used in a commercial product", and OpenAI (and similar) are commercial products.
So I guess users who use Skyvern are already not innocent "angels" by this logic, because they are using tools that are known to be in the gray legal - and even worse moral - zone. Including me, and - I guess - you.
With all that being said, I see no practical reasons to play saints with that browser flag now

[RonaldRuckus]

I understand what you're saying and for the most part agree with it.

I am aware that there's many ways to identify a botter, but this song and dance with DRM vs script kiddies always ends up poorly for the honest users.

Sure, I can walk into an establishment and say "I have no cameras". Until enough people take photos inside and then it turns into being guilty until proven innocent - which is where we are heading.

This happens everywhere. People betray & abuse good faith and then ruin it for everyone.

These services abuse the current freedoms of the WWW so that people can extract value, consume resources, and return nothing