Selenium Headless Chrome in Python: Complete Tutorial
Selenium remains the most popular browser automation framework for Python developers. Since Chrome 109+, Google introduced the new headless mode (--headless=new), which accurately emulates the full Chrome rendering engine. Learn how to configure ChromeOptions and prevent bot detection.
2Captcha Browser API
Bypasses Cloudflare Turnstile and DataDome challenges that consistently block standard Python Selenium scripts.
The Difference Between Old Headless and `--headless=new`
In older versions of Chrome, headless mode was a separate minimal browser engine that lacked full extension support and modern WebGL features. In Chrome 109+, Google replaced it with --headless=new, which runs the exact same Chromium codebase as headful Chrome, providing identical DOM layout and CSS rendering.
Essential ChromeOptions for Selenium Python
To run modern headless Chrome without immediate detection, you must specify: options.add_argument('--headless=new'), options.add_argument('--disable-gpu'), options.add_argument('--no-sandbox'), and options.add_argument('--window-size=1920,1080').
Why Selenium Scripts Get Blocked and How to Fix It
Anti-bot systems detect Selenium through navigator.webdriver = true and missing audio/canvas noise. While libraries like undetected-chromedriver help, Cloudflare Turnstile and DataDome actively update their fingerprint hashes weekly. Using a managed scraping browser endpoint like 2Captcha solves this at the protocol level.
Comparison & Technical Benchmark
| Provider / Setup | Pricing / Rate | Protocol | Turnstile Bypass | Proxies | Commitment |
|---|---|---|---|---|---|
| Standard Selenium Headless | Free | ChromeDriver WebDriver | Blocked (10-20% success) | Manual config | $0 |
| 2Captcha Browser API (#1) | $0.15 / hr | CDP & Remote Selenium | 99.4% Automated Bypass | 220+ Countries Built-in | $0 |
Frequently Asked Questions
What is the correct syntax for headless mode in Selenium 4 Python?
Use options = webdriver.ChromeOptions(); options.add_argument('--headless=new'). Do not use the legacy '--headless' flag as it uses the deprecated separate engine.
Does Selenium headless send a different User-Agent header?
Yes. Older headless Chrome appended 'HeadlessChrome' to the User-Agent string. With --headless=new, it sends standard Chrome headers, but navigator.webdriver must still be masked.