Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

Overview

undetected_chromedriver

https://github.com/ultrafunkamsterdam/undetected-chromedriver

Optimized Selenium Chromedriver patch which does not trigger anti-bot services like Distill Network / Imperva / DataDome / Botprotect.io Automatically downloads the driver binary and patches it.

  • Tested until current chrome beta versions
  • Works also on Brave Browser and many other Chromium based browsers, some tweaking
  • Python 3.6++**

July 2021: Currently busy implementing selenium 4 for undetected-chromedriver

newsflash: https://github.com/ultrafunkamsterdam/undetected-chromedriver/pull/255

Installation

pip install undetected-chromedriver

Usage

To prevent unnecessary hair-pulling and issue-raising, please mind the important note at the end of this document .


The Version 2 way

Literally, this is all you have to do. Settings are included and your browser executable is found automagically. This is also the snippet i recommend using in case you experience an issue.

import undetected_chromedriver.v2 as uc
driver = uc.Chrome()
with driver:
    driver.get('https://nowsecure.nl')  # known url using cloudflare's "under attack mode"

The Version 2 more advanced way, including setting profie folder

Literally, this is all you have to do. If a specified folder does not exist, a NEW profile is created. Data dirs which are specified like this will not be autoremoved on exit.

import undetected_chromedriver.v2 as uc
options = uc.ChromeOptions()

# setting profile
options.user_data_dir = "c:\\temp\\profile"

# another way to set profile is the below (which takes precedence if both variants are used
options.add_argument('--user-data-dir=c:\\temp\\profile2')

# just some options passing in to skip annoying popups
options.add_argument('--no-first-run --no-service-autorun --password-store=basic')
driver = uc.Chrome(options=options)

with driver:
    driver.get('https://nowsecure.nl')  # known url using cloudflare's "under attack mode"

The Version 2 expert mode, including Devtool/Wire events!

Literally, this is all you have to do. You can now listen and subscribe to the low level devtools-protocol. I just recently found out that is also on planning for future release of the official chromedriver. However i implemented my own for now. Since i needed it myself for investigation.

import undetected_chromedriver.v2 as uc
from pprint import pformat

driver = uc.Chrome(enable_cdp_event=True)

def mylousyprintfunction(eventdata):
    print(pformat(eventdata))
    
# set the callback to Network.dataReceived to print (yeah not much original)
driver.add_cdp_listener("Network.dataReceived", mylousyprintfunction)
driver.get('https://nowsecure.nl')  # known url using cloudflare's "under attack mode"


def mylousyprintfunction(message):
    print(pformat(message))


# for more inspiration checkout the link below
# https://chromedevtools.github.io/devtools-protocol/1-3/Network/

# and of couse 2 lousy examples
driver.add_cdp_listener('Network.requestWillBeSent', mylousyprintfunction)
driver.add_cdp_listener('Network.dataReceived', mylousyprintfunction)

# hint: a wildcard captures all events!
# driver.add_cdp_listener('*', mylousyprintfunction)

# now all these events will be printed in my console

with driver:
    driver.get('https://nowsecure.nl')


{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'type': 'other'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Upgrade-Insecure-Requests': '1',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'VeryHigh',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/'},
            'requestId': '449906A5C736D819123288133F2797E6',
            'timestamp': 190010.996717,
            'type': 'Document',
            'wallTime': 1621835932.112026}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [],
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/',
                        ':scheme': 'https',
                        'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'document',
                        'sec-fetch-mode': 'navigate',
                        'sec-fetch-site': 'none',
                        'sec-fetch-user': '?1',
                        'upgrade-insecure-requests': '1',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},
            'requestId': '449906A5C736D819123288133F2797E6'}}
{'method': 'Network.responseReceivedExtraInfo',
 'params': {'blockedCookies': [],
            'headers': {'alt-svc': 'h3-27=":443"; ma=86400, h3-28=":443"; '
                                   'ma=86400, h3-29=":443"; ma=86400',
                        'cache-control': 'private, max-age=0, no-store, '
                                         'no-cache, must-revalidate, '
                                         'post-check=0, pre-check=0',
                        'cf-ray': '65444b779ae6546f-LHR',
                        'cf-request-id': '0a3e8d7eba0000546ffd3fa000000001',
                        'content-type': 'text/html; charset=UTF-8',
                        'date': 'Mon, 24 May 2021 05:58:53 GMT',
                        'expect-ct': 'max-age=604800, '
                                     'report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
                        'expires': 'Thu, 01 Jan 1970 00:00:01 GMT',
                        'nel': '{"report_to":"cf-nel","max_age":604800}',
                        'permissions-policy': 'accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()',
                        'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=CAfobYlmWImQ90e%2B4BFBhpPYL%2FyGyBvkcWAj%2B%2FVOLoEq0NVrD5jU9m5pi%2BKI%2BOAnINLPXOCoX2psLphA5Z38aZzWNr3eW%2BDTIK%2FQidc%3D"}],"group":"cf-nel","max_age":604800}',
                        'server': 'cloudflare',
                        'vary': 'Accept-Encoding',
                        'x-frame-options': 'SAMEORIGIN'},
            'requestId': '449906A5C736D819123288133F2797E6',
            'resourceIPAddressSpace': 'Public'}}
{'method': 'Network.responseReceived',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'loaderId': '449906A5C736D819123288133F2797E6',
            'requestId': '449906A5C736D819123288133F2797E6',
            'response': {'connectionId': 158,
                         'connectionReused': False,
                         'encodedDataLength': 851,
                         'fromDiskCache': False,
                         'fromPrefetchCache': False,
                         'fromServiceWorker': False,
                         'headers': {'alt-svc': 'h3-27=":443"; ma=86400, '
                                                'h3-28=":443"; ma=86400, '
                                                'h3-29=":443"; ma=86400',
                                     'cache-control': 'private, max-age=0, '
                                                      'no-store, no-cache, '
                                                      'must-revalidate, '
                                                      'post-check=0, '
                                                      'pre-check=0',
                                     'cf-ray': '65444b779ae6546f-LHR',
                                     'cf-request-id': '0a3e8d7eba0000546ffd3fa000000001',
                                     'content-type': 'text/html; charset=UTF-8',
                                     'date': 'Mon, 24 May 2021 05:58:53 GMT',
                                     'expect-ct': 'max-age=604800, '
                                                  'report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
                                     'expires': 'Thu, 01 Jan 1970 00:00:01 GMT',
                                     'nel': '{"report_to":"cf-nel","max_age":604800}',
                                     'permissions-policy': 'accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()',
                                     'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=CAfobYlmWImQ90e%2B4BFBhpPYL%2FyGyBvkcWAj%2B%2FVOLoEq0NVrD5jU9m5pi%2BKI%2BOAnINLPXOCoX2psLphA5Z38aZzWNr3eW%2BDTIK%2FQidc%3D"}],"group":"cf-nel","max_age":604800}',
                                     'server': 'cloudflare',
                                     'vary': 'Accept-Encoding',
                                     'x-frame-options': 'SAMEORIGIN'},
                         'mimeType': 'text/html',
                         'protocol': 'h2',
                         'remoteIPAddress': '104.21.5.197',
                         'remotePort': 443,
                         'requestHeaders': {':authority': 'nowsecure.nl',
                                            ':method': 'GET',
                                            ':path': '/',
                                            ':scheme': 'https',
                                            'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
                                            'accept-encoding': 'gzip, deflate, '
                                                               'br',
                                            'accept-language': 'en-US,en;q=0.9',
                                            'sec-ch-ua': '" Not '
                                                         'A;Brand";v="99", '
                                                         '"Chromium";v="90", '
                                                         '"Google '
                                                         'Chrome";v="90"',
                                            'sec-ch-ua-mobile': '?0',
                                            'sec-fetch-dest': 'document',
                                            'sec-fetch-mode': 'navigate',
                                            'sec-fetch-site': 'none',
                                            'sec-fetch-user': '?1',
                                            'upgrade-insecure-requests': '1',
                                            'user-agent': 'Mozilla/5.0 '
                                                          '(Windows NT 10.0; '
                                                          'Win64; x64) '
                                                          'AppleWebKit/537.36 '
                                                          '(KHTML, like Gecko) '
                                                          'Chrome/90.0.4430.212 '
                                                          'Safari/537.36'},
                         'responseTime': 1621835932177.923,
                         'securityDetails': {'certificateId': 0,
                                             'certificateTransparencyCompliance': 'compliant',
                                             'cipher': 'AES_128_GCM',
                                             'issuer': 'Cloudflare Inc ECC '
                                                       'CA-3',
                                             'keyExchange': '',
                                             'keyExchangeGroup': 'X25519',
                                             'protocol': 'TLS 1.3',
                                             'sanList': ['sni.cloudflaressl.com',
                                                         '*.nowsecure.nl',
                                                         'nowsecure.nl'],
                                             'signedCertificateTimestampList': [{'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'Google '
                                                                                                   "'Argon2021' "
                                                                                                   'log',
                                                                                 'logId': 'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372839.0},
                                                                                {'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'DigiCert '
                                                                                                   'Yeti2021 '
                                                                                                   'Log',
                                                                                 'logId': '5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372823.0}],
                                             'subjectName': 'sni.cloudflaressl.com',
                                             'validFrom': 1598659200,
                                             'validTo': 1630238400},
                         'securityState': 'secure',
                         'status': 503,
                         'statusText': '',
                         'timing': {'connectEnd': 40.414,
                                    'connectStart': 0,
                                    'dnsEnd': 0,
                                    'dnsStart': 0,
                                    'proxyEnd': -1,
                                    'proxyStart': -1,
                                    'pushEnd': 0,
                                    'pushStart': 0,
                                    'receiveHeadersEnd': 60.361,
                                    'requestTime': 190011.002239,
                                    'sendEnd': 41.348,
                                    'sendStart': 41.19,
                                    'sslEnd': 40.405,
                                    'sslStart': 10.853,
                                    'workerFetchStart': -1,
                                    'workerReady': -1,
                                    'workerRespondWithSettled': -1,
                                    'workerStart': -1},
                         'url': 'https://nowsecure.nl/'},
            'timestamp': 190011.06449,
            'type': 'Document'}}
{'method': 'Page.frameStartedLoading',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700'}}
{'method': 'Page.frameNavigated',
 'params': {'frame': {'adFrameType': 'none',
                      'crossOriginIsolatedContextType': 'NotIsolated',
                      'domainAndRegistry': 'nowsecure.nl',
                      'gatedAPIFeatures': ['SharedArrayBuffers',
                                           'SharedArrayBuffersTransferAllowed'],
                      'id': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
                      'loaderId': '449906A5C736D819123288133F2797E6',
                      'mimeType': 'text/html',
                      'secureContextType': 'Secure',
                      'securityOrigin': 'https://nowsecure.nl',
                      'url': 'https://nowsecure.nl/'}}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 9835,
            'encodedDataLength': 0,
            'requestId': '449906A5C736D819123288133F2797E6',
            'timestamp': 190011.093343}}
{'method': 'Network.loadingFinished',
 'params': {'encodedDataLength': 10713,
            'requestId': '449906A5C736D819123288133F2797E6',
            'shouldReportCorbBlocking': False,
            'timestamp': 190011.064011}}
{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'stack': {'callFrames': [{'columnNumber': 51,
                                                    'functionName': '',
                                                    'lineNumber': 114,
                                                    'scriptId': '8',
                                                    'url': 'https://nowsecure.nl/'},
                                                   {'columnNumber': 9,
                                                    'functionName': '',
                                                    'lineNumber': 115,
                                                    'scriptId': '8',
                                                    'url': 'https://nowsecure.nl/'}]},
                          'type': 'script'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Referer': 'https://nowsecure.nl/',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'Low',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f'},
            'requestId': '17180.2',
            'timestamp': 190011.106133,
            'type': 'Script',
            'wallTime': 1621835932.221325}}
{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'columnNumber': 13,
                          'lineNumber': 117,
                          'type': 'parser',
                          'url': 'https://nowsecure.nl/'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Referer': 'https://nowsecure.nl/',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'Low',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f'},
            'requestId': '17180.3',
            'timestamp': 190011.106911,
            'type': 'Image',
            'wallTime': 1621835932.222102}}
{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'type': 'parser', 'url': 'https://nowsecure.nl/'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Referer': 'https://nowsecure.nl/',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'Low',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f'},
            'requestId': '17180.4',
            'timestamp': 190011.109527,
            'type': 'Image',
            'wallTime': 1621835932.224719}}
{'method': 'Page.domContentEventFired', 'params': {'timestamp': 190011.110345}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [],
            'clientSecurityState': {'initiatorIPAddressSpace': 'Public',
                                    'initiatorIsSecureContext': True,
                                    'privateNetworkRequestPolicy': 'WarnFromInsecureToMorePrivate'},
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f',
                        ':scheme': 'https',
                        'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'referer': 'https://nowsecure.nl/',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'image',
                        'sec-fetch-mode': 'no-cors',
                        'sec-fetch-site': 'same-origin',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},
            'requestId': '17180.3'}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [],
            'clientSecurityState': {'initiatorIPAddressSpace': 'Public',
                                    'initiatorIsSecureContext': True,
                                    'privateNetworkRequestPolicy': 'WarnFromInsecureToMorePrivate'},
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f',
                        ':scheme': 'https',
                        'accept': '*/*',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'referer': 'https://nowsecure.nl/',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'script',
                        'sec-fetch-mode': 'no-cors',
                        'sec-fetch-site': 'same-origin',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},
            'requestId': '17180.2'}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [],
            'clientSecurityState': {'initiatorIPAddressSpace': 'Public',
                                    'initiatorIsSecureContext': True,
                                    'privateNetworkRequestPolicy': 'WarnFromInsecureToMorePrivate'},
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f',
                        ':scheme': 'https',
                        'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'referer': 'https://nowsecure.nl/',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'image',
                        'sec-fetch-mode': 'no-cors',
                        'sec-fetch-site': 'same-origin',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},
            'requestId': '17180.4'}}
{'method': 'Network.responseReceivedExtraInfo',
 'params': {'blockedCookies': [],
            'headers': {'accept-ranges': 'bytes',
                        'cache-control': 'max-age=7200\npublic',
                        'cf-ray': '65444b781d1de604-LHR',
                        'content-length': '42',
                        'content-type': 'image/gif',
                        'date': 'Mon, 24 May 2021 05:58:53 GMT',
                        'etag': '"60a4d856-2a"',
                        'expires': 'Mon, 24 May 2021 07:58:53 GMT',
                        'last-modified': 'Wed, 19 May 2021 09:20:22 GMT',
                        'server': 'cloudflare',
                        'vary': 'Accept-Encoding',
                        'x-content-type-options': 'nosniff',
                        'x-frame-options': 'DENY'},
            'requestId': '17180.3',
            'resourceIPAddressSpace': 'Public'}}
{'method': 'Network.responseReceivedExtraInfo',
 'params': {'blockedCookies': [],
            'headers': {'accept-ranges': 'bytes',
                        'cache-control': 'max-age=7200\npublic',
                        'cf-ray': '65444b781d1fe604-LHR',
                        'content-length': '42',
                        'content-type': 'image/gif',
                        'date': 'Mon, 24 May 2021 05:58:53 GMT',
                        'etag': '"60a4d856-2a"',
                        'expires': 'Mon, 24 May 2021 07:58:53 GMT',
                        'last-modified': 'Wed, 19 May 2021 09:20:22 GMT',
                        'server': 'cloudflare',
                        'vary': 'Accept-Encoding',
                        'x-content-type-options': 'nosniff',
                        'x-frame-options': 'DENY'},
            'requestId': '17180.4',
            'resourceIPAddressSpace': 'Public'}}
{'method': 'Network.resourceChangedPriority',
 'params': {'newPriority': 'High',
            'requestId': '17180.4',
            'timestamp': 190011.171057}}
{'method': 'Network.responseReceived',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'loaderId': '449906A5C736D819123288133F2797E6',
            'requestId': '17180.3',
            'response': {'connectionId': 0,
                         'connectionReused': False,
                         'encodedDataLength': 214,
                         'fromDiskCache': False,
                         'fromPrefetchCache': False,
                         'fromServiceWorker': False,
                         'headers': {'accept-ranges': 'bytes',
                                     'cache-control': 'max-age=7200\npublic',
                                     'cf-ray': '65444b781d1de604-LHR',
                                     'content-length': '42',
                                     'content-type': 'image/gif',
                                     'date': 'Mon, 24 May 2021 05:58:53 GMT',
                                     'etag': '"60a4d856-2a"',
                                     'expires': 'Mon, 24 May 2021 07:58:53 GMT',
                                     'last-modified': 'Wed, 19 May 2021 '
                                                      '09:20:22 GMT',
                                     'server': 'cloudflare',
                                     'vary': 'Accept-Encoding',
                                     'x-content-type-options': 'nosniff',
                                     'x-frame-options': 'DENY'},
                         'mimeType': 'image/gif',
                         'protocol': 'h3-29',
                         'remoteIPAddress': '104.21.5.197',
                         'remotePort': 443,
                         'requestHeaders': {':authority': 'nowsecure.nl',
                                            ':method': 'GET',
                                            ':path': '/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f',
                                            ':scheme': 'https',
                                            'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                                            'accept-encoding': 'gzip, deflate, '
                                                               'br',
                                            'accept-language': 'en-US,en;q=0.9',
                                            'referer': 'https://nowsecure.nl/',
                                            'sec-ch-ua': '" Not '
                                                         'A;Brand";v="99", '
                                                         '"Chromium";v="90", '
                                                         '"Google '
                                                         'Chrome";v="90"',
                                            'sec-ch-ua-mobile': '?0',
                                            'sec-fetch-dest': 'image',
                                            'sec-fetch-mode': 'no-cors',
                                            'sec-fetch-site': 'same-origin',
                                            'user-agent': 'Mozilla/5.0 '
                                                          '(Windows NT 10.0; '
                                                          'Win64; x64) '
                                                          'AppleWebKit/537.36 '
                                                          '(KHTML, like Gecko) '
                                                          'Chrome/90.0.4430.212 '
                                                          'Safari/537.36'},
                         'responseTime': 1621835932265.169,
                         'securityDetails': {'certificateId': 0,
                                             'certificateTransparencyCompliance': 'compliant',
                                             'cipher': 'AES_128_GCM',
                                             'issuer': 'Cloudflare Inc ECC '
                                                       'CA-3',
                                             'keyExchange': '',
                                             'keyExchangeGroup': 'X25519',
                                             'protocol': 'QUIC',
                                             'sanList': ['sni.cloudflaressl.com',
                                                         '*.nowsecure.nl',
                                                         'nowsecure.nl'],
                                             'signedCertificateTimestampList': [{'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'Google '
                                                                                                   "'Argon2021' "
                                                                                                   'log',
                                                                                 'logId': 'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372839.0},
                                                                                {'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'DigiCert '
                                                                                                   'Yeti2021 '
                                                                                                   'Log',
                                                                                 'logId': '5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372823.0}],
                                             'subjectName': 'sni.cloudflaressl.com',
                                             'validFrom': 1598659200,
                                             'validTo': 1630238400},
                         'securityState': 'secure',
                         'status': 200,
                         'statusText': '',
                         'timing': {'connectEnd': 26.087,
                                    'connectStart': 0,
                                    'dnsEnd': 0,
                                    'dnsStart': 0,
                                    'proxyEnd': -1,
                                    'proxyStart': -1,
                                    'pushEnd': 0,
                                    'pushStart': 0,
                                    'receiveHeadersEnd': 40.709,
                                    'requestTime': 190011.109386,
                                    'sendEnd': 26.346,
                                    'sendStart': 26.182,
                                    'sslEnd': 26.087,
                                    'sslStart': 0,
                                    'workerFetchStart': -1,
                                    'workerReady': -1,
                                    'workerRespondWithSettled': -1,
                                    'workerStart': -1},
                         'url': 'https://nowsecure.nl/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=65444b779ae6546f'},
            'timestamp': 190011.174536,
            'type': 'Image'}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 42,
            'encodedDataLength': 0,
            'requestId': '17180.3',
            'timestamp': 190011.174737}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 0,
            'encodedDataLength': 44,
            'requestId': '17180.3',
            'timestamp': 190011.17524}}
{'method': 'Network.loadingFinished',
 'params': {'encodedDataLength': 258,
            'requestId': '17180.3',
            'shouldReportCorbBlocking': False,
            'timestamp': 190011.152073}}
{'method': 'Network.responseReceived',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'loaderId': '449906A5C736D819123288133F2797E6',
            'requestId': '17180.4',
            'response': {'connectionId': 0,
                         'connectionReused': True,
                         'encodedDataLength': 178,
                         'fromDiskCache': False,
                         'fromPrefetchCache': False,
                         'fromServiceWorker': False,
                         'headers': {'accept-ranges': 'bytes',
                                     'cache-control': 'max-age=7200\npublic',
                                     'cf-ray': '65444b781d1fe604-LHR',
                                     'content-length': '42',
                                     'content-type': 'image/gif',
                                     'date': 'Mon, 24 May 2021 05:58:53 GMT',
                                     'etag': '"60a4d856-2a"',
                                     'expires': 'Mon, 24 May 2021 07:58:53 GMT',
                                     'last-modified': 'Wed, 19 May 2021 '
                                                      '09:20:22 GMT',
                                     'server': 'cloudflare',
                                     'vary': 'Accept-Encoding',
                                     'x-content-type-options': 'nosniff',
                                     'x-frame-options': 'DENY'},
                         'mimeType': 'image/gif',
                         'protocol': 'h3-29',
                         'remoteIPAddress': '104.21.5.197',
                         'remotePort': 443,
                         'requestHeaders': {':authority': 'nowsecure.nl',
                                            ':method': 'GET',
                                            ':path': '/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f',
                                            ':scheme': 'https',
                                            'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                                            'accept-encoding': 'gzip, deflate, '
                                                               'br',
                                            'accept-language': 'en-US,en;q=0.9',
                                            'referer': 'https://nowsecure.nl/',
                                            'sec-ch-ua': '" Not '
                                                         'A;Brand";v="99", '
                                                         '"Chromium";v="90", '
                                                         '"Google '
                                                         'Chrome";v="90"',
                                            'sec-ch-ua-mobile': '?0',
                                            'sec-fetch-dest': 'image',
                                            'sec-fetch-mode': 'no-cors',
                                            'sec-fetch-site': 'same-origin',
                                            'user-agent': 'Mozilla/5.0 '
                                                          '(Windows NT 10.0; '
                                                          'Win64; x64) '
                                                          'AppleWebKit/537.36 '
                                                          '(KHTML, like Gecko) '
                                                          'Chrome/90.0.4430.212 '
                                                          'Safari/537.36'},
                         'responseTime': 1621835932268.067,
                         'securityDetails': {'certificateId': 0,
                                             'certificateTransparencyCompliance': 'compliant',
                                             'cipher': 'AES_128_GCM',
                                             'issuer': 'Cloudflare Inc ECC '
                                                       'CA-3',
                                             'keyExchange': '',
                                             'keyExchangeGroup': 'X25519',
                                             'protocol': 'QUIC',
                                             'sanList': ['sni.cloudflaressl.com',
                                                         '*.nowsecure.nl',
                                                         'nowsecure.nl'],
                                             'signedCertificateTimestampList': [{'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'Google '
                                                                                                   "'Argon2021' "
                                                                                                   'log',
                                                                                 'logId': 'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372839.0},
                                                                                {'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'DigiCert '
                                                                                                   'Yeti2021 '
                                                                                                   'Log',
                                                                                 'logId': '5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372823.0}],
                                             'subjectName': 'sni.cloudflaressl.com',
                                             'validFrom': 1598659200,
                                             'validTo': 1630238400},
                         'securityState': 'secure',
                         'status': 200,
                         'statusText': '',
                         'timing': {'connectEnd': -1,
                                    'connectStart': -1,
                                    'dnsEnd': -1,
                                    'dnsStart': -1,
                                    'proxyEnd': -1,
                                    'proxyStart': -1,
                                    'pushEnd': 0,
                                    'pushStart': 0,
                                    'receiveHeadersEnd': 42.415,
                                    'requestTime': 190011.110341,
                                    'sendEnd': 25.713,
                                    'sendStart': 25.609,
                                    'sslEnd': -1,
                                    'sslStart': -1,
                                    'workerFetchStart': -1,
                                    'workerReady': -1,
                                    'workerRespondWithSettled': -1,
                                    'workerStart': -1},
                         'url': 'https://nowsecure.nl/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=65444b779ae6546f'},
            'timestamp': 190011.175727,
            'type': 'Image'}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 42,
            'encodedDataLength': 0,
            'requestId': '17180.4',
            'timestamp': 190011.175856}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 0,
            'encodedDataLength': 44,
            'requestId': '17180.4',
            'timestamp': 190011.176133}}
{'method': 'Network.loadingFinished',
 'params': {'encodedDataLength': 222,
            'requestId': '17180.4',
            'shouldReportCorbBlocking': False,
            'timestamp': 190011.153335}}
{'method': 'Network.responseReceivedExtraInfo',
 'params': {'blockedCookies': [],
            'headers': {'alt-svc': 'h3-27=":443"; ma=86400, h3-28=":443"; '
                                   'ma=86400, h3-29=":443"; ma=86400',
                        'cache-control': 'max-age=0, must-revalidate',
                        'cf-ray': '65444b781d1ee604-LHR',
                        'cf-request-id': '0a3e8d7f140000e60496387000000001',
                        'content-encoding': 'br',
                        'content-type': 'text/javascript',
                        'date': 'Mon, 24 May 2021 05:58:53 GMT',
                        'expect-ct': 'max-age=604800, '
                                     'report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
                        'nel': '{"report_to":"cf-nel","max_age":604800}',
                        'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=ZtI%2Bx8B7DpI8%2FsDA72maecFVCPvIsfBOyJjT8weyiqfmrHrmcBYpRhc%2FI%2F6JmIlnxW%2F%2BBohxLi1F8mpjAUabJ0kXLYnmjGKp2Ndio9M%3D"}],"group":"cf-nel","max_age":604800}',
                        'server': 'cloudflare',
                        'vary': 'Accept-Encoding'},
            'requestId': '17180.2',
            'resourceIPAddressSpace': 'Public'}}
{'method': 'Network.responseReceived',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'loaderId': '449906A5C736D819123288133F2797E6',
            'requestId': '17180.2',
            'response': {'connectionId': 0,
                         'connectionReused': True,
                         'encodedDataLength': 510,
                         'fromDiskCache': False,
                         'fromPrefetchCache': False,
                         'fromServiceWorker': False,
                         'headers': {'alt-svc': 'h3-27=":443"; ma=86400, '
                                                'h3-28=":443"; ma=86400, '
                                                'h3-29=":443"; ma=86400',
                                     'cache-control': 'max-age=0, '
                                                      'must-revalidate',
                                     'cf-ray': '65444b781d1ee604-LHR',
                                     'cf-request-id': '0a3e8d7f140000e60496387000000001',
                                     'content-encoding': 'br',
                                     'content-type': 'text/javascript',
                                     'date': 'Mon, 24 May 2021 05:58:53 GMT',
                                     'expect-ct': 'max-age=604800, '
                                                  'report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
                                     'nel': '{"report_to":"cf-nel","max_age":604800}',
                                     'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report?s=ZtI%2Bx8B7DpI8%2FsDA72maecFVCPvIsfBOyJjT8weyiqfmrHrmcBYpRhc%2FI%2F6JmIlnxW%2F%2BBohxLi1F8mpjAUabJ0kXLYnmjGKp2Ndio9M%3D"}],"group":"cf-nel","max_age":604800}',
                                     'server': 'cloudflare',
                                     'vary': 'Accept-Encoding'},
                         'mimeType': 'text/javascript',
                         'protocol': 'h3-29',
                         'remoteIPAddress': '104.21.5.197',
                         'remotePort': 443,
                         'requestHeaders': {':authority': 'nowsecure.nl',
                                            ':method': 'GET',
                                            ':path': '/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f',
                                            ':scheme': 'https',
                                            'accept': '*/*',
                                            'accept-encoding': 'gzip, deflate, '
                                                               'br',
                                            'accept-language': 'en-US,en;q=0.9',
                                            'referer': 'https://nowsecure.nl/',
                                            'sec-ch-ua': '" Not '
                                                         'A;Brand";v="99", '
                                                         '"Chromium";v="90", '
                                                         '"Google '
                                                         'Chrome";v="90"',
                                            'sec-ch-ua-mobile': '?0',
                                            'sec-fetch-dest': 'script',
                                            'sec-fetch-mode': 'no-cors',
                                            'sec-fetch-site': 'same-origin',
                                            'user-agent': 'Mozilla/5.0 '
                                                          '(Windows NT 10.0; '
                                                          'Win64; x64) '
                                                          'AppleWebKit/537.36 '
                                                          '(KHTML, like Gecko) '
                                                          'Chrome/90.0.4430.212 '
                                                          'Safari/537.36'},
                         'responseTime': 1621835932301.817,
                         'securityDetails': {'certificateId': 0,
                                             'certificateTransparencyCompliance': 'compliant',
                                             'cipher': 'AES_128_GCM',
                                             'issuer': 'Cloudflare Inc ECC '
                                                       'CA-3',
                                             'keyExchange': '',
                                             'keyExchangeGroup': 'X25519',
                                             'protocol': 'QUIC',
                                             'sanList': ['sni.cloudflaressl.com',
                                                         '*.nowsecure.nl',
                                                         'nowsecure.nl'],
                                             'signedCertificateTimestampList': [{'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'Google '
                                                                                                   "'Argon2021' "
                                                                                                   'log',
                                                                                 'logId': 'F65C942FD1773022145418083094568EE34D131933BFDF0C2F200BCC4EF164E3',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '30450221008A25458182A6E7F608FE1492086762A367381E94137952FFD621BA2E60F7E2F702203BCDEBCE1C544DECF0A113DE12B33E299319E6240426F38F08DFC04EF2E42825',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372839.0},
                                                                                {'hashAlgorithm': 'SHA-256',
                                                                                 'logDescription': 'DigiCert '
                                                                                                   'Yeti2021 '
                                                                                                   'Log',
                                                                                 'logId': '5CDC4392FEE6AB4544B15E9AD456E61037FBD5FA47DCA17394B25EE6F6C70ECA',
                                                                                 'origin': 'Embedded '
                                                                                           'in '
                                                                                           'certificate',
                                                                                 'signatureAlgorithm': 'ECDSA',
                                                                                 'signatureData': '3046022100A95A49C7435DBFC73406AC409062C27269E6E69F443A2213F3A085E3BCBD234A022100DEA878296F8A1DB43546DC1865A4C5AD2B90664A243AE0A3A6D4925802EE68A8',
                                                                                 'status': 'Verified',
                                                                                 'timestamp': 1598706372823.0}],
                                             'subjectName': 'sni.cloudflaressl.com',
                                             'validFrom': 1598659200,
                                             'validTo': 1630238400},
                         'securityState': 'secure',
                         'status': 200,
                         'statusText': '',
                         'timing': {'connectEnd': -1,
                                    'connectStart': -1,
                                    'dnsEnd': -1,
                                    'dnsStart': -1,
                                    'proxyEnd': -1,
                                    'proxyStart': -1,
                                    'pushEnd': 0,
                                    'pushStart': 0,
                                    'receiveHeadersEnd': 78.885,
                                    'requestTime': 190011.107975,
                                    'sendEnd': 27.934,
                                    'sendStart': 27.809,
                                    'sslEnd': -1,
                                    'sslStart': -1,
                                    'workerFetchStart': -1,
                                    'workerReady': -1,
                                    'workerRespondWithSettled': -1,
                                    'workerStart': -1},
                         'url': 'https://nowsecure.nl/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=65444b779ae6546f'},
            'timestamp': 190011.188468,
            'type': 'Script'}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 31556,
            'encodedDataLength': 0,
            'requestId': '17180.2',
            'timestamp': 190011.188663}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 6737,
            'encodedDataLength': 11251,
            'requestId': '17180.2',
            'timestamp': 190011.198249}}
{'method': 'Network.dataReceived',
 'params': {'dataLength': 0,
            'encodedDataLength': 2049,
            'requestId': '17180.2',
            'timestamp': 190011.200943}}
{'method': 'Network.loadingFinished',
 'params': {'encodedDataLength': 13810,
            'requestId': '17180.2',
            'shouldReportCorbBlocking': False,
            'timestamp': 190011.198142}}
{'method': 'Page.loadEventFired', 'params': {'timestamp': 190011.204711}}
{'method': 'Page.frameScheduledNavigation',
 'params': {'delay': 12,
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'reason': 'metaTagRefresh',
            'url': 'https://nowsecure.nl/'}}
{'method': 'Page.frameStoppedLoading',
 'params': {'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700'}}
{'method': 'Network.requestWillBeSent',
 'params': {'documentURL': 'https://nowsecure.nl/',
            'frameId': 'F42BAE4BDD4E428EE2503CB5A7B4F700',
            'hasUserGesture': False,
            'initiator': {'type': 'other'},
            'loaderId': '449906A5C736D819123288133F2797E6',
            'request': {'headers': {'Referer': 'https://nowsecure.nl/',
                                    'User-Agent': 'Mozilla/5.0 (Windows NT '
                                                  '10.0; Win64; x64) '
                                                  'AppleWebKit/537.36 (KHTML, '
                                                  'like Gecko) '
                                                  'Chrome/90.0.4430.212 '
                                                  'Safari/537.36',
                                    'sec-ch-ua': '" Not A;Brand";v="99", '
                                                 '"Chromium";v="90", "Google '
                                                 'Chrome";v="90"',
                                    'sec-ch-ua-mobile': '?0'},
                        'initialPriority': 'High',
                        'method': 'GET',
                        'mixedContentType': 'none',
                        'referrerPolicy': 'strict-origin-when-cross-origin',
                        'url': 'https://nowsecure.nl/favicon.ico'},
            'requestId': '17180.5',
            'timestamp': 190011.210491,
            'type': 'Other',
            'wallTime': 1621835932.325683}}
{'method': 'Network.requestWillBeSentExtraInfo',
 'params': {'associatedCookies': [{'blockedReasons': [],
                                   'cookie': {'domain': 'nowsecure.nl',
                                              'expires': 1621839532,
                                              'httpOnly': False,
                                              'name': 'cf_chl_prog',
                                              'path': '/',
                                              'priority': 'Medium',
                                              'sameParty': False,
                                              'secure': False,
                                              'session': False,
                                              'size': 12,
                                              'sourcePort': 443,
                                              'sourceScheme': 'Secure',
                                              'value': 'e'}}],
            'clientSecurityState': {'initiatorIPAddressSpace': 'Public',
                                    'initiatorIsSecureContext': True,
                                    'privateNetworkRequestPolicy': 'WarnFromInsecureToMorePrivate'},
            'headers': {':authority': 'nowsecure.nl',
                        ':method': 'GET',
                        ':path': '/favicon.ico',
                        ':scheme': 'https',
                        'accept': 'image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8',
                        'accept-encoding': 'gzip, deflate, br',
                        'accept-language': 'en-US,en;q=0.9',
                        'cookie': 'cf_chl_prog=e',
                        'referer': 'https://nowsecure.nl/',
                        'sec-ch-ua': '" Not A;Brand";v="99", '
                                     '"Chromium";v="90", "Google '
                                     'Chrome";v="90"',
                        'sec-ch-ua-mobile': '?0',
                        'sec-fetch-dest': 'image',
                        'sec-fetch-mode': 'no-cors',
                        'sec-fetch-site': 'same-origin',
                        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; '
                                      'x64) AppleWebKit/537.36 (KHTML, like '
                                      'Gecko) Chrome/90.0.4430.212 '
                                      'Safari/537.36'},

# hopefullly you get the idea.


the easy way (v1 old stuff)

import undetected_chromedriver as uc
driver = uc.Chrome()
driver.get('https://distilnetworks.com')

target specific chrome version (v1 old stuff)

import undetected_chromedriver as uc
uc.TARGET_VERSION = 85
driver = uc.Chrome()

monkeypatch mode (v1 old stuff)

Needs to be done before importing from selenium package

import undetected_chromedriver as uc
uc.install()

from selenium.webdriver import Chrome
driver = Chrome()
driver.get('https://distilnetworks.com')

the customized way (v1 old stuff)

import undetected_chromedriver as uc

#specify chromedriver version to download and patch
uc.TARGET_VERSION = 78    

# or specify your own chromedriver binary (why you would need this, i don't know)

uc.install(
    executable_path='c:/users/user1/chromedriver.exe',
)

opts = uc.ChromeOptions()
opts.add_argument(f'--proxy-server=socks5://127.0.0.1:9050')
driver = uc.Chrome(options=opts)
driver.get('https://distilnetworks.com')

datadome.co example (v1 old stuff)

These guys have actually a powerful product, and a link to this repo, which makes me wanna test their product. Make sure you use a "clean" ip for this one.

#
# STANDARD selenium Chromedriver
#
from selenium import webdriver
chrome = webdriver.Chrome()
chrome.get('https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/')
chrome.save_screenshot('datadome_regular_webdriver.png')
True   # it caused my ip to be flagged, unfortunately


#
# UNDETECTED chromedriver (headless,even)
#
import undetected_chromedriver as uc
options = uc.ChromeOptions()
options.headless=True
options.add_argument('--headless')
chrome = uc.Chrome(options=options)
chrome.get('https://datadome.co/customers-stories/toppreise-ends-web-scraping-and-content-theft-with-datadome/')
chrome.save_screenshot('datadome_undetected_webddriver.png')

Check both saved screenhots here

Owner
Leon
DevOps | Reverse-engineering (mobile) apps | Hacking (certified) | IDA, Frida, RetDec, Ghidra, Simplify | Python, Flutter/Dart, PHP, js , Powershell, html5
Leon
Generic automation framework for acceptance testing and RPA

Robot Framework Introduction Installation Example Usage Documentation Support and contact Contributing License Introduction Robot Framework is a gener

Robot Framework 7.7k Jan 07, 2023
A friendly wrapper for modern SQLAlchemy and Alembic

A friendly wrapper for modern SQLAlchemy (v1.4 or later) and Alembic. Documentation: https://jpsca.github.io/sqla-wrapper/ Includes: A SQLAlchemy wrap

Juan-Pablo Scaletti 129 Nov 28, 2022
buX Course Enrollment Automation

buX automation BRACU - buX course enrollment automation Features: Automatically enroll into multiple courses at a time. Find courses just entering cou

Mohammad Shakib 1 Oct 06, 2022
A modern API testing tool for web applications built with Open API and GraphQL specifications.

Schemathesis Schemathesis is a modern API testing tool for web applications built with Open API and GraphQL specifications. It reads the application s

Schemathesis.io 1.6k Jan 06, 2023
A web scraping using Selenium Webdriver

Savee - Images Downloader Project using Selenium Webdriver to download images from someone's profile on https:www.savee.it website. Usage The project

Caio Eduardo Lobo 1 Dec 17, 2021
This repository contains a testing script for nmigen-boards that tries to build blinky for all the platforms provided by nmigen-boards.

Introduction This repository contains a testing script for nmigen-boards that tries to build blinky for all the platforms provided by nmigen-boards.

S.J.R. van Schaik 4 Jul 23, 2022
Hypothesis is a powerful, flexible, and easy to use library for property-based testing.

Hypothesis Hypothesis is a family of testing libraries which let you write tests parametrized by a source of examples. A Hypothesis implementation the

Hypothesis 6.4k Jan 05, 2023
Mockoon is the easiest and quickest way to run mock APIs locally. No remote deployment, no account required, open source.

Mockoon Mockoon is the easiest and quickest way to run mock APIs locally. No remote deployment, no account required, open source. It has been built wi

mockoon 4.4k Dec 30, 2022
Hamcrest matchers for Python

PyHamcrest Introduction PyHamcrest is a framework for writing matcher objects, allowing you to declaratively define "match" rules. There are a number

Hamcrest 684 Dec 29, 2022
Automatically mock your HTTP interactions to simplify and speed up testing

VCR.py 📼 This is a Python version of Ruby's VCR library. Source code https://github.com/kevin1024/vcrpy Documentation https://vcrpy.readthedocs.io/ R

Kevin McCarthy 2.3k Jan 01, 2023
Lightweight, scriptable browser as a service with an HTTP API

Splash - A javascript rendering service Splash is a javascript rendering service with an HTTP API. It's a lightweight browser with an HTTP API, implem

Scrapinghub 3.8k Jan 03, 2023
Silky smooth profiling for Django

Silk Silk is a live profiling and inspection tool for the Django framework. Silk intercepts and stores HTTP requests and database queries before prese

Jazzband 3.7k Jan 04, 2023
PyBuster A directory busting tool for web application penetration tester, written in python

PyBuster A directory busting tool for web application penetration tester, written in python. Supports custom wordlist,recursive search. Screenshots Pr

Anukul Pandey 4 Jan 30, 2022
Ab testing - The using AB test to test of difference of conversion rate

Facebook recently introduced a new type of offer that is an alternative to the current type of bidding called maximum bidding he introduced average bidding.

5 Nov 21, 2022
This is a Python script for Github Bot which uses Selenium to Automate things.

github-follow-unfollow-bot This is a Python script for Github Bot which uses Selenium to Automate things. Pre-requisites :- Python A Github Account Re

Chaudhary Hamdan 10 Jul 01, 2022
This is a simple software for fetching new changes to remote repositories automatically.

Git Autofetch Git Autofetch is a simple software for fetching new changes from a repo to local repositories after a set time interval. This program is

Shreyas Ashtamkar 10 Jul 21, 2022
Aplikasi otomasi klik di situs popcat.click menggunakan Python dan Selenium

popthe-popcat Aplikasi Otomasi Klik di situs popcat.click. aplikasi ini akan secara otomatis melakukan click pada kucing viral itu, sehingga anda tida

cndrw_ 2 Oct 07, 2022
Data App Performance Tests

Data App Performance Tests My hypothesis is that The different architectures of

Marc Skov Madsen 6 Dec 14, 2022
Bayesian A/B testing

bayesian_testing is a small package for a quick evaluation of A/B (or A/B/C/...) tests using Bayesian approach.

Matus Baniar 35 Dec 15, 2022
Donors data of Tamil Nadu Chief Ministers Relief Fund scrapped from https://ereceipt.tn.gov.in/cmprf/Interface/CMPRF/MonthWiseReport

Tamil Nadu Chief Minister's Relief Fund Donors Scrapped data from https://ereceipt.tn.gov.in/cmprf/Interface/CMPRF/MonthWiseReport Scrapper scrapper.p

Arunmozhi 5 May 18, 2021