A guide to identify and solve network related issues in requests or responses during BrowserStack test session, using the Requests Debugger tool.
If you are facing network related issues while trying to run your tests on BrowserStack, you can use the Requests Debugger tool (a proxy and reverse proxy tool) that assists in identifying and solving errors on your test setup. This document will help you with:
The network logs will be captured at your end in the following scenarios:
./RequestsDebugger-Linux-x86/64 <args>
./RequestsDebugger-Mac <args>
RequestsDebugger.exe <args>
System Preferences
→ Security & Privacy
→ General
tab.
You can use the following <args>
with Requests Debugger tool to modify the default settings and also provide the external/upstream proxy (if any) details:
Flags | Details |
---|---|
--reverse-proxy-port <port> |
Port on which the Requests Debugger tool’s reverse proxy will run. Default: 9688 |
--port <port> |
Port on which the Requests Debugger tool’s proxy will run. Default: 9687 |
--scheme <https/http> |
Scheme for requests from tool to BrowserStack. Default: https |
--proxy-host <hostname> |
Hostname of the external/upstream proxy |
--proxy-port <port> |
Port of the external/upstream proxy. Default: 3128 (if hostname is provided) Example: ./RequestsDebugger-Mac --proxy-host <hostname of the proxy> --proxy-port <port of the proxy> |
--proxy-user <username> |
Username for authentication in the external/upstream proxy |
--proxy-pass <password> |
Password for authentication in the external/upstream Proxy. Default: empty (if username is provided) Example: ./RequestsDebugger-Mac --proxy-host <hostname of the proxy> --proxy-port <port of the proxy> --proxy-user <auth username of the proxy> --proxy-pass <auth password of the proxy> |
--retry-delay <milliseconds> |
Set the delay after which the failed request will be retried. Note: Failure of a request is not judged on its response code. It depends on the error which occurs while the tool makes the request. For example, ENOTFOUND .Default: 1000ms Example: ./RequestsDebugger-Mac --retry-delay 4000 |
--request-timeout <milliseconds> |
Hard timeout for killing any request fired from the tool while waiting for its response, also an error response will be sent back to the client. Default: 260000ms. Example: ./RequestsDebugger-Mac --request-timeout 30000 |
--logs-path <relative/absolute path> |
Directory where the RequestsDebuggerLogs folder will be created for storing logs. Default: Current working directory Example: ./RequestsDebugger-Mac --logs-path /path/to/your/directory |
--del-logs |
Deletes any existing logs from the RequestsDebuggerLogs/ directory and initializes new files for loggingExample: ./RequestsDebugger-Mac --del-logs |
--help |
Help from the Requests Debugger tool |
--version |
Version of the Requests Debugger tool |
https
during WebDriver initialization, change it to http
.
The Requests Debugger tool can be used in either of the following two modes:
To use tool as reverse proxy, you will have to replace hub-cloud.browserstack.com
in hub url with localhost:9688
in you test script.
To use tool as proxy, use the language specific instructions in you test script as shown below:
Set the environment variables for host and port before initializing the WebDriver
// default port: 9687
System.getProperties().put("http.proxyHost", "localhost");
System.getProperties().put("http.proxyPort", "9687");
// Initialize WebDriver
Set the WebDriver Proxy
// default port: 9687
var driver = new webdriver.Builder()
.usingServer('http://hub-cloud.browserstack.com/wd/hub')
.withCapabilities(capabilities)
.usingWebDriverProxy("http://localhost:9687")
.build();
HttpCommandExecutor commandExecutor = new HttpCommandExecutor(new Uri("http://BROWSERSTACK_USERNAME:BROWSERSTACK_ACCESS_KEY@hub-cloud.browserstack.com/wd/hub/"), TimeSpan.FromSeconds(60));
commandExecutor.Proxy = new WebProxy("http://localhost:9687", false);
// add caps, for e.g.
// ChromeOptions options = new ChromeOptions();
// options.AddAdditionalCapability("browser", "Chrome",true);
IWebDriver driver = new RemoteWebDriver(commandExecutor, options.ToCapabilities());
app.config
<!--default port: 9687-->
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<system.net>
<defaultProxy enabled="true">
<proxy
proxyaddress="http://localhost:9687"
bypassonlocal="false"
/>
</defaultProxy>
</system.net>
</configuration>
$web_driver = RemoteWebDriver::create(HUB_URL, $capabilities, null, null, 'localhost', 9687);
Environment variables can’t be used since Python doesn’t read from them. It requires modifying the client binding code. Follow the below steps to modify the client binding code:
selenium
folder in python/site-packages
. Refer the link to search for python modules in your system.selenium/webdriver/remote/
directory and open remote_connection.py
file in your editor.http = urllib3.PoolManager(timeout=self._timeout)
and change it to http = urllib3.ProxyManager('localhost:9687')
.Set environment variable in your test script before initializing the WebDriver
# default port: 9687
ENV['http_proxy'] = 'localhost:9687'
# Initialize WebDriver
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->proxy(['http'], 'http://localhost:9687');
my $driver = Selenium::Remote::Driver->new('ua'=>$ua);
Once you run your script, the following logs will be generated within RequestsDebuggerLogs/
directory. You can either mail us the logs or upload to Google Drive, OneDrive, Dropbox or any other storage provider.
TIMESTAMP_IN_UTC [#2::UNIQUE_IDENTIFIER] [Request Start] [INFO] POST http://<URL>/wd/hub/status, {"headers":{"......."}}
TIMESTAMP_IN_UTC [#2::UNIQUE_IDENTIFIER] [Request End] [INFO] POST http://<URL>/wd/hub/status, {"data":"{\"key\": \"value\"}"}
TIMESTAMP_IN_UTC [#2::UNIQUE_IDENTIFIER] [Tool Request - Retries Left: 1] [INFO] POST https://<URL>/wd/hub/status, {"method":"POST","headers":{".......","Proxy-Authorization":"Basic AaBbCcDdEeFfGg==","X-Requests-Debugger":"2::UNIQUE_IDENTIFIER"},"host":"<EXTERNAL_PROXY_URL>","port":"3130","path":"http://<URL>/wd/hub/status","data":"{\"key\": \"value\"}"}
TIMESTAMP_IN_UTC [#2::UNIQUE_IDENTIFIER] [Tool Request - Retries Left: 1] [ERROR] POST https://<URL>/wd/hub/status, {"errorMessage":"Error: getaddrinfo ENOTFOUND <EXTERNAL_PROXY_URL> <EXTERNAL_PROXY_URL>:3130"}
TIMESTAMP_IN_UTC [#2::UNIQUE_IDENTIFIER] [Tool Request - Retries Left: 0] [ERROR] POST https://<URL>/wd/hub/status, {"errorMessage":"Error: getaddrinfo ENOTFOUND <EXTERNAL_PROXY_URL> <EXTERNAL_PROXY_URL>:3130"}
TIMESTAMP_IN_UTC [#2::UNIQUE_IDENTIFIER] [Response End] [ERROR] POST http://<URL>/wd/hub/status, Status Code: 500, {"message":"Error: getaddrinfo ENOTFOUND <EXTERNAL_PROXY_URL> <EXTERNAL_PROXY_URL>:3130. Request Failed At Requests Debugger","error":"Request Failed At Requests Debugger"}
Tags | Meaning |
---|---|
TIMESTAMP_IN_UTC |
Timestamp of each event in UTC. |
#2 |
This represents the nth request which the tool served from the time it was started. |
UNIQUE_IDENTIFIER |
uuid for the request. This is used in X-Requests-Debugger custom header for usage and debugging at BrowserStack’s end. |
Request Start |
Client request entering the Requests Debugger tool. |
Request End |
This is logged when the client request is finished, that is, in case of POST request, when the client request has pushed all the data. |
Tool Request - Retries Left: X |
Request which was fired from the tool for the respective client request. It also mentions the retries left for the request. Max retries = 1. A request is retried in case it fails at the tool itself, i.e ENOTFOUND etc. |
Response End |
This specifies the final response sent to the client. |
loadPercentage
.Contact our Support team for immediate help while we work on improving our docs.
Contact our Support team for immediate help while we work on improving our docs.