Thursday, May 30, 2019

Handling Authentication Requests with Selenium - Part 2: Using a Web Proxy for Basic Authentication

As I mentioned in the immediately prior post in this series, the way to avoid having the browser prompt for credentials while using a Selenium test is by supplying the correct information in the Authorization header. Since Selenium's focus is automating the browser as close to how a user does so as possible, there's not a built-in way to examine or modify the headers. However, Selenium does make it very easy to configure the browser being automated to use a web proxy. A web proxy is a piece of software that stands between your browser and any request made of a web server, and can be made to examine, modify, or even block requests based on any number of rules. When configured to use a proxy, every request made by your browser flows through the proxy. Many businesses use proxies to ensure that only authorized resources are being accessed via business computers, or making sure that requests only come from authorized computers, or any number of other legitimate business purposes.

How do you configure your browser to use a proxy with Selenium? The code looks something like this:

Since we're Selenium users, we'll be using a proxy that allows us to programmatically start and stop it, and hook into the request/response chain via our code, and modify the results in order to interpret and replace the headers as needed. Any number of proxies could be used in this project. Many Selenium users have had great success using BrowserMob Proxy, or there are commercial options like Fiddler. Since I personally prefer FOSS options, and don't want to leave the .NET ecosystem, for our examples here, we'll be using BenderProxy. Here's the code for setting that up.

Now, how do we wire up the proper processing to mimic the browser's processing of an authentication prompt? We need to implement the addition of an Authorization header that provides the correct value, for the authentication scheme requested by the server. BenderProxy's OnResponseReceived handler happens after the response has been received from the web server, but before it's forwarded along to the browser for rendering. That gives us the opportunity to examine it, and resend another request with the proper credentials in the proper format. We're using the Basic authentication scheme in this example, and once again using The Internet sample application. Here's the code for the method:

Running the code, we'll see that when the Selenium code is run, the browser will show the authorized page, as we intended. As you can tell from the implementation code, Basic authentication is pretty simple, sending the Base64 encoding of "userName:passsword". Its simplicity is also one reason it's not used very often, as it sends the credentials across the wire, essentially in clear text. There are other, more secure authentication schemes available, and they can be automated in similar ways. The trick is knowing how to specify the value for the Authentication header. In the next post in the series, we'll look at another authentication mechanism, and how to handle something a little more complicated.

2 comments:

  1. Thanks a lot for a great post, and for .NET BrowserMob proxy alternative!

    There is one question I'm wondering about. Is there any way to make it work with Selenium Grid? Or with services like BrowserStack?
    Do I understand correctly, that in this case machine, executing test code, should have public IP address - so that node would be able to connect to proxy server that we created?

    ReplyDelete
    Replies
    1. Yes, the machine on which the browser is executing must be able to see the machine on which the proxy is running. Remember, use of a proxy is a browser feature. You can use one outside the context of Selenium WebDriver. Indeed, from the context of .NET test code, it's probably best that the proxy server be running on the same machine as the test code itself, particularly if you want to take advantage of being able to analyze or edit requests and responses. There is nothing, however, preventing you from packaging up your proxy execution code, and running on some other machine. The only issue would be how to communicate with that proxy from your test code.

      Delete