A common question I often hear bandied about with WebDriver is, "How can I capture JavaScript errors on the page?" There is an open issue for this feature in the Selenium issue tracker, but there has been little-to-no development effort expended on solving the problem. One of the major issues is that not all browsers allow WebDriver to hook into the JavaScript execution process in a way that we could retrieve the errors effectively. Internet Explorer is especially bad about this, insisting on not providing COM methods to retrieve the JavaScript errors.
The most common suggestion for making JavaScript errors available to WebDriver code involves installing an event handler to the onerror event, capturing any uncaught errors to a global variable, and using a script execution to retrieve them. If you have access to the source code of the page you're automating, this is easy, as Alister Scott has pointed out in the past. In fact, that's the method I'd strongly prefer if I ever need to capture JavaScript errors on a page. However, many people ask about how to do it if they don't have access to modify the source code, and there, the challenge is that it's very hard to inject such an event handler early enough in the page load process to catch errors that may happen in the onload event.
As we learned in my previous series on retrieving HTTP response codes, using a proxy is an incredibly powerful way to extend the reach of your WebDriver code, working around things the browser won't, by nature, let you have. With that in mind, I've put together a brief example how to retrieve the JavaScript errors on a page, even those occuring during the onload event. Once again, I'll be using Eric Lawrence's (now Telerik's) excellent Fiddler proxy. For the reasons why, you can check out the posts I referred to previously. Also, much of the browser launch code and setup and teardown of the proxy is identical to the previous posts, so I'll omit that for the sake of brevity.
The typical approach for finding JavaScript errors is a two-phase affair. First, we must inject a script into the page to catch all uncaught JavaScript errors. Such a script usually looks something like this:
window.__webdriver_javascript_errors = [];
window.onerror = function(errorMsg, url, lineNumber) {
window.__webdriver_javascript_errors.push(
errorMsg +' (found at ' + url + ', line ' + lineNumber + ')');
};
Then, those errors can be retrieved by with WebDriver by using something like this:string errorRetrievalScript =
"return window.__webdriver_javascript_errors;";
IJavaScriptExecutor executor = driver as IJavaScriptExecutor;
ReadOnlyCollection<object> returnedList =
executor.ExecuteScript(errorRetrievalScript)
as ReadOnlyCollection<object>;
But let's assume that I have a test page with the following HTML:<!DOCTYPE html>
<html>
<head>
<title>Page with JavaScript errors on load</title>
<script>
function loadError() {
var xx = document.propertyThatDoesNotExist.xyz;
}
</script>
<head>
<body onload="loadError()">
This page has a JavaScript error in the onload event.
Usually a problem to trap.
</body>
</html>
One would expect that, since the aptly-named propertyThatDoesNotExist actually doesn't exist on the document object, a JavaScript error would be produced attempting to access the xyz property. Furthermore, since the function is called during the onload event of this page, the error will occur in that event, and indeed that's what happens. Parenthetically, you can see a page with exactly this structure, as part of Dave Haeffner's super-cool "The Internet" project, which exists to provide sample pages of "stuff you'll probably run into someday when using WebDriver."So how do we make sure our error-capture script gets injected into the page in time to catch the onload event? Luckily, with a proxy, we can do exactly that. Let's take a look at how we might perform both steps of the process with WebDriver code plus the Fiddler proxy.
Let's start with the navigation portion. Here's the method I created for that:
public static void NavigateTo(this IWebDriver driver,
string targetUrl)
{
string errorScript =
@"window.__webdriver_javascript_errors = [];
window.onerror = function(errorMsg, url, line) {
window.__webdriver_javascript_errors.push(
errorMsg + ' (found at ' + url + ', line ' + line + ')');
};";
SessionStateHandler beforeRequestHandler =
delegate(Session targetSession)
{
// Tell Fiddler to buffer the response so that we can modify
// it before it gets back to the browser.
targetSession.bBufferResponse = true;
};
SessionStateHandler beforeResponseHandler =
delegate(Session targetSession)
{
if (targetSession.fullUrl == targetUrl &&
targetSession.oResponse
.headers
.ExistsAndContains("Content-Type", "html"))
{
targetSession.utilDecodeResponse();
string responseBody =
targetSession.GetResponseBodyAsString();
string headTag =
Regex.Match(responseBody,
"<head.*>",
RegexOptions.IgnoreCase).ToString();
string addition =
headTag + "<script>" + errorScript + "</script>";
targetSession.utilReplaceOnceInResponse(headTag,
addition,
false);
}
};
FiddlerApplication.BeforeRequest += beforeRequestHandler;
FiddlerApplication.BeforeResponse += beforeResponseHandler;
driver.Url = targetUrl;
FiddlerApplication.BeforeResponse -= beforeResponseHandler;
FiddlerApplication.BeforeRequest -= beforeRequestHandler;
}
Looking closely at the code in this method, what we are doing here is attaching event handlers to manipulate the traffic sent over the wire. In the BeforeRequest event handler, we simply tell Fiddler that we want to examine and modify the response before it is sent along to the browser by setting the bBufferResponse property to true. The BeforeResponse event occurs after the response content has been received by the proxy, but before it has been forwarded to the browser. Here, we look for the close of the <head> tag in the response body, and add a <script> tag with our error-handling script immediately following. This ensures that our error-handling script is the first script executed by the browser. Note that this is an extremely crude and naive method of determining where to inject the script tag; in your implementation, you may require something a bit more sophisticated.Okay, now we have the code in place to capture the errors, we need a method to retrieve them. The earlier fragment gives you the idea how this will look, but here's the more complete version:
public static IList<string> GetJavaScriptErrors(
this IWebDriver driver, TimeSpan timeout)
{
string errorRetrievalScript =
@"var errorList = window.__webdriver_javascript_errors;
window.__webdriver_javascript_errors = [];
return errorList;";
DateTime endTime = DateTime.Now.Add(timeout);
List<string> errorList = new List<string>();
IJavaScriptExecutor executor = driver as IJavaScriptExecutor;
ReadOnlyCollection<object> returnedList =
executor.ExecuteScript(errorRetrievalScript)
as ReadOnlyCollection<object>;
while (returnedList == null && DateTime.Now < endTime)
{
System.Threading.Thread.Sleep(250);
returnedList =
executor.ExecuteScript(errorRetrievalScript)
as ReadOnlyCollection<object>;
}
if (returnedList == null)
{
return null;
}
else
{
foreach (object returnedError in returnedList)
{
errorList.Add(returnedError.ToString());
}
}
return errorList;
}
A few features to note here. First, the retrieval script clears the cached JavaScript errors as it retrieves them. That allows you to use the same technique to check for errors after any particular action that might yield JavaScript errors. Secondly, I've added a timeout to this method, just in case no JavaScript can load on the page for some reason. This will return null, and allow us to distinguish between that error condition and legitimately having no JavaScript errors on the page.One further thing you'll notice is that, as before in other examples, I'm using the "this" keyword as part of the argument for the driver argument. That allows these methods to be seen as .NET extension methods, making the syntax when using them a little cleaner. All that remains is to put these in action, like this:
private static void TestJavaScriptErrors(IWebDriver driver)
{
string url = "http://path/to/your/jserror.html";
Console.WriteLine("Navigating to {0}", url);
driver.NavigateTo(url);
IList<string> javaScriptErrors = driver.GetJavaScriptErrors();
if (javaScriptErrors == null)
{
Console.WriteLine("Could not access JavaScript errors.");
}
else
{
if (javaScriptErrors.Count > 0)
{
Console.WriteLine("Found the following JavaScript errors:");
foreach (string javaScriptError in javaScriptErrors)
{
Console.WriteLine(javaScriptError);
}
}
else
{
Console.WriteLine("No JavaScript errors found.");
}
}
}
When run against the test page above, the you will receive output similar to the following (specific error text varies from browser-to-browser, output from Internet Explorer is shown):Navigating to http://path/to/your/jserror.html
Found the following JavaScript errors:
Unable to get property 'xyz' of undefined or null reference
(found at http://path/to/your/jserror.html, line 7)
As with previous examples featuring proxies, you can see the full example in the GitHub repository for them. This particular example can be seen in the JavaScriptErrorsExample project within that solution.
I have a test responsibility concern with js error detection with webdriver. js error can happen on page load, doing some ajax operation on page etc etc. Given the Beta feature org.openqa.selenium.logging.Logs#get it is even easier to capture browser console log and so the js errors now.
ReplyDeleteShould js error detection be a different test case in itself or some thing which is part of each test case and checked implicitly by using EventFiringWebDriver and ,org.openqa.selenium.support.events.AbstractWebDriverEventListener
If former than one test case is responsible for more than one thing, especially if js error does not have a devastating impact on feature being tested than js error would still cause test to fail.
If not then should js error detection be a different test cases. In this case it may result in test steps being repeated one for functional test and other to detect js errors.
What are your thoughts on this?