PageRenderTime 61ms CodeModel.GetById 21ms RepoModel.GetById 0ms app.codeStats 0ms

/markdown/tutorials/java.md

https://github.com/dbruning/docs
Markdown | 1083 lines | 866 code | 217 blank | 0 comment | 0 complexity | 18da9f4ae3abfc3791613bc4117d6c23 MD5 | raw file
Possible License(s): Apache-2.0
  1. {
  2. title: "Java",
  3. description: "How to run Selenium tests on Sauce Labs using Java",
  4. category: 'Tutorials',
  5. index: 0,
  6. image: "/images/tutorials/java.png"
  7. }
  8. ## Getting Started
  9. We suggest that you consider using [Maven](http://maven.apache.org) to build your Java project and either the
  10. [JUnit](http://www.junit.org) or the [TestNG](http://www.testng.org) library to write Selenium tests. While neither
  11. Maven, JUnit nor TestNG are required to write Java tests for Sauce, this is the framework that we'll use for this
  12. tutorial.
  13. Although this tutorial is not a comprehensive guide for getting Java and Maven set up on
  14. your system, here are some guidelines:
  15. ## Java and Maven Setup
  16. You will need to use **JDK** 1.6 (or higher) (*not the JRE*) and Maven 2.2.1 (or higher) in order to complete this tutorial.
  17. Download and install [Java](http://www.java.com/en/download/manual.jsp) if it isn't already installed on your system.
  18. **Note:** Ensure the **JAVA_HOME** environment variable is defined appropriately.
  19. For MacOS, add the following to ~/.bash_profile:
  20. ```bash
  21. export JAVA_HOME="$( /usr/libexec/java_home )”
  22. ```
  23. Go to the [Maven download](http://maven.apache.org/download.html) page to download the Maven binary distribution and extract it to your file system. Add the `bin` directory to your path, for example,
  24. On Mac or Linux:
  25. ```bash
  26. export PATH=YOUR_MAVEN_PATH/bin:$PATH
  27. ```
  28. On Windows:
  29. ```bash
  30. set PATH=YOUR_MAVEN_PATH/bin:%PATH%
  31. ```
  32. ## Setting Up a Project
  33. Setting up a project is a process of either building one from scratch or pulling a pre-existing project. For this tutorial you will pull a sample project we'll call *quickstart-webdriver-junit* (The artifact) from the project *com.saucelabs* (The GroupId). Refer to the [maven documentation](http://maven.apache.org/guides/index.html) for more information about these maven terms.
  34. First, create a project directory:
  35. ```bash
  36. mkdir -p ~/sauce-tutorial/sauce-project && cd ~/sauce-tutorial/sauce-project
  37. ```
  38. Next, download and install a sample project using your chosen testing framework using one of these Maven commands. You will be prompted to enter a group id (for example, `com.yourcompany`), artifact id (for example, `sauce-project`), version (defaults to `1.0-SNAPSHOT`), and package (default to the group id).
  39. **Note:** This step uses your Sauce username and access key. You can find your Sauce access key on your [Sauce account page]/(https://saucelabs.com/account).
  40. **JUnit example:**
  41. ```bash
  42. mvn archetype:generate -DarchetypeRepository=http://repository-saucelabs.forge.cloudbees.com/release -DarchetypeGroupId=com.saucelabs -DarchetypeArtifactId=quickstart-webdriver-junit -DgroupId=com.yourcompany -DartifactId=sauce-project -DarchetypeVersion=1.0.19 -Dversion=1.0-SNAPSHOT -Dpackage=com.yourcompany -DsauceUserName=<!—- SAUCE:USERNAME -—> -DsauceAccessKey=<!-- SAUCE:ACCESS_KEY -->
  43. ```
  44. **Note:** There may be a few prompts, use the Defaults except for ```Y: :``` enter ```Y```.
  45. There should be quite a bit of output. If there are any errors check the ```-D``` values above and ensure there are no errors. If values are left off the command line, they will be prompted for instead.
  46. ## Running Your First Test
  47. Now that you've got a JUnit or TestNG Maven project created, let's run the tests that were created by the Maven
  48. archetype generation to make sure that everything works.
  49. Run this command from your `sauce-project` directory:
  50. ```bash
  51. mvn test
  52. ```
  53. This launches Maven and will download the dependencies, compiles the source code and run the tests. After a few
  54. moments you should see that JUnit/TestNG has started. You might not see any output instantaneously, but
  55. eventually you will see the following output:
  56. ```
  57. ------------------------------------------------------
  58. T E S T S
  59. -------------------------------------------------------
  60. Running WebDriverTest
  61. Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.384 sec
  62. Running WebDriverWithHelperTest
  63. Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.743 sec
  64. Results :
  65. Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
  66. ```
  67. (The exact output will depend on the test framework you chose, but you
  68. should see all tests passing.)
  69. While the tests are running, navigate to your [Sauce Labs tests page](https://saucelabs.com/tests).
  70. From there you'll be able to see each test as it queues, runs, and finishes.
  71. You'll notice that each test has a name -- that information is sent
  72. automatically at the beginning of each test.
  73. Right now each test runs one at a time because the sample project created by the archetype isn't setup to run multiple tests in parallel, however we'll describe how to enable this in one of the later tutorials. For now,
  74. take advantage of the serial nature of the tests and click on a running test
  75. on your tests page. You'll jump to a detail view where, if you caught the
  76. test while it was running, you'll be able to watch Selenium controlling the
  77. browser. When the test finishes, the page updates with a video of the test
  78. and a list of the various Selenium commands that were sent.
  79. If you don't catch a test while it's running, you can click the test's link on the
  80. [Sauce Labs tests page](https://saucelabs.com/tests) to see the test's details and video.
  81. Now that you know that your setup worked and you were able to run your first
  82. test suite on Sauce, let's look at what actually happened under the
  83. hood. The simplest test is in the file
  84. `src/test/java/com/yourcompany/WebDriverTest.java`.
  85. The tests are very similar for both JUnit and TestNG, so we'll
  86. only describe the JUnit test in detail.
  87. ## JUnit
  88. ```java
  89. public class WebDriverTest {
  90. private WebDriver driver;
  91. @Before
  92. public void setUp() throws Exception {
  93. // Choose the browser, version, and platform to test
  94. DesiredCapabilities capabilities = DesiredCapabilities.firefox();
  95. capabilities.setCapability("version", "5");
  96. capabilities.setCapability("platform", Platform.XP);
  97. // Create the connection to Sauce Labs to run the tests
  98. this.driver = new RemoteWebDriver(
  99. new URL("http://sauceUsername:sauceAccessKey@ondemand.saucelabs.com:80/wd/hub"),
  100. capabilities);
  101. }
  102. @Test
  103. public void webDriver() throws Exception {
  104. // Make the browser get the page and check its title
  105. driver.get("http://www.amazon.com/");
  106. assertEquals("Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more", driver.getTitle());
  107. }
  108. @After
  109. public void tearDown() throws Exception {
  110. driver.quit();
  111. }
  112. }
  113. ```
  114. Let's break this test class down, chunk by chunk. First, we use the
  115. `setUp()` method to initialize the browser testing environment we will
  116. need for the tests:
  117. ```java
  118. @Before
  119. public void setUp() throws Exception {
  120. // Choose the browser, version, and platform to test
  121. DesiredCapabilities capabilities = DesiredCapabilities.firefox();
  122. capabilities.setCapability("version", "5");
  123. capabilities.setCapability("platform", Platform.XP);
  124. // Create the connection to Sauce Labs to run the tests
  125. this.driver = new RemoteWebDriver(
  126. new URL("http://sauceUsername:sauceAccessKey@ondemand.saucelabs.com:80/wd/hub"),
  127. capabilities);
  128. }
  129. ```
  130. This method is run before every test in the class (by virtue of the
  131. JUnit `org.junit.Before` annotation). We create an
  132. `org.openqa.selenium.remote.DesiredCapabilities` instance and use it
  133. to specify the browser version and platform. We then create an
  134. `org.openqa.selenium.remote.RemoteWebDriver` instance pointing at
  135. `ondemand.saucelabs.com` and using the `DesiredCapabilities`
  136. instance. This driver makes the tests use the specified browser and
  137. platform running on Sauce Labs servers to execute the test.
  138. Next, we write a simple test (annotated with org.junit.Test):
  139. ```java
  140. @Test
  141. public void webDriver() throws Exception {
  142. // Make the browser get the page and check its title
  143. driver.get("http://www.amazon.com/");
  144. assertEquals("Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more", driver.getTitle());
  145. }
  146. ```
  147. The test accesses www.amazon.com and uses a [JUnit assertion](https://github.com/junit-team/junit/wiki/Assertions)
  148. to check that the page title has the expected value. The call to
  149. `driver.getTitle()` is a
  150. [Selenium RemoteWebDriver method](http://selenium.googlecode.com/git/docs/api/java/org/openqa/selenium/remote/RemoteWebDriver.html#getTitle%28%29)
  151. that tells Selenium to return the title of the
  152. current page.
  153. ```java
  154. @After
  155. public void tearDown() throws Exception {
  156. driver.quit();
  157. }
  158. ```
  159. Finally, the `tearDown()` method is run after every test in the class (by virtue of the JUnit `org.junit.After` annotation). We call `driver.quit()` to close the Selenium session.
  160. ## TestNG
  161. The TestNG version of the `WebDriverTest` class is very similar. The
  162. main difference is that the desired browser settings are provided as
  163. parameters using the `org.testng.annotations.Parameters` and
  164. `org.testng.annotations.Optional` annotations.
  165. ```java
  166. public class WebDriverTest {
  167. private WebDriver driver;
  168. @Parameters({"username", "key", "os", "browser", "browserVersion"})
  169. @BeforeMethod
  170. public void setUp(@Optional("sauceUsername") String username,
  171. @Optional("sauceAccessKey") String key,
  172. @Optional("mac") String os,
  173. @Optional("iphone") String browser,
  174. @Optional("5.0") String browserVersion,
  175. Method method) throws Exception {
  176. // Choose the browser, version, and platform to test
  177. DesiredCapabilities capabilities = new DesiredCapabilities();
  178. capabilities.setBrowserName(browser);
  179. capabilities.setCapability("version", browserVersion);
  180. capabilities.setCapability("platform", Platform.valueOf(os));
  181. capabilities.setCapability("name", method.getName());
  182. // Create the connection to Sauce Labs to run the tests
  183. this.driver = new RemoteWebDriver(
  184. new URL("http://" + username + ":" + key + "@ondemand.saucelabs.com:80/wd/hub"),
  185. capabilities);
  186. }
  187. @Test
  188. public void webDriver() throws Exception {
  189. // Make the browser get the page and check its title
  190. driver.get("http://www.amazon.com/");
  191. assertEquals("Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more", driver.getTitle());
  192. }
  193. @AfterMethod
  194. public void tearDown() throws Exception {
  195. driver.quit();
  196. }
  197. }
  198. ```
  199. This is a very simple test, but the creation of the [`RemoteWebDriver`
  200. instance](http://selenium.googlecode.com/git/docs/api/java/index.html?org/openqa/selenium/remote/RemoteWebDriver.html)
  201. gives you access to the full power of Selenium.
  202. This test gives you the basic structure for any Selenium test that
  203. will run on Sauce Labs. Next, let's look at how you can use more
  204. Selenium functionality to create more realistic tests of your own web
  205. app.
  206. ## Using the Java Helper Library
  207. The [Java helper library](https://github.com/saucelabs/sauce-java) provides additional test functionality when
  208. using Sauce (like pass/fail reporting), and it only requires minimal changes to the test class. There are
  209. JUnit and TestNG versions of the Java helper library. The version you are using is included as a dependency in the
  210. Maven pom file.
  211. The sample project from this tutorial already includes the dependency and a
  212. test that uses it. There is nothing new to add or run here: the rest of this
  213. page explains how to include the dependency in your own project and use it in
  214. your tests. However, you can see the effect of using these features on your
  215. [Sauce Labs tests page](https://saucelabs.com/tests). The tests for the
  216. `WebDriverWithHelperTest.java` test will have a name specified in the
  217. Session column and be marked as Pass in the Results column,
  218. whereas all other tests will simply be marked as Finished.
  219. ## JUnit Test Helper
  220. To include the Java helper libraries in a JUnit project, add the following dependency to the pom.xml file (this was
  221. already created by Maven for this tutorial):
  222. ```xml
  223. <dependency>
  224. <groupId>com.saucelabs</groupId>
  225. <artifactId>sauce_junit</artifactId>
  226. <version>2.0.5</version>
  227. <scope>test</scope>
  228. </dependency>
  229. ```
  230. In addition to the WebDriver.java class, Maven creates the `WebDriverWithHelperTest` class that demonstrates how
  231. to update tests to use the Java helper library. You can find this class in the
  232. `src/test/java/com/yourcompany/WebDriverWithHelperTest.java` file shown below:
  233. ```java
  234. public class WebDriverWithHelperTest implements SauceOnDemandSessionIdProvider {
  235. public SauceOnDemandAuthentication authentication = new SauceOnDemandAuthentication(
  236. "sauceUsername", "sauceAccessKey");
  237. /**
  238. * JUnit Rule that marks the Sauce Job as passed/failed when the test succeeds or fails.
  239. * You can see the pass/fail status on your [Sauce Labs test page](https://saucelabs.com/tests).
  240. */
  241. public @Rule
  242. SauceOnDemandTestWatcher resultReportingTestWatcher = new SauceOnDemandTestWatcher(this, authentication);
  243. /**
  244. * JUnit Rule that will record the test name of the current test. This is referenced when creating the
  245. * {@link DesiredCapabilities}, so the Sauce Job is created with the test name.
  246. */
  247. public @Rule TestName testName = new TestName();
  248. private WebDriver driver;
  249. private String sessionId;
  250. @Before
  251. public void setUp() throws Exception {
  252. DesiredCapabilities capabilities = DesiredCapabilities.firefox();
  253. capabilities.setCapability("version", "17");
  254. // Note: XP is tested as Windows 2003 Server on the Sauce Cloud
  255. capabilities.setCapability("platform", Platform.XP);
  256. capabilities.setCapability("name", testName.getMethodName());
  257. this.driver = new RemoteWebDriver(
  258. new URL("http://" + authentication.getUsername() + ":" + authentication.getAccessKey() + "@ondemand.saucelabs.com:80/wd/hub"), capabilities);
  259. this.sessionId = ((RemoteWebDriver)driver).getSessionId().toString();
  260. }
  261. @Override
  262. public String getSessionId() {
  263. return sessionId;
  264. }
  265. @Test
  266. public void webDriverWithHelper() throws Exception {
  267. driver.get("http://www.amazon.com/");
  268. assertEquals("Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more", driver.getTitle());
  269. }
  270. @After
  271. public void tearDown() throws Exception {
  272. driver.quit();
  273. }
  274. }
  275. ```
  276. The `WebDriverWithHelperTest` class is fundamentally the same as the WebDriverTest class, with a couple of additions.
  277. First it implements the Sauce `SauceOnDemandSessionIdProvider` interface, which requires that a `getSessionId()` method
  278. be implemented:
  279. ```java
  280. public class WebDriverWithHelperTest implements SauceOnDemandSessionIdProvider {
  281. ```
  282. Pass your Sauce user name and Sauce access key as parameters to the `SauceOnDemandAuthentication` constructor. (You can find your
  283. Sauce access key on your [Sauce account page](https://saucelabs.com/account).) The
  284. object that is returned is passed as a parameter to the `SauceOnDemandTestWatcher` constructor. `SauceOnDemandTestWatcher`
  285. notifies Sauce if the test passed or failed.
  286. ```java
  287. public SauceOnDemandAuthentication authentication = new SauceOnDemandAuthentication("sauceUsername", "sauceAccessKey");
  288. public @Rule
  289. SauceOnDemandTestWatcher resultReportingTestWatcher = new SauceOnDemandTestWatcher(this, authentication);
  290. ```
  291. The SauceOnDemandTestWatcher instance invokes the [Sauce REST API](http://saucelabs.com/docs/rest). This is how JUnit
  292. notifies the Sauce environment if the test passed or failed. It also outputs the Sauce session id
  293. to stdout so the Sauce plugins for [Jenkins](https://wiki.jenkins-ci.org/display/JENKINS/Sauce+OnDemand+Plugin)
  294. and [Bamboo](https://marketplace.atlassian.com/plugins/com.saucelabs.bamboo.bamboo-sauceondemand-plugin)
  295. can parse the session id.
  296. Finally, notice the `testName` rule, which is used when building the
  297. `DesiredCapabilities`. This lets you specify a name for the test which
  298. will be included in reports on the Sauce Labs site so you can quickly
  299. identify which tests are failing.
  300. ### TestNG Java Helpers
  301. To include the Java helper libraries in a TestNG project, add the following dependency to the pom.xml file (this
  302. was automatically created by Maven for this tutorial):
  303. ```xml
  304. <dependency>
  305. <groupId>com.saucelabs</groupId>
  306. <artifactId>sauce_testng</artifactId>
  307. <version>2.0.5</version>
  308. <scope>test</scope>
  309. </dependency>
  310. ```
  311. As with the JUnit example, the TestNG Maven archetype creates a `WebDriverWithHelperTest` class that demonstrates
  312. how to update tests to use the Java helper library. This class is located in the
  313. `src/test/java/com/yourcompany/WebDriverWithHelperTest.java` file shown below:
  314. ```java
  315. @Listeners({SauceOnDemandTestListener.class})
  316. public class WebDriverWithHelperTest implements SauceOnDemandSessionIdProvider, SauceOnDemandAuthenticationProvider {
  317. public SauceOnDemandAuthentication authentication;
  318. private WebDriver driver;
  319. @Parameters({"username", "key", "os", "browser", "browserVersion"})
  320. @BeforeMethod
  321. // Note: XP is tested as Windows 2003 Server on the Sauce Cloud
  322. public void setUp
  323. (@Optional("sauceUsername") String username,
  324. @Optional("sauceAccessKey") String key,
  325. @Optional("XP") String os,
  326. @Optional("firefox") String browser,
  327. @Optional("17") String browserVersion, Method method) throws Exception {
  328. if (StringUtils.isNotEmpty(username) && StringUtils.isNotEmpty(key)) {
  329. authentication = new SauceOnDemandAuthentication(username, key);
  330. } else {
  331. authentication = new SauceOnDemandAuthentication();
  332. }
  333. DesiredCapabilities capabilities = new DesiredCapabilities();
  334. capabilities.setBrowserName(browser);
  335. capabilities.setCapability("version", browserVersion);
  336. capabilities.setCapability("platform", Platform.valueOf(os));
  337. capabilities.setCapability("name", method.getName());
  338. this.driver = new RemoteWebDriver(
  339. new URL("http://" + authentication.getUsername() + ":" + authentication.getAccessKey() + "@ondemand.saucelabs.com:80/wd/hub"), capabilities);
  340. }
  341. @Override
  342. public String getSessionId() {
  343. SessionId sessionId = ((RemoteWebDriver)driver).getSessionId();
  344. return (sessionId == null) ? null : sessionId.toString();
  345. }
  346. @Test
  347. public void webDriverWithHelper() throws Exception {
  348. driver.get("http://www.amazon.com/");
  349. assertEquals("Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more", driver.getTitle());
  350. }
  351. @AfterMethod
  352. public void tearDown() throws Exception {
  353. driver.quit();
  354. }
  355. @Override
  356. public SauceOnDemandAuthentication getAuthentication() {
  357. return authentication;
  358. }
  359. }
  360. ```
  361. This WebDriverWithHelperTest class is fundamentally the same as the WebDriverTest class, with a couple of additions. It
  362. is annotated with the TestNG annotation `@Listeners`, which includes the
  363. `SauceOnDemandTestListener` class. The SauceOnDemandTestListener class invokes
  364. the [Sauce REST API](http://saucelabs.com/docs/rest), which notifies Sauce if the test passed or failed.
  365. It also outputs the Sauce session id to stdout so the Sauce plugins
  366. for [Jenkins](https://wiki.jenkins-ci.org/display/JENKINS/Sauce+OnDemand+Plugin)
  367. and [Bamboo](https://marketplace.atlassian.com/plugins/com.saucelabs.bamboo.bamboo-sauceondemand-plugin)
  368. can parse the session id.
  369. ## Running Tests Against Web Applications
  370. Testing a static sandbox is one thing. Testing a real application's functionality
  371. is another. In this tutorial we'll run Selenium tests against a real
  372. live app to test signup, login and logout
  373. behaviours. When you have finished this tutorial, you'll have a good
  374. idea how to write Selenium tests for basic, multi-page interactions
  375. with a web app.
  376. The Test App
  377. ---
  378. Normally, you would test your own web app. For the purposes of this
  379. tutorial, we provide a [demo app at
  380. `tutorialapp.saucelabs.com`](http://tutorialapp.saucelabs.com) that
  381. we can run
  382. Selenium scripts against. It is an "idea competition" app called
  383. [Shootout](https://github.com/Pylons/shootout) where users vote for ideas designed for the Pyramid Python
  384. web framework. We won't test voting functionality in this demo, but feel free to play around with it.
  385. ## The Test Class
  386. The sample project in your `sauce-project` directory includes a test
  387. for this app, reproduced below. Since you have already run all the
  388. tests for the project, you have already run these 8 tests. You can
  389. view them in your [Sauce Labs tests
  390. page](https://saucelabs.com/tests).
  391. The key idea of Selenium tests is to specify input to the browser and
  392. make sure the app's response is what we want. In this case, we use
  393. Selenium to set form input values, such as username and password, and
  394. then click the submission button and check the output.
  395. The goal of this particular test is to verify login, logout, and registration
  396. functionality. To do so, we first build a few utilities to make
  397. testing each of these processes simple. Then, we use these to write
  398. short tests of both successful and unsuccessful attempts at these
  399. operations. Recall that the test code is all standard Selenium
  400. functionality -- we have only had to request that the Selenium code
  401. execute on browsers hosted by Sauce Labs.
  402. ```java
  403. public class WebDriverDemoShootoutTest {
  404. private SauceOnDemandAuthentication authentication = new SauceOnDemandAuthentication("sauceUsername", "sauceAccessKey");
  405. private WebDriver driver;
  406. @Before
  407. public void setUp() throws Exception {
  408. DesiredCapabilities capabilities = DesiredCapabilities.firefox();
  409. capabilities.setCapability("version", "17");
  410. capabilities.setCapability("platform", Platform.XP);
  411. this.driver = new RemoteWebDriver(
  412. new URL("http://" + authentication.getUsername() + ":" + authentication.getAccessKey() + "@ondemand.saucelabs.com:80/wd/hub"),
  413. capabilities);
  414. driver.get("http://tutorialapp.saucelabs.com");
  415. }
  416. @After
  417. public void tearDown() throws Exception {
  418. driver.quit();
  419. }
  420. @Test
  421. public void testLoginFailsWithBadCredentials() throws Exception {
  422. String userName = getUniqueId();
  423. String password = getUniqueId();
  424. driver.findElement(By.name("login")).sendKeys(userName);
  425. driver.findElement(By.name("password")).sendKeys(password);
  426. driver.findElement(By.cssSelector("input.login")).click();
  427. assertNotNull("Text not found", driver.findElement(By.id("message")));
  428. }
  429. @Test
  430. public void testLogout() throws Exception {
  431. Map<String, String> userDetails = createRandomUser();
  432. doRegister(userDetails, true);
  433. }
  434. @Test
  435. public void testLogin() throws Exception {
  436. Map<String, String> userDetails = createRandomUser();
  437. doRegister(userDetails, true);
  438. doLogin(userDetails.get("username"), userDetails.get("password"));
  439. }
  440. @Test
  441. public void testRegister() throws Exception {
  442. Map<String, String> userDetails = createRandomUser();
  443. doRegister(userDetails, false);
  444. assertTrue("Message not found", driver.findElement(By.cssSelector(".username")).getText().contains("You are logged in as "));
  445. }
  446. @Test
  447. public void testRegisterFailsWithoutUsername() throws Exception {
  448. Map<String, String> userDetails = createRandomUser();
  449. userDetails.put("username", "");
  450. doRegister(userDetails, false);
  451. assertEquals("Message not found", "Please enter a value", driver.findElement(By.cssSelector(".error")).getText());
  452. }
  453. @Test
  454. public void testRegisterFailsWithoutName() throws Exception {
  455. Map<String, String> userDetails = createRandomUser();
  456. userDetails.put("name", "");
  457. doRegister(userDetails, false);
  458. assertEquals("Message not found", "Please enter a value", driver.findElement(By.cssSelector(".error")).getText());
  459. }
  460. @Test
  461. public void testRegisterFailsWithMismatchedPasswords() throws Exception {
  462. Map<String, String> userDetails = createRandomUser();
  463. userDetails.put("confirm_password", getUniqueId());
  464. doRegister(userDetails, false);
  465. assertEquals("Message not found", "Fields do not match", driver.findElement(By.cssSelector(".error")).getText());
  466. }
  467. @Test
  468. public void testRegisterFailsWithBadEmail() throws Exception {
  469. Map<String, String> userDetails = createRandomUser();
  470. userDetails.put("email", "test");
  471. doRegister(userDetails, false);
  472. assertEquals("Message not found", "An email address must contain a single @", driver.findElement(By.cssSelector(".error")).getText());
  473. driver.findElement(By.id("email")).clear();
  474. driver.findElement(By.id("email")).sendKeys("@example.com");
  475. driver.findElement(By.id("form.submitted")).click();
  476. assertEquals("Message not found", "The username portion of the email address is invalid (the portion before the @: )", driver.findElement(By.cssSelector(".error")).getText());
  477. driver.findElement(By.id("email")).clear();
  478. driver.findElement(By.id("email")).sendKeys("test@example");
  479. driver.findElement(By.id("form.submitted")).click();
  480. assertEquals("Message not found", "The domain portion of the email address is invalid (the portion after the @: bob)", driver.findElement(By.cssSelector(".error")).getText());
  481. }
  482. private String getUniqueId() {
  483. return Long.toHexString(Double.doubleToLongBits(Math.random()));
  484. }
  485. private void doRegister(Map<String, String> userDetails, boolean logout) {
  486. userDetails.put("confirm_password", userDetails.get("confirm_password") != null ?
  487. userDetails.get("confirm_password") : userDetails.get("password"));
  488. driver.get("http://tutorialapp.saucelabs.com/register");
  489. driver.findElement(By.id("username")).sendKeys(userDetails.get("username"));
  490. driver.findElement(By.id("password")).sendKeys(userDetails.get("password"));
  491. driver.findElement(By.id("confirm_password")).sendKeys(userDetails.get("confirm_password"));
  492. driver.findElement(By.id("name")).sendKeys(userDetails.get("name"));
  493. driver.findElement(By.id("email")).sendKeys(userDetails.get("email"));
  494. driver.findElement(By.id("form.submitted")).click();
  495. if (logout) {
  496. doLogout();
  497. }
  498. }
  499. private void doLogout() {
  500. driver.get("http://tutorialapp.saucelabs.com/logout");
  501. assertEquals("Message not found", "Logged out successfully.", driver.findElement(By.id("message")).getText());
  502. }
  503. private Map<String, String> createRandomUser() {
  504. Map<String, String> userDetails = new HashMap<String, String>();
  505. String fakeId = getUniqueId();
  506. userDetails.put("username", fakeId);
  507. userDetails.put("password", "testpass");
  508. userDetails.put("name", "Fake " + fakeId);
  509. userDetails.put("email", fakeId + "@example.com");
  510. return userDetails;
  511. }
  512. private void doLogin(String username, String password) {
  513. driver.findElement(By.name("login")).sendKeys(username);
  514. driver.findElement(By.name("password")).sendKeys(password);
  515. driver.findElement(By.cssSelector("input.login")).click();
  516. assertEquals("Message not found", "Logged in successfully.", driver.findElement(By.id("message")).getText());
  517. }
  518. }
  519. ```
  520. Let's start with the utilities.
  521. The `createRandomUser()` function generates unique random user details for the registration
  522. and login tests. The randomness is important because it allows our tests to
  523. run in parallel as many times as we want without fear of collisions.
  524. ```java
  525. private Map<String, String> createRandomUser() {
  526. Map<String, String> userDetails = new HashMap<String, String>();
  527. String fakeId = getUniqueId();
  528. userDetails.put("username", fakeId);
  529. userDetails.put("password", "testpass");
  530. userDetails.put("name", "Fake " + fakeId);
  531. userDetails.put("email", fakeId + "@example.com");
  532. return userDetails;
  533. }
  534. ```
  535. The `doRegister()`, `doLogin()` and `doLogout()` issue requests that
  536. register, login, and logout the user, respectively. Recall that the
  537. tests, Selenium web browser client, and web app are all running on
  538. separate servers communicating over the internet. We use Selenium to
  539. manipulate the browser state (e.g. fill in forms) and submit requests
  540. (e.g. click the submit button) in order to put the user into these
  541. states.
  542. ```java
  543. private void doRegister(Map<String, String> userDetails, boolean logout) {
  544. userDetails.put("confirm_password", userDetails.get("confirm_password") != null ?
  545. userDetails.get("confirm_password") : userDetails.get("password"));
  546. driver.get("http://tutorialapp.saucelabs.com/register");
  547. driver.findElement(By.id("username")).sendKeys(userDetails.get("username"));
  548. driver.findElement(By.id("password")).sendKeys(userDetails.get("password"));
  549. driver.findElement(By.id("confirm_password")).sendKeys(userDetails.get("confirm_password"));
  550. driver.findElement(By.id("name")).sendKeys(userDetails.get("name"));
  551. driver.findElement(By.id("email")).sendKeys(userDetails.get("email"));
  552. driver.findElement(By.id("form.submitted")).click();
  553. if (logout) {
  554. doLogout();
  555. }
  556. }
  557. private void doLogout() {
  558. driver.get("http://tutorialapp.saucelabs.com/logout");
  559. assertEquals("Message not found", "Logged out successfully.", driver.findElement(By.id("message")).getText());
  560. }
  561. private void doLogin(String username, String password) {
  562. driver.findElement(By.name("login")).sendKeys(username);
  563. driver.findElement(By.name("password")).sendKeys(password);
  564. driver.findElement(By.cssSelector("input.login")).click();
  565. assertEquals("Message not found", "Logged in successfully.", driver.findElement(By.id("message")).getText());
  566. }
  567. ```
  568. In our first test, we check that logging in doesn't work with a bad
  569. username/password. Instead of using the `doLogin()` utility, which
  570. checks for successful login, we use a similar process but check that
  571. an error message is returned.
  572. ```java
  573. @Test
  574. public void testLoginFailsWithBadCredentials() throws Exception {
  575. String userName = getUniqueId();
  576. String password = getUniqueId();
  577. driver.findElement(By.name("login")).sendKeys(userName);
  578. driver.findElement(By.name("password")).sendKeys(password);
  579. driver.findElement(By.cssSelector("input.login")).click();
  580. assertNotNull("Text not found", driver.findElement(By.id("message")));
  581. }
  582. ```
  583. Next we test normal login and logout functionality using the
  584. `doLogin()` and `doLogout()` methods. The first test creates a new user
  585. and uses the `doLogout()` helper method to assert that the logout message
  586. appears. The second test creates a random user, logs it out, and then uses
  587. the `doLogin()` helper method to assert that the successful login message
  588. appears.
  589. ```java
  590. @Test
  591. public void testLogin() throws Exception {
  592. Map<String, String> userDetails = createRandomUser();
  593. doRegister(userDetails, true);
  594. doLogin(userDetails.get("username"), userDetails.get("password"));
  595. }
  596. @Test
  597. public void testLogout() throws Exception {
  598. Map<String, String> userDetails = createRandomUser();
  599. doRegister(userDetails, true);
  600. }
  601. ```
  602. Then we test Shootout's signup functionality by using the
  603. registration helper function to create a new user and assert that the
  604. user is logged in (which happens after a successful registration).
  605. ```java
  606. @Test
  607. public void testRegister() throws Exception {
  608. Map<String, String> userDetails = createRandomUser();
  609. doRegister(userDetails, false);
  610. assertTrue("Message not found", driver.findElement(By.cssSelector(".username")).getText().contains("You are logged in as "));
  611. }
  612. ```
  613. And finally we have a set of tests for validation logic in the signup form. First we test that each of the required
  614. fields results in an error on signup if the field is empty. Next we test if a mismatched password and password
  615. confirmation generate the desired error, and then we test to make sure that the app doesn't allow a successful
  616. registration if various incorrect email formats are used.
  617. ```java
  618. @Test
  619. public void testRegisterFailsWithoutUsername() throws Exception {
  620. Map<String, String> userDetails = createRandomUser();
  621. userDetails.put("username", "");
  622. doRegister(userDetails, false);
  623. assertEquals("Message not found", "Please enter a value", driver.findElement(By.cssSelector(".error")).getText());
  624. }
  625. @Test
  626. public void testRegisterFailsWithoutName() throws Exception {
  627. Map<String, String> userDetails = createRandomUser();
  628. userDetails.put("name", "");
  629. doRegister(userDetails, false);
  630. assertEquals("Message not found", "Please enter a value", driver.findElement(By.cssSelector(".error")).getText());
  631. }
  632. @Test
  633. public void testRegisterFailsWithMismatchedPasswords() throws Exception {
  634. Map<String, String> userDetails = createRandomUser();
  635. userDetails.put("confirm_password", getUniqueId());
  636. doRegister(userDetails, false);
  637. assertEquals("Message not found", "Fields do not match", driver.findElement(By.cssSelector(".error")).getText());
  638. }
  639. @Test
  640. public void testRegisterFailsWithBadEmail() throws Exception {
  641. Map<String, String> userDetails = createRandomUser();
  642. userDetails.put("email", "test");
  643. doRegister(userDetails, false);
  644. assertEquals("Message not found", "An email address must contain a single @", driver.findElement(By.cssSelector(".error")).getText());
  645. driver.findElement(By.id("email")).clear();
  646. driver.findElement(By.id("email")).sendKeys("@example.com");
  647. driver.findElement(By.id("form.submitted")).click();
  648. assertEquals("Message not found", "The username portion of the email address is invalid (the portion before the @: )", driver.findElement(By.cssSelector(".error")).getText());
  649. driver.findElement(By.id("email")).clear();
  650. driver.findElement(By.id("email")).sendKeys("test@example");
  651. driver.findElement(By.id("form.submitted")).click();
  652. assertEquals("Message not found", "The domain portion of the email address is invalid (the portion after the @: bob)", driver.findElement(By.cssSelector(".error")).getText());
  653. }
  654. ```
  655. ## Next Steps for Testing
  656. As simple as they are, these signup/login/logout tests are extremely
  657. valuable. Running them before every deployment helps to ensure that
  658. you can welcome new users into your community and get them where they
  659. need to go.
  660. If you are new to Selenium, try adding a new test to this suite. For
  661. example, reuse the registration and login methods to setup a user and
  662. test the voting functionality. To do so, you will need to try the
  663. voting functionality for yourself to understand how it *should*
  664. function. You could test that a logged in user sees voting buttons,
  665. can click them, and that the next page shows the adjusted scores.
  666. When you are comfortable with writing these types of tests, you can
  667. move on to learn more about how Sauce Labs lets you do more with
  668. Selenium.
  669. ## Running Tests in Parallel
  670. As you may recall from earlier tutorials, Selenium tests can take a long time! They may take even longer on Sauce
  671. because we start each test on a new virtual machine that has never been used before (don't worry, we don't charge
  672. you for the spin-up time).
  673. To make tests run faster, run more than one test at a time. As long as
  674. the tests are independent --- whether you're running the same test
  675. across different browsers or the tests just don't interact with each
  676. other --- there should be no problem running them
  677. simultaneously. Since we have thousands of
  678. clean virtual machines on standby, we encourage you to run as many tests
  679. as you can at once. For an overview of how many tests you can run in parallel, see the parallelization section of the
  680. [Sauce plan page](http://saucelabs.com/pricing).
  681. Keep in mind, Sauce Labs accounts have limits on the number of parallel tests they can run at once. Try to start too many, and you'll end up with tests timing out. You can find the number of parallel tests your account can run in the sidebar of your [account page](http://www.saucelabs.com/account).
  682. ### Parallel Tests in JUnit
  683. Tests can be run in parallel using JUnit, but it takes a bit of work.
  684. The [Java helper library](https://github.com/saucelabs/sauce-java) includes a `Parallelized`
  685. class that creates a dynamic thread pool that holds each thread that is running a test.
  686. **Parallelizing the WebDriverTest Class**
  687. The following `WebDriverParallelTest` class demonstrates how to update
  688. the `WebDriverTest` class so its tests run in parallel. The test is
  689. parallelized by specifying the different parameters to test with, in this
  690. case the browser and platform. Behind the scenes, the test framework
  691. creates a different instance of the test class for each set of parameters
  692. and runs them in parallel. The parameters are passed to the
  693. constructor so each instance customizes it's behavior using those
  694. parameters.
  695. In this example, we're parallelizing tests across different browsers
  696. on different platforms. Since testing an app in Firefox on Linux is
  697. independent of testing it in Chrome on Windows, we can safely run both
  698. tests in parallel. The static method `browsersStrings()` is
  699. annotated with `org.junit.runners.Parameterized.Parameters`,
  700. indicating it should be used to determine the parameters for each
  701. instance of the test. The method returns a `LinkedList` of parameters
  702. to use for each test instance's constructor. The
  703. `WebDriverParallelTest` constructor captures these
  704. parameters and `setUp()` uses them to configure the `DesiredCapabilities`.
  705. ```java
  706. @RunWith(Parallelized.class)
  707. public class WebDriverParallelTest {
  708. private String browser;
  709. private String os;
  710. private String version;
  711. public WebDriverParallelTest(String os, String version, String browser) {
  712. super();
  713. this.os = os;
  714. this.version = version;
  715. this.browser = browser;
  716. }
  717. @Parameterized.Parameters
  718. public static LinkedList browsersStrings() throws Exception {
  719. LinkedList browsers = new LinkedList();
  720. browsers.add(new String[]{Platform.XP.toString(), "17", "firefox"});
  721. //add any additional browsers here
  722. return browsers;
  723. }
  724. private WebDriver driver;
  725. @Before
  726. public void setUp() throws Exception {
  727. DesiredCapabilities capabilities = new DesiredCapabilities();
  728. capabilities.setCapability(CapabilityType.BROWSER_NAME, browser);
  729. capabilities.setCapability(CapabilityType.VERSION, version);
  730. capabilities.setCapability(CapabilityType.PLATFORM, os);
  731. this.driver = new RemoteWebDriver(
  732. new URL("http://sauceUsername:sauceAccessKey@ondemand.saucelabs.com:80/wd/hub"), capabilities);
  733. }
  734. @Test
  735. public void webDriver() throws Exception {
  736. driver.get("http://www.amazon.com/");
  737. assertEquals("Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more", driver.getTitle());
  738. }
  739. @After
  740. public void tearDown() throws Exception {
  741. driver.quit();
  742. }
  743. }
  744. ```
  745. As shown above (and as included in the sample project) only one
  746. platform is returned, so only that one test will be run in
  747. parallel. Let's fix that! Add a few more platforms or browser versions
  748. (you might need to refer to [the Selenium
  749. `org.openqa.selenium.Platform`
  750. documentation](http://selenium.googlecode.com/git/docs/api/java/index.html?org/openqa/selenium/Platform.html)
  751. to specify other platforms). Now, when you run the
  752. tests, you should see these tests running in
  753. parallel on the [Sauce Labs tests page](https://saucelabs.com/tests).
  754. ### Setting a parallelism limit
  755. To stop tests from timing out when you're already using all your Sauce Labs parallel slots, we need to limit the number of threads.
  756. The Sauce Labs Parallelized JUnit runner we used above uses the `junit.parallel.threads` System property to control how many threads it runs. Let's set this to 2, to match the limit for free accounts:
  757. ```bash
  758. mvn test -Djunit.parallel.threads=2
  759. ```
  760. ### Parallel Tests in TestNG
  761. TestNG has built in support for running tests in parallel that is configured by the following line in the
  762. `src\test\resources\xml\testng.xml` file:
  763. ```xml
  764. <suite name="ParallelTests" verbose="5" parallel="methods" thread-count="10">
  765. ```
  766. Don't forget to match the `thread-count` to your concurrency limit, as mentioned above.
  767. For more information about the options available for running parallel tests using TestNG, see the
  768. [TestNG website](http://testng.org/doc/documentation-main.html#parallel-running)
  769. Next Steps
  770. ---
  771. Parallelizing tests will make them run significantly faster. It is
  772. only a little bit of work to parallelize them and it lets you test
  773. your code for deployment much more quickly. Use parallelization to run
  774. the same test across many browsers and platforms at once, or just to
  775. run many tests that are independent simultaneously.
  776. You're almost done! We have covered all the major functionality. Now,
  777. we'll give you a few tips about how to get the best performance out of
  778. Selenium and Sauce Labs.
  779. ## Tips for Better Selenium Test Performance
  780. This section discusses some ideas for improving the performance of Selenium tests.
  781. ## Avoid Test Dependencies
  782. Dependencies between tests prevent tests from being able to run in parallel. And running tests in parallel is by far
  783. the best way to speed up the execution of your entire test suite. It's much easier to add a virtual machine than to
  784. try to figure out how to squeeze out another second of performance from a single test.
  785. What are dependencies? Imagine if we had a test suite with these two tests:
  786. ```java
  787. @Test
  788. public void testLogin()
  789. {
  790. // do some stuff to trigger a login
  791. assertEquals("My Logged In Page", driver.getTitle());
  792. }
  793. @Test
  794. public void testUserOnlyFunctionality()
  795. {
  796. driver.findElement(By.id("userOnlyButton")).click();
  797. assertEquals("Result of clicking userOnlyButton", driver.findElement(By.id("some_result")));
  798. }
  799. ```
  800. The `testLogin()` method in the first function's pseudocode triggers the browser to log in
  801. and asserts that the login was successful. The second test clicks a button on the
  802. logged-in page and asserts that a certain result occurred.
  803. This test class works fine as long as the tests run in order. But the
  804. assumption the second test makes (that we are already logged in) creates a
  805. dependency on the first test. If these tests run at the same time, or if the
  806. second one runs before the first test, the browser's cookies will
  807. not allow Selenium to access the logged-in page and the second test fails.
  808. The right way to remove these dependencies is to make sure that each test can
  809. run completely independently of the other. Let's fix the example above so
  810. there are no dependencies:
  811. ```java
  812. private void doLogin()
  813. {
  814. // do some stuff to trigger a login
  815. assertEquals("My Logged In Page", driver.getTitle());
  816. }
  817. @Test
  818. public void testLogin()
  819. {
  820. doLogin();
  821. }
  822. @Test
  823. public void testUserOnlyFunctionality()
  824. {
  825. doLogin();
  826. driver.findElement(By.id("userOnlyButton")).click();
  827. assertEquals("Result of clicking userOnlyButton", driver.findElement(By.id("some_result")));
  828. }
  829. ```
  830. The main point is that it's dangerous to assume any state whatsoever when
  831. developing tests for your app. Instead, find ways to quickly generate
  832. desired states for individual tests the way we did in the `doLogin()` method above
  833. -- it generates a logged-in state instead of assuming it.
  834. You might even want to develop an API for the development and test versions of your app
  835. that provides URL shortcuts that generate common states. For example,
  836. a URL that's only available in test that creates a random user account and
  837. logs it in automatically.
  838. ## Don't Use Brittle Locators
  839. WebDriver provides a number of
  840. [locator strategies](http://code.google.com/p/selenium/wiki/JsonWireProtocol#/session/:sessionId/element) for accessing
  841. elements on a webpage.
  842. It's tempting to use complex XPath expressions like `//body/div/div/*[@class="someClass"]`
  843. or CSS selectors like `#content .wrapper .main`. While these might work when
  844. you are developing your tests, they will almost certainly break when you make
  845. unrelated refactoring changes to your HTML output.
  846. Instead, use sensible semantics for CSS IDs and form element names and try to
  847. restrict yourself to using `By.id()` or `By.name()`. This makes it
  848. much less likely that you'll inadvertently break your page by shuffling some
  849. lines of code around.
  850. ## Use WebDriverWait
  851. Selenium doesn't know when it's supposed to wait
  852. before executing the next command. When users click a submit button,
  853. they know not to try another action until the next page
  854. loads. Selenium, however, immediately executes the next command. If
  855. the next command is part of an assertion in your code the assertion may
  856. fail, even though if Selenium had waited a little bit longer the assertion would have succeeded.
  857. The solution is to use
  858. [WebDriverWait](http://selenium.googlecode.com/svn/trunk/docs/api/java/org/openqa/selenium/support/ui/WebDriverWait.html),
  859. which in conjunction with an ExpectedConditions instance
  860. will wait until the expected condition is found before continuing to your next line in the test as shown below:
  861. ```java
  862. WebDriverWait wait = new WebDriverWait(driver, 5); // wait for a maximum of 5 seconds
  863. wait.until(ExpectedConditions.presenceOfElementLocated(By.id("pg")));
  864. ```
  865. Using WebDriverWait is a great way to make tests less brittle and more
  866. accepting of differences in network speeds, surges in traffic, and other challenges in the test environment.