Thursday, March 7, 2013

Robot Framework testing hints

Updates:
2013-03-15 Remove external content loaded by javascripts
2013-03-14 Be careful with Element Should Not Be Visible
2013-03-13 Using Chrome driver in RF tests
2013-03-12 Notes about Plone-specificity

Some things that come in handy for Plone developers working on Robot Framework tests:

Tags

RF supports tags. Add a line [Tags] tag1 tag2:
*** Test cases ***

Scenario: Clicking the submit button hides it
  Given i am logged in
    and i am on an article
   When i simulate clicking the comment submit button
   Then the submit button has class disabled

Scenario: Submitting a comment displays it in the page
[Tags] working_on_it
  Given i am logged in
    and i am on an article
   When i type something in the comment box
    and i click the comment submit button
   Then the page shows the comment
You can now run only the latter test: ./bin/test -m der.freitag -t working_on_it (This is Plone-specific. See Asko's comment below.)

"Typing" in TinyMCE

TinyMCE is in an iframe, get to it with Select Frame:
i type something in the comment box
    Select frame  form-widgets-comment-text_ifr
    Input text  tinymce  ${COMMENT}
    Unselect Frame
More useful commands and selectors at http://rtomac.github.com/robotframework-selenium2library/doc/Selenium2Library.html

Using Chrome driver

In order to also test in Google Chrome, get the chromedriver binary from http://code.google.com/p/chromedriver/downloads/list, install it as per http://code.google.com/p/selenium/wiki/ChromeDrive (short version: put the driver on your $PATH and chmod +x), and run your (Plone) tests (on Linux) with:
$ ROBOT_BROWSER=Chrome ./bin/test -m der.freitag -t wip 

Be careful with Element Should Not Be Visible

"Element should not be visible" will also return True (ie. not fail) if the element is not in the page... so probably you'll first want to check if it's there at all.

Remove external content loaded by javascripts

If your site loads external javascripts like ads, these may inject invalid HTML, or open popups, or just make the page load slower. How they behave is unpredictable, which can make creating tests all but impossible. You'll probably want to disable everything that depends on something from the outside.
To do this in Plone, you may create a Generic Setup profile for RF tests that disables the viewlets that load these external scripts. In our testing.py, we now have:
class DerFreitagAcceptanceLayer(PloneSandboxLayer):

    def setUpPloneSite(self, portal):
        applyProfile(portal, 'der.freitag:default')
        applyProfile(portal, 'der.freitag.tests:testing')
        applyProfile(portal, 'der.freitag.tests:robot')
The der.freitag.tests:robot profile lives in tests/robot/profile, it just has a viewlets.xml.
The tests layer DerFreitagAcceptanceLayer is a separate layer created specifically for RF tests. Actually DerFreitagRobotLayer would have been a better name.