The first Black Ops Testing Webinar Debrief Notes

The webinar has finished, but you can watch a replay of the recording.

During the webinar we showed a bunch of videos and our notes for the sessions.

We will link to, or include, all those resources here.

Outline

Steve's Notes

Consider how you would test http://blackopstesting.com/

Identify the context * Scope (what features, content and types of testing are in and out of scope?) * Time (how long have I got?) * Risk (what really matters?) * Rationale (why are we testing, and who for?)

Testing initial ideas * Input constraint and data validation * Mandatory fields * Email address format enforced? * Is there a maxlength? * Client-side or server-side validation? * Can we enter multiple email addresses with delimiters? * Already-registered email address * Whitespace * Leading and trailing whitespace. * Accessibility

Testing * Input constraint and data validation * Mandatory? Yes, there is an error message if it no email address is entered. * All validation is done server-side. * Email address format enforced? Yes, but not correctly. * BUG - [email protected] is accepted (domains must be at least 2 characters) * INTERESTING - there is a banned list e.g. [email protected]

    * Is there a maxlength? Not client-side. The server enforces a 100-character limit.
        *BUG – the server says the email address must be less than 100 characters but it accepts 100 characters (but not more).
    * Is there a maximum domain name length? No (subject to the 100-character total limit)
    * Are numbers permitted in the domain name? Yes
    * Are numbers permitted in the top-level domain name? No
        * QUERY - Cannot find a definitive reference for TLD validation
    * Lots of subdomains. Registers ok.
    * Lots of dots in the name. Registers ok.
    * Two @ symbols. Correctly rejected.
    * Can we enter multiple email addresses with delimiters? No, it does not allow more than one @ symbol.

* Already-registered email address
    * Register with an already-registered email address. OK – there is an error message.
    * Register with an already-registered email address followed by a space. OK – there is an error message.
    * Register with an already-registered email address preceded by a space. OK – there is an error message.

* Whitespace
    * Leading and trailing whitespace are stripped out.

* Interesting findings
    * If there is a space in an email address the error page asks if you meant to include a %20 - Cannot replicate
    * BUG - If you put %20 in the name, it is accepted but you do not receive a subscription confirmation request email.

* Accessibility
    * The ? link is not keyboard accessible. The page reloads if the Enter key is pressed while it has focus.
    * The horizontal line is an <h1> heading.
    * The ‘Declassified’ image does not have an ‘alt’ attribute so the link destination is not conveyed to screen reader users (or anyone who cannot see the image).
    * INTERESTING – There is a ‘spam trap’ textbox that is accessible to screen reader users but it has no label so they do not know what it is for.
    * The ‘spam trap’ textbox is on the tab sequence. The focus disappears for no apparent reason.
    * INTERESTING – While entering an email address there are a series of “invalid entry” audible warnings whenever the email address format is not valid. This is not uncommon but it is interesting that the warning is not provided in a visual form for all users.
    * The semantic structure is bollocks. The Breaking News headings appear to be associated with the following image instead of the preceding one.
    * The RSS feed icon does not have an ‘alt’ attribute so it’s purpose is not apparent to screen reader users.
    * The ‘binoculars’ image has a descriptive ‘alt’ attribute but it should have a null one.

* How would you continue to test until it is fixed?
    * Turn off styles in the browser (that’s interesting – you see an extra unlabelled textbox. It appears to be a spam trap because you get the message “Too many subscribe attempts for this email address. Please try again in about 5 minutes. (#8364)” if it contains any characters when the form is submitted)
    * Use Firebug to change the text colour (works fine)
    * Turn off JavaScript (works fine)
    * Test in a text browser
    * Use a screen reader
    * Copy and paste text from Notepad into the textbox
    * Save a local version of the page and edit it
    * Use Fiddler to block the stylesheet and insert a modified one

Alan’s Notes

http://blackopstesting.com/

*Bug*: The email address input field has white text on a white background so you can't see the text you enter as you enter it.

*Better? bug*: css styling makes the text white on a white background in the email address input field so you can't see the text you enter as you type it.

*Automation* - functional automation tests would work fine on this, because they don't need to see, they just find the field, type it in, and good to go. Unless it is an image based tool. In which case they might fail.

*Workarounds*:

1. click on field and ctrl+a to select the text and see it, before you type a range of letters - use memory to fill it in
2. Just click subscribe and the 'alternative' error checked input form comes up
3. Inspect element and amend the css

    1. stop the color from inheriting (chrome, firefox)
    2. change the color on the blackops header

4. I can look at the css itself - a wrong id has been used on the div (in stealth mode - this is the 'real' bug so I could amend the id to 'mc_embed_signup'

    1. I can edit the CSS in the Sources, and work

        1. I could edit and save the CSS locally then put an autoresponder in the proxy to pull the local file


Scope of testing?
Scope of testing - do we want to test mailchimp? or the input form

James's Notes

Coming Soon - james created a couple of videos - we haven’t uploaded those yet

James supplied his notes as a pdf, which you can download here