Archive for January, 2006

I'm in charge

Monday, January 30th, 2006

(evil laugh)

I’m the QA lead for Firefox 1.5.0.2. We had a meeting today to discuss whether we should rotate the release lead responsibilities through the whole group, or just amongst the folks focused on testing the releases. For 1.5.0.2, we’re going to try the whole-group approach.

I have a few goals for my term as qa release lead:

  • Leave the campground cleaner than when I got there. Also known as process improvement.
  • Gain better understanding of where I as a toolsmith should focus my efforts by finding where the pain points are in our current release testing process.
  • Put my organizational system to the test as I switch between release tasks and development test infrastructure projects.

Suggestions welcome.

Bug in first check-in, first approval

Monday, January 30th, 2006

Of course. I’m not immune. Christian Biesinger reported that my change broke the tests on linux. He wrote a fix, and I reviewed and approved it.

So, the first bug reported against my code, and my first review. Lots of firsts. This is cool.

I incorporated the fix into the patch I’m hoping to land on the Firefox 2 branch.

I’m also very impressed that someone is running the tests!

First Check-In

Friday, January 27th, 2006

Small steps. One of many. I hope.

As part of the overall test harness effort, I checked in the patch for bug 322894 today, refactoring the necko and content test harness code into a common location.

Must everything be in XML

Thursday, January 26th, 2006

I’m looking at a test harness written for Firefox (I’ll link to it after I’ve looked through the code a bit more). One thing that struck me is that the javascript test code needs to be embedded in an XML file.

I’ve seen this kind of thing before – making it harder to write tests so that it is easier to run tests. I’m not sure why it seems so common an occurrence to me, and perhaps in this case it is the right trade-off to make, but I would much rather optimize for maintaining test case data (including the oracle code).

Transportation Mechanism

Thursday, January 26th, 2006

My wife told me that we had too many chocolate chips around the house, so she decided to make cookies to bring to the mandatory pre-school meeting tonight. So the cookies are merely a transportation mechanism for the chips.

As long as she does not let my 4.5 year old taste the espresso chips, I’ll be fine putting the kids to bed.

A lot going on

Tuesday, January 24th, 2006

Home: My daughter broke her wrist, or as she would say, sustained a non-displaced fracture of the radius. Off to the store to purchase Dragon NaturallySpeaking, so she could do her essays and thesis in the next two weeks.

I also ran some quick plots using gnuplot and aquaterm for her math homework. Upon reflection, that was kind of fun.

Work: Firefox 1.5.0.1 release is coming up, so most of the qa folks are busy. I’m working on the python code to drive jssh, and getting the simple xpcshell harness checked in. I’m also researching topics for conference papers and following up on leads for help from the community.

Which is to say, I better start scheduling time to blog.

Simplification? Add some tests

Sunday, January 22nd, 2006

Ben Goodger writes about simplifying code in firefox to increase the general maintainability of the code. This is a great opportunity to add some tests to increase confidence in this refactoring.

I hope Ben and everyone else involved in this type of code change adds in tests. First, feel free to ask me for help. I’d love to jump in and write tests.

I’m tracking two test harnesses – one that uses xpcshell and one that uses the full browser. Details of the xpcshell harness project are here, and details on the other harness will be on the Software Testing wiki page soon.

Thanks, Grig!

Saturday, January 21st, 2006

Grig Gheorghiu has been a huge help, cleaning up and adding to the python code to drive a browser via jssh. The code now conforms to python naming and spacing conventions, and there is a module that loops over a collection of files to test the conversion of html to innerHTML.

I handed the current set of files over to the programmer who originally asked for this utility, and I’m working on the result reporting format and tweaking the test to make it more robust.

Hopefully, this code will go into the mozilla cvs repository soon, and lots of folks will add individual test cases for this aspect of the layout engine.

I learn new programming languages best when I have a real problem to solve, and a set of examples to study. So thanks, Grig, for helping speed my learning of python.

A good afternoon's work

Tuesday, January 17th, 2006

I’ve spent most of the afternoon working on the python code to drive a browser for jssh. After yesterday’s spike, I wanted to create a simple class to eliminate the duplication in jssh_driver.py.

But I also wanted to write tests for the driver class. And I did not want to have to run a live jssh-enabled browser in order to run the tests. So my first task became using a test double for telnetlib that would respond enough like jssh that I could write tests.

So I wrote the test double using TDD. And learned more python details. The whole file is at the end of this post.

I’m sure I’m missing out on some basic python coding practices that everybody knows (except me), so if anyone is reading this and wants to criticize the code, please go ahead.

#!/usr/bin/env python
import unittest
class Telnet:
    def init(self, host=None, port=0):
        """Constructor.

    Description of ctor

    """
    self.host = host
    self.port = port
    self.sent_commands = 1
    self.output_buffer = "\n> "

def write(self, command_string):
    self.sent_commands += 1
    self.output_buffer += command_string
    if command_string[0:5] == "print":
        self.output_buffer += "[26]<head></head><body></body>\n"
    if command_string[0:5] == "quit(":
        return
    if command_string[0:5] == "exit(":
        return
    self.output_buffer += "> "

def read_until(self, until_string):
    if self.sent_commands < 1:
        raise Telnet, "read without pending output"
    output_string = self.output_buffer
    self.output_buffer = ""
    self.sent_commands -= 1
    return output_string

def read_all(self):
    output_string = self.output_buffer
    self.output_buffer = ""
    self.sent_commands = 0
    self.host = None
    self.port = 0
    return output_string

def close(self):
    self.read_all()

def open(self,host,port=0):
    self.close()
    self.__init__(host,port)

def get_sock(self):
    return self.host

class jssh_mock_Tests(unittest.TestCase):

def setUp(self):
    self.tn = Telnet('localhost',9997)
    self.tn.read_until("\n> ")

def testTelnet(self):
    self.failUnless((self.tn.host == 'localhost')
                    and (self.tn.port == 9997))

def testWrite(self):
    self.tn.write("foo\n")
    self.failUnless(self.tn.sent_commands == 1)
    self.failUnless(self.tn.output_buffer == "foo\n> ")
    self.failUnless(self.tn.read_until("\n> ") == "foo\n> ")

def testPrint(self):
    self.tn.write("print(document.documentElement.innerHTML)\n")
    response = self.tn.read_until("\n>")
    self.failUnless(response == "print(document.documentElement.innerHTML)\n[26]<head></head><body></body>\n> ")

def testRead_all(self):
    self.tn.write("quit()\n")
    response = self.tn.read_all()
    self.failUnless(self.tn.sent_commands == 0)
    self.failUnless(self.tn.output_buffer == "")
    self.failUnless(response == "quit()\n")

def testClose(self):
    self.tn.close()
    self.failUnless(self.tn.host == None)
    self.failUnless(self.tn.port == 0)

def testOpen(self):
    self.tn.open('foobar')
    self.failUnless(self.tn.host == "foobar")
    self.failUnless(self.tn.port == 0)

def testGetSock(self):
    self.failUnless(self.tn.get_sock())
    self.tn.close()
    self.failUnless(not self.tn.get_sock())

def main(): unittest.main()

if name == 'main': main()

Python driver for jssh-based test harness, initial spike

Saturday, January 14th, 2006

Might as well get some blog postings out of this effort.

Here’s the python code I wrote as a spike for controlling Firefox via jssh. The goal is to develop a harness that will load a html file into the browser, extract innerHTML from the document object, and compare the innerHTML to a golden master. The html files and their golden master innerHTML equivalents will be in a directory somewhere.

I see a lot of duplication ripe for refactoring. But first, I need to read up on python syntax a bit more.

#!/usr/bin/env python
import sys
import telnetlib
import re

tn = telnetlib.Telnet('localhost',9997) COMMAND_PROMPT = '\n> '

jssh_response = tn.read_until(COMMAND_PROMPT)

tn.write('setProtocol(\'synchronous\')' + "\n") jssh_response = tn.read_until(COMMAND_PROMPT)

tn.write('var browser = getWindows()[0].getBrowser()' + "\n") jssh_response = tn.read_until(COMMAND_PROMPT)

tn.write ('var document = browser.contentDocument' + "\n") jssh_response = tn.read_until(COMMAND_PROMPT)

tn.write ('var window = browser.contentWindow' + "\n") jssh_response = tn.read_until(COMMAND_PROMPT)

tn.write ('print(document.documentElement.innerHTML)' + "\n") jssh_response = tn.read_until(COMMAND_PROMPT)

m = re.compile(r"[(?P<len>\d+)](?P<rest>.*)", re.DOTALL).match(jssh_response)

print ">>" + m.group('rest')[0:int(m.group('len'))] + "<<\n"

tn.write ('browser.loadURI("http://www.mozilla.com")' + "\n") jssh_response = tn.read_until(COMMAND_PROMPT)

tn.write ('var document = browser.contentDocument' + "\n") jssh_response = tn.read_until(COMMAND_PROMPT)

tn.write ('print(document.documentElement.innerHTML)' + "\n") jssh_response = tn.read_until(COMMAND_PROMPT)

m = re.compile(r"[(?P<len>\d+)](?P<rest>.*)", re.DOTALL).match(jssh_response)

print ">>" + m.group('rest')[0:int(m.group('len'))] + "<<\n"

tn.write ('quit()' + "\n")

jssh_response = tn.read_all()