• Start Firefox with a URL ending with a query parameter like “start=”<current time in ms since EPOC>
• The page that the URL points to does a JavaScript “new Date().getTime();” as part of its onload handler and subtracts that from the value in the “start” query parameter
• The URL prints out the date to the console (because we can do that since we control the browser and the profile)
• Automation reads the console and puts the value in a database

Pretty simple.  Applying this to different browsers, you have to nix the “print to console” idea.  But, how hard could it be to POST to a web service that stuffs your result in a database?  Do that and the rest of it will all “just work”, right?

Well, not really.  Every browser implements the cross-origin access policy to a different degree, and since we did this on android, some of them don’t seem to support it at all.  Once we found a way around that, we realized that not all the data was making it into the database because the automation would kill the browser before it had a chance to POST its results.  So we slowed that down, forcing the automation to wait 20s before closing the browser.  Then our database crashed, this part we had nothing to do with, but Murphy’s law states that you can’t have an automation project without at least one bonfire igniting under your chair.

Add to this cross-browser headache that we’re automating this on multiple phones.  The older Nexus phones (Nexus One and Nexus S) will not stay connected to a wireless network after reboot (appears fixed with Galaxy-Nexus or with ICS, not sure which).  Even if you put these phones on an open network with no contention and they are set to “join automatically”, they will at some point boot into a state with their wireless disabled. We had to write some service code to ensure the wireless remained on and connected to our specific network on boot.  Our other phones (a Droid Pro and a Samsung Galaxy S2) have no problem staying connected to the network, but they alternately “freeze”.  I’m still trying to debug what this “freeze” actually is, but everything is functioning fine on the phone – network, logcat, process list etc are all normal.  However, the phone stops running the automation.  It’s interesting that the Nexus phones never encounter this issue and they are all running the same version of the automation code and browsers.

At long last, we have fought through enough of these issues so that we can start to see the results of the data coming into our database (select “2 months” or “all” to see data).  Because we are merely firing our “timing” function when the “onload” event happens for the page, we can see the different interoperability issues with measuring this event. We knew it wasn’t perfect, but the results we are seeing on Android make me call into question the usefulness of this as a cross-browser comparison tool at all.

• Opera seems to fire the onload event randomly.  I’m not sure what they are doing, but their timing is all over the place.  Note that this could be a fluke in the automation as the Samsung/Droid Pro hang usually occurs during the Opera test (which, by chance, is also the first test).  However, note that the Opera numbers for the Nexus phones are also wild, and they are not afflicted by this unusual hang.

• Dolphin and the stock Android browser are both webkit based browsers and we have always known that webkit tends to fire this event very early in the page-load sequence.  This is reinforced by the fact that the event always happens at roughly the same time regardless of the underlying phone hardware, especially on the stock Android browser

• Fennec – this automation measures the new native Fennec product.  Currently, the system contains results from the beginning of the project to the point at which we moved from the birch tree into the mozilla-central tree.  I have another set of jobs to run that will get us the last two weeks from the mozilla-central tree, once the phones finish their jobs from the previous two months.  Of the four browsers being measured, the only one changing versions is Fennec; therefore, you can see the effect of our developers’ work as they add features and battle regressions.  Native Fennec is still under heavy development, and this is why the Fennec number jumps around as much as it does.

The system is far from perfect.  Measuring onload is at best an artificial metric, and not at all indicative of what the user sees.  In desktop automation, we don’t even use onload, we use the “mozafterpaint” event notification.  For the next stage of the cross-browser test we are going to automate some visual comparison tests to get closer to measuring the metric that really matters: real-life user experience.  In the meantime, the onload tests will continue to give us a rough barometer of our regressions and performance, especially against our own historical data.  To that end, I am going to undertake the next few improvements to this automation:

• Understand what the hang is on the Galaxy S2 and Droid Pro phones and fix it
• Add more phones to the system so that it doesn’t take so long to run through a set of jobs (we only need these temporarily until the system catches up on old data).
• Experiment with lowering the timeout period between “results uploaded” and killing the browser under test. (This might work better now that we have changed database backends).
• Get a better front end UI for the results.  If you’d like to contribute to this, let me know, because this website could sure use your help!

I made oatmeal and coffee, walked the Ru-Dog, and was southbound by 5:30.  The only problem with my plan is that Cowell’s only works at low-ish tides.  And today’s high tide was a six footer which would crest about 10:30.  Since the low tide was at 3AM, I figured that if I got there around 6ish, things would be fine for an hour or two.  It was a great drive down highway one, starlight glittered over the big black waves I could hear crashing through the car windows.

I parked the car and watched the sun come up over the mountains across the bay from Cowell’s.  The waves were perfect.  Nice lines, rolling all the way through Cowell’s, except for one small problem.  They never broke.  They just rolled all the way up to the beach.  I thought about it.  I might could catch one, but more likely, it’d roll right out from under me.  And since the tide was only going up, it was just going to get worse.

Instead, I sipped my lukewarm coffee and watched two amazing surfers tear it up on the overhead-and-a-half at Steamer Lane before deciding to drive back north and check out some of the more exposed breaks along highway one.  So much for my grand plans of spending the morning in Santa Cruz.  All the exposed breaks on highway one were ginormous.  At this point, I was halfway back to Half Moon Bay, so I kept going.  I stopped in Half Moon, stared at the break outside the harbor, shook my head, and kept driving.  I returned to Pacifica about 9AM.

My local break was crowded as hell, with waves larger than I’d have liked.  And the tide was so high there was no beach.  I had to pick my way over stones just to get into the water, but I was determined to at least paddle out before giving up.  I was so frustrated, I forgot to put my hood on before paddling.  About twenty feet in front of a rushing wall of whitewater, I realized my hood was still hanging around my neck.  I duck dove without it, and it somehow managed to un-velcro the top flap of my wet suit.  As I paddled toward the next wave, I pulled the hood on as best I could, but couldn’t deal with the velcro so I got more frigid water down my back.  I managed three more duck dives before finally coming out the other side of the break zone cold and clammy.  I paddled for a bit more to warm myself up, then sat up and fixed my wetsuit.

Because of the tide and my extra paddling, I was now almost fifty feet past the lineup.  Everything had gone wrong this morning.  Was I even going to catch one of these waves with all these other surfers around?  I started paddling back toward the lineup when a dolphin crested six feet in front of my board.  I was close enough to see the creases on its gray skin.  My eyes bugged out of my head.  I could have almost touched it!

It made me think: How many events this morning had to go so perfectly wrong to culminate in that one, perfect moment?

However, week after week as I asked for each project’s status and listened to it, I wondered why on earth would anyone want to come to this?  And why were we spending an hour each week boring ourselves to tears when we could be doing something useful like being silly on IRC? So, the A-team and I talked about it, and we decided to do an experiment with the meeting.  Here’s what we’ve been doing for November:

• One person spends an hour or so a week collecting the status from everyone on the team.
• That person puts together the wiki page.
• At the meeting on Monday, that person is the emcee and does a five minute run down of the week’s highlights.  This is the toughest job.  We have a great team, and there are always a lot of highlights.
• After that, we raise any issues that need raising and discuss them, five to ten minutes.
• The emcee gets to pick the emcee for the following week.
• Then we remind people to check the wiki page for the schedule of project-specific meetings that week, and we’re done.

The entire thing takes no more than twenty minutes, and most weeks it takes less than ten. So far, I have to say I’m a fan of the new meeting.  I worried that I’d lose my ability to stay abreast of what is happening on our projects, but that hasn’t been the case.  In fact, if you compare the wiki pages from before with these new ones, you’ll see that our emcees do an amazing job pulling together the data and communicating the highlights.

The other benefit this gives us is that as we grow into a larger team, it’s harder for all of us to interact.  Our rotating emcee gives each person a chance to talk with everyone else on the team and learn something about everyone’s projects.

I don’t know if this would work well for other teams, but it has worked really well for us so far.  If you’d like to drop in, here’s the information about our meeting.  This week’s emcee is our illustrious maple-bacon-cake-baking, cowboy-boot-wearing intern, Tfair.

I worked at a company called SimDesk that decided to reuse the Thunderbird and Sunbird code bases and make a great email application–this was long before the Lightning extension came into being.  Like any good closed-source company, we stole the code and worked on it in secret until we had a shining example of an “Outlook killer” (well, more or less).

Then we started feeling like we should contribute some of that code back to Mozilla.  We had a bunch of very awkward meetings with Dan Mosedale and Mike Shaver as they tried to teach us how to do open source.  They kept saying, “just submit a patch”, we kept wondering which lawyers we’d have to get involved to do that.

Eventually, Mike Hovis (an old friend and superior developer) and I started writing those patches.  It became clear that our changes wouldn’t apply cleanly to the newly refactored “Lightning” source base.  We decided that I’d make it part of my job (20% of my time, as I recall) to make patches for functionality we cared about and get it to the Mozilla calendar team.

I started attending the calendar team’s public meetings, and during one, when they asked if anyone wanted to lead a calendar QA team, I volunteered.  I had no idea how to actually do this, but I wanted to try organizing online to see if some of my offline organizing skills would translate.  My contribution of time grew.  As SimDesk directed me to work on Outlook extensions rather than an Outlook killer, I spent more and more of my time working with my calendar team, writing patches, mentoring, and aiding volunteers as they found their roles as leaders and developers in the calendar project.

And one day, when I could plainly see the writing on the wall, I asked Dan if Mozilla would actually consider a resume from me.  After his enthusiastic “yes”, I applied, and the rest is history.

Starting in the calendar project was incredible.  It was smaller (of course so was Mozilla in those days–even though it felt huge to me at the time).  It was easier to see your impact in such a small space, easier to identify volunteers, and easier to mentor people through the process and watch them become leaders.

Starting in that small area was also fortuitous because there was so much that needed to be done and opportunities were everywhere.

I still think that there are small areas across Mozilla where people can start and have a similar experience.  However, I think that Mozilla seems so monolithic these days that it is daunting to even try to find those niches where you can start out as a volunteer.  It is up to us on our teams to identify those areas where people can start, publicize them, and help people make that leap from “casually interested party” to “volunteer”.  In that vein, I tried articulating the roles that we’d like to see people step up to fill on my team.  If you’re interested, you know where to find me.

Well, PandaBoards come with nothing, not even a power supply.  They can be powered off USB, but it’s pretty difficult to get adb working in that state (if you have steps, I’d love to hear them).  So, here are the steps to getting something usable working (See the official getting started too):

1. Order power cord, specifically the adapter and the cord
2. Order 8-16Gb SDCard
3. Ensure you have a mini USB cord
4. Ensure you have a CAT 5 network cable.

Once you have this, you can build or download a build onto your SDCard.  Oh yeah, you’ll need an SDCard writer/reader.  Most computers have them by default these days, thankfully.

Then, plug it all in, and it should work.  I’ve noticed a few oddities:

• Our SUTAgent had some difficulties at first, but now it seems to be working fine.  Still debugging this.
• ADB won’t work if the card is plugged in when it boots.  I think this is due to the build, as I seem to recall seeing an issue on it earlier.  I’ll keep researching and will try some different builds to find something more stable.  In the mean time, unplug when you reboot the card, plug in after the card is up and running.  Also, you won’t see the “USB” notification that you usually see in Android.  So, don’t expect that.
• There is something going on with the package manager.  I installed Fennec, but the pm doesn’t list it, and claims that it is not installed.  However, it runs fine, appears in the applications, and can’t be re-installed.  I just can’t uninstall it.  Still investigating that too, and like the other OS level issues, I’m wondering about this downloaded build.

There is a ton of truth in that.  But you also have to be ready to be effective when you show up.  Here’s a real life example from my life today as a case study in what not to do:

• 12 noon: Decide to work on Call of Trees, draft 6. Get out computer.
• 12:30pm: Spend half an hour reading older material that’s not related.
• 12:30pm: Stare at beginning of novel
• 1pm: Attach computer to screen so I can see both the edits from Gabrielle and my draft.  Stare at novel.
• 1:30pm: Go tweet about something cute the dog does
• 2:45pm: Make myself stop reading cool stuff on the internet, realize I’m wasting time.  Return to staring at beginning of novel.
• 3:30pm: Not making any progress, go make lunch and read
• 4:30pm: Stare at the novel more.
• 5:00pm: Realize that due to recent computer crash, I don’t have all my music on new laptop, but I do on this old computer. \o/ Try to move songs from one computer to another.  Stare at novel while gigabytes of music get copied.
• 6:00pm: Quit wasting time re-creating playlists.  Stare at the novel again.
• 6:30pm: Finally open a new document and start playing with re-writing the beginning.
• 7:30pm: Have something of a new beginning drafted.
• 8:00pm: Go watch my neighborhood try to blow up our street in celebration of July 4th.
• 9:00pm: Finish the draft of the new beginning
• 10pm, 11pm, 12am, tweak, tweak, and retweak until I’ve got something that sounds pretty good.
• 12:30am: start this blog post so I’ll remember to never do this again

So the moral of the story?  Re-drafting beginnings sucks.  It’s more fun to go surfing.

On a serious note, opening that blank file and starting to play with the opening sentences is what unblocked me.  Typing in my new “first draft” sentences next to my current “draft 6″ sentences in that existing, giant document was demoralizing. Having a blank scratchpad with nothing but my new sentences in it freed me to throw down crap until I got something good.

And now that I’ve started, I’m pretty excited about this last draft.  After all, the novel’s written, so it’s all downhill from here.  Right?

Right?

Fear is the same.  It can stop you, peg you like a moth to a yellowed piece of styrofoam where your dreams will collect dust, trapped for eternity behind glass.  I’ve seen it happen to several would-be writers.  Their trembling hands won’t hold their papers, their voice won’t carry through the room.  They refuse to read or even be read.  I was once in their ranks too.

Fear cannot be a refuge for the would-be writer.  There is no sanctuary there.  We do not want to see our dreams collecting dust in some forgotten glass case showcasing what “might have been.”  We must take our stories in our hands and speak louder so that the audience won’t hear the trembling in our voices.

This week at the Santa Barbara Writer’s Conference, I’ve watched many timid writers step to the front of the room, plunging into icy waters of critique after critique.  I celebrate them.  We will not be collected into the showcases of the unheard, the unread.  We will speak loudly and follow our dreams.

Being scared just means that you’re doing it right.

Today I found a new trail run in Pacifica.    It’s intense, it’s high, and it’s exceptionally cold.  Sounds like fun, right?  So get on your hat, gloves, leggings, and get ready.  It’s the Sweeney Ridge trail.  I found it by accident when poring over Google Maps trying to plan a bike route in Pacifica that doesn’t involve gear crushing, chain derailing hills (turns out that is impossible, fyi).

So, today, since my topo map told me it was going to be rough, I made an exception and drove to the trail head.  Follow the signs for the Shelldance Orchid gardens off route 1.  The trailhead is in their parking lot.  I’d recommend doing laps in the parking lot or something because you really need a warmup for what’s about to happen.  The trail launches out of that parking lot like a rocket.  It’s damn near straight up for about a mile.  You go from about 250ft above sea level to about 900 before the trail begins to soften into something you would call a “moderate incline”.  Here’s a picture after finally getting up there, looking back at the climb.

Up there, things get easier.  You pass the ruins of the Nike Missile Command center.  It’s a nice break to stop and poke around the buildings.  From there you keep going up along the ridge, cresting at around 1200ft.  Up there, you come to the place where the Spanish Portola expedition first saw the San Francisco Bay.  It has a wonderful view of the bay (as you might expect), but it was really cloudy so I couldn’t get a good picture.

Then you come down near Linda Mar (that’s the shovel shaped bay down in front):You end up on Fassler Ave, and you can make a giant loop out of it for a grand total of 6 miles.  I have dubbed one of my hilly runs here “Escape from Pacifica“.  This one, I think I’ll call “Escape From Pacifica Sudden Death” in homage to that ass kicking beginning. (Turn on the topo display on that map and take a look!)

Also, the GGNRA is thinking of closing Sweeney Ridge to Dogs.  Then there’d be no more trail pictures like this one:

You can make comments about the closures they are proposing here (the proposal covers much of Marin, San Francisco, and San Mateo, so it might be worth your time to take a look).

If you plan to do this run (and I think you should), dress warm.  This is not the wimpy Texan-can’t-deal-with-anything-under-85-degrees talking.  The wind comes up off the ocean and rips along that ridge. It was blowing so hard, it was a struggle to hold the phone still to take pictures.  Dress in layers, take water.  Have fun.

But, today, Mozmill 1.5.3 is released.  It’s already on Pypi, and it was just submitted for review on AMO.

We fixed:

• Bug 643697 – JSBridge limits amount of data transferred (fixed on
  branch, fix still pending on master)
• Bug  645234 – Fix for addon compat prefs (short term).  More changes may be needed as we change our version numbers to accommodate faster releases.
• Bug 636746 – Add Mozmill version to test report
• Bug 646970 – Remove controller argument from _buildMenu call in Menu::open
• Bug 647030 – version bump
• Bug 648523 – Mozmill 1.5.x doesn’t remove temporary addon folders
• Bug 506760 – Mozmill tests executed with UTF-8 characters cannot be executed
• Bug 648523 – Make windows process handling more robust

Thanks very much to the tireless folk who’ve fixed all these bugs and verified them.

Mozmill 1.5.2 is now on pypi, you can easy_install it.  The Mozmill 1.5.2 add-on is currently under review on addons.mozilla.org (AMO), and will be posted live after the AMO editors complete their review.

There are getting to be way too many people to thank (which is an awesome problem to have), but I want to make sure to thank the following folk who definitely went above and beyond the call of duty:

• M.A. Darche
• Merike Sell
• Adrian Kalla
• Henrik Skupin
• Geo Mealer
• Anthony Hughes
• Aaron Train
• Heather Arthur
• Jeff Hammel
• Andrew Halberstadt
• And several others who’ve reviewed patches, looked at downstream bugs, and reported issues

Some of the big bugs that were fixed in this release were:

• Manifest support for easily disabling tests
• Fixes to compatibility checking for addons
• Adding the ability to forward command line parameters to the program under test (like the debugger for instance)
• Several improvements to the stopping/starting code to fix several hang bugs
• Drastically improved support for simulating drag and drop
• You can now get to elements using CSS selectors
• Syntax errors inside tests are now more clearly reported when running from the command line.

The full list of fixed bugs are here: http://bit.ly/e1Oe9f

If you want to get involved and help out with the Mozmill 2.0 effort, it’s not too late.  Stop by #mozmill on IRC (irc.mozilla.org).