Posted by: cmtalbert | November 9, 2012

Boot2Gecko Work Week Automation Wrap Up

This week the A*Team came together as part of the larger Boot2Gecko work week in SF. We worked with a bunch of people across various teams and got a bunch of stuff landed.

TBPL

We’ve been working toward automation on TBPL for a while. This week we worked with Aki from releng to get mochitest, reftest, and emulator-based WebAPI tests running on Mozilla-Central, Try, Services-Central, Mozilla-Inbound, Fx-Team, Cedar, and Ash. We are set to add them to Mozilla-Aurora on Monday.

  • Cedar is our staging area for mozilla-central. If you would like to turn on a new set of tests for mochitest or reftest, you can land that change here to see what happens. (If you break something, please back out)
  • Ash is our staging area for mozilla-aurora. Here too, you can land things to help expand the automation destined for aurora. (If you break something please back out).
  • Try support does exist, but the try chooser is not yet updated, so to run all B2G builds on try, use this try chooser syntax: try: -b o -p ics_armv7a_gecko -t none -u all

Gaia tests/QA Support

The Stephen Donner’s Web QA team is working on automating the Gaia smoketests. This week we pivoted hard to get those stood up in CI. We have them running in our Jenkins instance on an Unagi device and reporting to Autolog.

We met with Geo’s Web API team and made some fixes to the B2G mochitest suite so that they are unblocked and can continue creating webAPI mochitests.

Rob Wood and Dave Hunt continue working closely with the QA team to help them generate more tests. This week, Rob added ten more webSMS  emulator tests alone.

PandaBoards

Pandaboards are our on-change solution for automation that needs to run on device, like the Gaia automated smoketests. There are many moving parts to stand these up at scale. Here is a run down of what happened this week:

  • Thomas Zimmermann worked with us and we now have a more reliable and stable pandaboard kernel.
  • Chris Atlee from releng got our buildbot-provided panda builds to include the proper version of Gaia as well as this updated kernel which is going to be critical to the testing we’ll do next week.
  • We have rolled out this new build to our test boards in IT’s colo, and are troubleshooting some networking issues now. Once those are resolved, we will be clear to start testing our pandaboard automation.

Eideticker Performance System

This week, Will Lachance got all the existing Eideticker benchmarks running using a pandaboard hooked to an Eideticker system. He has uploaded a screencast of the tasks.js panning test being performed on B2G here.

Next Steps

We’re not done yet. Here are the things we’re working on next:

  • Turn on B2G reftest and mochitest on Aurora (target: Monday, November 12)
  • Perform flash testing to ensure we can flash pandaboards per push (target: November )
  • Create B2G specific Eideticker tests to measure specific B2G performance point (target: next week, by Nov 16)
  • Fix the QA Automated Smoketest reliability issues (target: next week, by Nov 16)
  • Add xpcshell to our B2G automated test suite (target: November 23)
  • Get pandaboards running Gaia Smoketest and Gaia Integration tests per checkin (target: optimistically, end of November)
  • Continue expanding the set of mochitest, reftest, and xpcshell tests that we are running in the B2G test automation (ongoing)
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: