escape(window): blog

A lot's happened since mozharness 0.4 landed in late September. We:

• added a bunch of Android native support.
• enabled Android single locale repacks with nightly update snippet generation
• ported Android signing with passphrase verification, signature verification, single locale support, and update snippet creation.
• rolled peptest out to production on Try, across all desktop test platforms.
• improved mozharness virtualenv support, with real-life mozbase usage in peptest.
• fixed actions-in-config-files.
• separated the output parser from ShellMixin.run_command(), so we can
• parse output from subprocess or get_output_from_command(),
• eventually add context lines to output parsing, and
• potentially split serial tasks into multiple parallel jobs with their own log parsing.
• added chunking support to split up jobs across machines.
• added query_exe() and which() support to specify or find executables.
• added add_failure(), query_failure(), and summarize_success_count() support to track granular status across a list of tasks.
• added a BuildbotMixin and ReleaseMixin to tie into our existing buildbot infrastructure/configs.
• precompiled the error_list regexes.
• made various OSMixin and ShellMixin method improvements.
• added a setup.py.
• added a pyflakes call and Debian/Ubuntu support to unit.sh.
• moved mozilla-specific modules into mozharness.mozilla.*.
• retired Maemo scripts (Maemo tier 3).

This feels like an as-good-as-any time to arbitrarily increment the arbitrary version number: mozharness 0.5.

I think the best part of this release is how more people got involved; I can feel the momentum building. I certainly don't want to wait another 5 months before the next arbitrary version bump.

Today we moved mobile to mobile/xul in mozilla-central, in preparation for landing the native UI code in mobile/android.
Mozilla-aurora, mozilla-beta, and mozilla-release are currently unaffected.

NOTE: mobile/android has not yet landed. The builds are switched over and will need to be hidden. I'm doing that as I see them, but would appreciate any help in this matter.

The mobile team has said that mobile/android is at least a few days away from landing.

Attention project branch owners! If you have Android builds on your project branch:

1. You will need to merge from mozilla-central for the 'android-xul' builds to go green.
2. The 'android' builds will burn until mobile/android lands and you re-merge from mozilla-central. These builds need to be hidden.

Attention trychooser users! You probably want the 'android-xul' platform. The mozconfig situation will be a little convoluted for now, so it would be best to land a mozconfig-extra-android-xul at the top of your push to make sure you get the right settings.

Attention l10n users! Due to the move of mobile/locales, we have broken mobile localization on mozilla-central. We will be fixing this and finding a single locale repack solution ASAP.

See also:

bug 701833 - hg move mobile to mobile/xul
bug 701864 - support mobile builds+repacks out of mobile/, mobile/xul/, and mobile/android/

[what is it?]

device_talosrunner.py is a work-in-progress mozharness script that sets up talos and runs it against a device using either the sut or adb protocols.

It's designed to:

• allow anyone with a [supported and rooted] Android device to run talos without building the tree;
• allow someone with access to a staging/production tegra to run a production-like talos run without setting up buildbot;
• at some point, become the official way we run talos on Android in production.

This is similar to, and parallel to, the work on make talos-remote. However, aiui, that is developer-oriented, whereas device_talosrunner.py is targeted towards someone who wants to test pre-built apks (as well as replacing our current production code).

I have used this script to run talos on both a tegra over sut, and an Asus Eee Pad Transformer using adb-over-ip, successfully.

[how do i use it?]

You need:

• python (2.5 - 2.7.x)
• virtualenv
• hg, to clone the various repos
• a copy of my github mozharness repo, on the branch 'talosrunner';
• a rooted, supported Android device that is either running the sutagent, or is attach{ed,able} via adb. The adb calls require busybox.

Once you have those things,

1. Create a config file. I've got a wip tablet config that I use for my Asus Eee Pad Transformer, and a wip tegra config that I use for a staging tegra.
   
   
2. If you're using adb-over-usb, the device needs to be connected to your desktop/laptop. If you're using adb-over-ip or sut, specifying the device_ip should be sufficient.
   
   
3. If this is the first time you're running device_talosrunner.py, you need a virtualenv. Run

   mozharness/scripts/device_talosrunner.py --cfg users/aki/tablet1.py --create-virtualenv

   (replacing the config file path with the path to your config file). This will create a python virtualenv in ./venv and install PyYAML, a Talos dependency, into it.
   
   
4. Run

   mozharness/scripts/device_talosrunner.py --cfg users/aki/tablet1.py

   If the start_python_webserver option is set, the script will start a webserver and point the device at it. Otherwise it'll point the device at whatever talos_webserver is set.
   
   When this script is polished and mature, I definitely want people to be comfortable using it on their personal phones and tablets, but it's still rough around the edges.
   
   This will uninstall Fennec and possibly reboot your device, so please be careful if you're trying this on a device/profile that you haven't backed up.

[what's left to do?]

• I've only tested a handful of talos suites against two devices; for this to be a production replacement, it needs to be able to run reliably against a large pool of devices.
• This relies on a talos patch in bug 688604 for the python webserver; I most likely need to use mozhttpd.
• I've hit issues with both m-c and birch builds, and will most likely need to make changes for the native UI.
• I have an idea which would allow us to find the next available and working tegra, which may reduce red and purple Tegra results; this needs writing and testing. More on that later.

[Endeavor -> Baltra airport]

I woke early Saturday, since I needed to put my checked luggage outside the door of my cabin before breakfast. And since the crew had two hours between our vacating our cabins and the next week's passengers arriving to change the sheets. I decided to give them a bit more time.

The ship has a two week agenda. We saw the south and east islands; the next week's group would see the young, western islands. Lava. Isabela and Fernandina. Flightless cormorants. Between this and the dry/wet season split, I feel like I've seen about a quarter of what there is to see, and I didn't see everything this time around, either.

They played Lindblad Expedition videos as we waited to leave after breakfast... and I want to go on them all. The Antarctic, yes. Seeing the polar bear pressing its face against the ship's porthole to look inside (and hearing Birgit talk about that, the other night) definitely whetted my appetite for the Arctic, before it's gone. The Amazon seemed fun. Baja is close but similarly matches desert and ocean. We didn't get to the Costa Rica portion; it was time to depart.

Our last zodiac ride was a bit sad. I gave Gaby a hug at the dock; all the naturalists were great but I had been with her for some of the most memorable walks.

gaby


a blurry gaby, fernando, aura, celso


sofia, juan carlos, aura

I spotted the last two Giftmas items I needed for the nieces -- bought 'em and then headed into the airport, where there was free wifi. Ahhhh.

[Guayaquil]

Back at Guayaquil, where we were all a bit frazzled. I wanted to go on the city tour despite only having an hour between arriving at the hotel and leaving -- essentially only a half hour in the hotel room to wash up and go out again.

I did enjoy it though, the boardwalk and the colorful old town. Tempered by the sight of malnourished kids trying to sell candy and the knowledge that hailing a taxi on the street was asking to get robbed.

It was nice traveling with only my point and shoot and iPhone camera, after lugging two DSLRs around all week.

Got back, worked out, showered. Headed down to the bar, where many of us were gathered before dinner; it was like the ship, except no soothing announcement from Cindy.

I ate with the nineteen -- those who had gone on the Machu Picchu extension trip before the Galapagos cruise. Now I'm wondering why I hadn't gone, as well; it sounded fun and the group was very tight. I think I was worried about needing more clothes and more camera gear, which was valid. I think limiting myself on this first solo trip might have been good as well.

The prawns were frickin HUGE and seriously tasty.

Afterwards I joined Kristie and Maureen back at the bar and got tipsy on pricy booze and stayed up past my bedtime. It was fun though.

[JFK]

The smooth travels so far? GYE -> JFK not so much.

• Getting called in to have my luggage opened + inspected at GYE, passing by drug sniffing dogs; not being able to communicate well with the Spanish-speaking security staff;
• the screaming babies on the plane all the way to JFK, and I mean screaming. all. the. way;
• noticing that the airline had failed to give me a US customs form after waiting in line, forcing me to double back, find a form, fill it out, and wait in line again;
• US customs stopping me to check my bags again at JFK

all added up to a relieved Aki once I was at my hotel shortly after 11pm.

Staying in Manhattan for a day seems odd, especially after a trip to the Galápagos. I think it was a good end to my vacation, though. A full day to myself, guided only by my iPhone, Yelp, and vague "hm, I want to [eat|see|do] ____".

• a bowl of ramen;
• Adorama. I got to B&H just when they closed, but that's definitely on my list for next time;
• stopped by a Barnes&Noble to pick up The Beak of the Finch, which Alan was trying to read on the plane. It's a fascinating read so far, and I'm only a few chapters in. Much better/more thorough than the Daphne Major documentary we watched, though the documentary whetted my appetite, surely;
• stopped by Guitar Center on a whim;
• a slice of New York pizza;
• Zuccotti Park and #ows;
• a quick walk by the 9/11 tribute grounds;
• pastrami sammich;
• just a general feel for the city that you can only get by walking through it. Bus tours have a different feel.

Google maps tells me a direct route from my hotel to Zuccotti Park was ~4miles. I'm guessing I walked 9-10 miles that day.

I definitely want to spend more time there at some point.

[JFK -> SFO]

The flight home was uneventful. I'm still not all unpacked, nearly a week later.

I'm so glad I brought:

• my cotton carrier; this allowed me to lug around my 100-400mm with ease.
• both my dslrs, though I wish I had brought my 24-105mm instead of my 17-40mm, given how often I was photographing at the 40mm end of that lens. I was using my 7D + 100-400mm between 80-90% of the time. still, being able to switch from telephoto to wide angle near instantaneously helped me capture a lot of shots I wouldn't have gotten otherwise.
• my point and shoot and underwater housing were useful, though other passengers were getting better results with smaller cheaper underwater cameras.

The bulk of my camera accessories (filters, mini tripod, 1.4x extender) were all unused. Speed and not having to fiddle with things are so important when you have limited time on the islands, and when you're traveling with a group and need to keep up.

... So many great memories. I'd recommend this trip in a second. Already thinking about where I want to go next.

[Prince Philip's Steps]

Genovesa was the bird island. The steps led up a cliff wall; at the top was a bunch of leafless trees (dry season) where red-footed boobies nested. Red-footed boobies can, despite their webbed feet, grasp tree branches. The branches give them cover against other birds (frigates and hawks iirc).

red feet, blue beaks

I definitely got my practice in photographing birds in flight this day.

We spent a lot of time looking for owls, and finally spotted one, far in the distance. I managed to get a number of shots at the far end of my 100-400mm, which was the highlight of the morning.

[Snorkeling]

The snorkeling was decent. Our group managed to find Helen's sunglasses that she dropped in the water on the way down from Prince Philip's steps. I think my underwater point-and-shoot skills were the best of the trip, though. Part of it was getting rid of the orange filter; the rest was knowing the settings that worked for me and knowing to get close.

Practice makes perfect, I think.

[Departure briefing]

Afterwards we had our departure briefing, which was sad. I left a tip for the crew+naturalists and donated to the Galápagos Lindblad fund.

[Regatta]

There was a bit of a tradition -- the passengers against the crew, in a kayak race around a zodiac and back. Cindy insisted the night before that the crew never win, but despite our best efforts, the crew came away with first prize. However, there were prizes for the first crew team and the first passenger team, and lots of amusement all around.

[Darwin Bay beach trail]

Finally, we had a wet landing on the beach, the one wet landing where you needed shoes at all, really. I wasn't looking forward to it since I had rubbed my feet raw in my flip-flops the very first day at Cerro Brujo, but I changed into my hiking boots on the beach and all was good.

Birds were nesting all around and flying overhead; more bird-on-the-wing photography! I spent a decent amount of time trying to catch the sunlight behind the white feathers of the swallow-tailed gull.

After hiking over the lava rocks to see the view, we saw gulls mating.

(slideshow)

We woke early, again, to hike to the top of Bartolomé Islet, to see and photograph the iconic Galápagos view.

The snorkeling off the beach of Bartolomé might have been the best of the entire trip. A couple penguins jumped in the water with us, and I saw 3 different white tip reef sharks, one almost close enough to touch.

fish


shark

I'm still very much a bad underwater photographer, but had fun doing it. Some of the others are getting some great shots underwater. I think I'd throw all mine out if they came out like that on land.

After lunch, on the way to Sombrero Chino, we passed by an island with an old crater, and in passing caught a maybe 30 second glimpse of flamingos in the lagoon inside that crater. I managed to get a few not-too-bad-for-being-very-distant shots. It's kind of awesome how they were able to tell us practically to the minute when that view would appear.

I missed the 2nd round of snorkeling, but caught the zodiac ride near sunset, where we saw sea lions playing near the shore. We circled several times, waiting for penguins, but nothing. As it grew dark and we prepared to turn around, a penguin swam up to the lava rocks; we converged; it climbed up and we all were taking a ton of pictures. Then, as we watched, a bunch more penguins swam up and congregated on the rocks. Very, very special.

I cranked the ISO on these last two, as it got dark, so they look brighter than the earlier ones... Can you tell I don't do a lot of editing?

Dinner was an outdoor barbeque where Cindy, our expedition leader, told us we needed to visit South Georgia... Falklands, South Georgia, and the Antarctic peninsula is a full 3 weeks. "Galapagos on steroids" is her description of it.

A bunch more people were out looking for activity in the water; word spreads fast. Wasn't quite the same; headed inside and made the mistake of checking some work email.

(slideshow)

In the morning we went ashore at Cerro Dragon to see land iguanas -- we saw a few, but the highlight was probably both the arid landscape and the flamingos.

Later in the morning we went snorkeling by the cliffside to see invertebrates -- I saw coral, sea urchins, stars, a sea snake, and a sleeping sea lion -- others saw a manta ray, two octopi, and a swimming sea lion that passed right by them. Slightly jealous, but I'll live :)

After lunch Celso gave a presentation on Darwin. I didn't go kayaking, again, but I did go on a zodiac ride... and got some pictures, notably some distorted pictures of a ray.

The "What Darwin never saw" documentary was cool -- essentially, the Galápagos are well suited for noticeable evolution because there are multiple islands that have different climates and ecosystems that favor different traits, and there are fairly regular events where a large percentage of populations (finches, in this case) die out, favoring the traits that are best adapted for that island's ecology.

Champagne on the bow, though it got dark before we could circumnavigate Daphne Major. After dinner, however, while I was talking with Betty about which five photos I should submit to the slideshow, Helen called us out to the deck -- there were sea lions and sharks in the water, drawn to the light. We hung out for a long while out there, watching the dark forms of sea lions hunt and the forms of sharks swimming through the water, some turtles too. That was cool.

(Slideshow)

We went ashore at Puerto Ayora, the economic hub of the islands, where we went to the Charles Darwin Research Station to see their tortoise breeding program. So many cute tortoises!

There are tortoise species specific to each island, evolved to match the food supply. At the CDRS they breed the tortoises and bring them back to each island when they're old enough to survive on their own.

We got to see Lonesome George, who may be the last Pinta tortoise.

They also had some Galápagos land iguanas.

A few of the naturalists were able to visit their families, which was good to see. The walk to town was fun; I picked up some presents for the family. Then we took a bus to the highlands.

The sugar cane mill was a nice diversion. I quite enjoyed lunch at a highlands restaurant called Altair. They had an owl nesting in a building in their backyard; when I went to photograph it, everyone handed me their camera to take a picture. That was fun, though I learned I don't have a clue how to navigate around manual mode on a Nikon.

The pit craters weren't particularly interesting to me, since I'm not a huge botanist, and the photo opportunities were slim. The walk through the tortoise migratory route was pretty amazing, though, since there were dozens of tortoises, in the wild, along the way.

During that walk we also walked into a lava tube. We also spotted a rare vermillion flycatcher.

We picked up some high schoolers who had won a school contest, and get to ride with us on the ship. They had never been on a boat and have never snorkeled; I learned second hand that most of the locals aren't allowed into most of the national park areas at all (97% of the islands). I do think that showing them what they're preserving will help local conservation efforts.

(slideshow)

Up at 5:45, 15 minutes before the optional wakeup call for the wet landing at Point Cormorant.

This was a sea turtle nesting spot. The nesting mostly happens at night, so we caught sight of a sea turtle in the surf, but none on land. The salt water lagoon was low on water, which meant no shrimp, which meant no flamingos.

dry season

I skipped the quick Zodiac ride around the islands, but went for the deep water snorkeling at Champion Islet.

When we jumped overboard, the water was cool, again, but what first struck me was the hundreds and hundreds of fish swimming maybe ten feet below us.

We swam along the rock wall to our right, with the ocean floor going deep below us to the left, and saw many fish, some starfish, and thousands of spiny sea urchins. The highlight was probably the diving and somersaulting young sea lion; Celso, our naturalist, somersaulted with it for a little while, which was fun to watch. I wish I had thought of getting video of that.

(Here, again, unfamiliarity with the camera bit me; I took some video of some fish, or thought I did. Afterwards I realized I had *started* recording when I thought I was *stopping* the recording, so my two underwater videos were of odd angles of nothing particularly interesting, as I swam around looking for more fish. Good thing it was only my underwater point-n-shoot that this was true for.)

After lunch, a nap sounded more appealing than kayaking + zodiac ride + the post office barrel, so I dozed off for the afternoon. Later I found that they had spotted penguins [!] though not terribly close and against the black lava background. Hopefully I'll be able to catch some penguins later in the week. (*me, reading this now: [!] :)

Birgit's post-dinner crittercam presentation was pretty cool. We saw clips of emperor penguins hunting, sharks hunting, seals and sea lions capturing and eating octopi, whales feeding, sea turtles looking for mates, and even a cat catching and eating rats, all from their points of view.

Tomorrow looks amazing, at Santa Cruz -- the Tortoise Rearing Center, the Charles Darwin Research Center, a sugar cane mill, a highlands forest, and some local shops during a full day out all sound wonderful.

(slideshow)