Yesterday I upgraded LineageOS from 14.1 to 15.1, and TWRP from 3.1.1.0 to 3.2.1.0. It was a lot harder than it should have been, because I was missing a few details on how it all works.

  1. The phone has a few “modes” -

    • booted to the regular OS (system mode)

    • booted to recovery — this is like an alternate OS (think dual boot) on the phone rather than a boot loader or single user mode

    • booted to fastboot — this is like stopping the boot in the bootloader, before the bootloader proceeds into an OS.

  2. The phone has more than one vendor/product identifier for USB. It uses one for system and recovery modes and one for fastboot mode. As far as I can tell, you never see both at once. The udev rules (
    /etc/udev/rules.d/51-android.rules,
    in my case) had to be updated for both. (Extra note to self:


    udevadm control -R

    to make udevd reload rules.)

  3. There are a lot of flash partitions. I really don’t know what all of them are for, but one interesting one is the vendor partition that needs a “vendor.img” to go in it. You can get a “vendor.img” from a stock image (say from Google) by unpacking the vendor update. It will be called “vendor.img” all ready for flashing into the vendor partition. When I first booted into LineageOS 15.1 there was a message


    A vendor image mismatch has been
    detected. Typically this means your
    vendor image is out of date. Please
    ensure your vendor image matches OPM6.171019.030.B1.

    Later boots only showed:


    SYSTEM UI HAS STOPPED

    And that big fat message was plunked right in the middle of the screen, obscuring the <power on> and <power off> icons whenever I tried to reboot … a tad inconvenient.

    Eventually I remembered the earlier message (fortunately had noted it down), flashed the appropriate vendor.img (I was able to do that in a separate step, flashing just the vendor partition), and LineageOS booted properly. Phew.

    In hindsight, I could probably have added the vendor image into the LineageOS update (see next point on how that might work).

  4. One more problem I ran into: with all the mistakes I was making, my phone lost its “device” property. That meant that attempts to install LineageOS were failing with an error (yet another one …):


    Starting ADB sidelaod feature...
    Installing zip file '/sideload/package.zip'
    Warning: No file_contexts
    E3004: This package is for device: bullhead; this device is .
    Updater process ended with ERROR: 7

    The device type check is made in an install script that is packaged into the upgrade package. The upgrade package is delivered as a .zip file, which is really a “java archive file” (according to the “file” command). It contains a few files in a directory hierarchy, including


    META-INF/com/google/android/updater-script

    Edit this file to remove the first line, which contains the failing check.


    getprop("ro.product.device") == "N5100" || abort("This package is for \" N5100\" devices; this is a \"" + getprop("ro.product.device") + "\".");

    This is an example I got off the web, not the line I actually deleted — mine had “bullhead” where it says “N5100” above.

    Obviously, you want to be very sure you have the right package before you flash it. This is also an example of why it is better to install the prepared and released packages from LineageOS or Google or the phone vendor, than to flash things straight into the flash partitions with dd — they have these safeguards against shooting yourself in the foot.

    Of course, once the package was unpacked and that file edited, it had to put it back together.

    Take the package apart

    Open the ROM zip file with winrar. (I used unzip.)


    mkdir work
    cd work
    cp ../lineage-15.1-20180618-nightly-bullhead-signed.zip .
    unzip lineage-15.1-20180618-nightly-bullhead-signed.zip

    This produced the following directory structure:


    ./system.patch.dat
    ./boot.img
    ./file_contexts.bin
    ./system/build.prop
    ./system.new.dat.br
    ./system.transfer.list
    ./install/bin/backuptool.sh
    ./install/bin/otasigcheck.sh
    ./install/bin/backuptool.functions
    ./lineage-15.1-20180618-nightly-bullhead-signed.zip
    ./META-INF/com/google/android/updater-script
    ./META-INF/com/google/android/update-binary
    ./META-INF/com/android/metadata
    ./META-INF/com/android/otacert
    ./META-INF/org/lineageos/releasekey

     

    Edit the updater-script

    I edited META-INF/com/google/android/updater-script with a plain-text editor and removed the first line.

    Put the package back together

    Then to put it back together again, from directory containing META_INF:


    rm lineage-15.1-20180618-nightly-bullhead-signed.zip # I could do this because it was a copy, still had original in directory above)
    zip -rZ store lineage-15.1-20180618-nightly-bullhead-signed.zip * # I deleted lineage because I wanted to use * here

    My first attempt omitted the -r, and failed to apply (since all the subdirs in the .zip file were empty). Sigh.

    One last note: the file utility reported the type of the package before being taken apart as a java archive file. After reassembly, file said it was a zip archive. It seemed to work anyway.

    Hmm. I seem to remember unpacking that second zip-file … and it was full of .img files as well as the updater script. I’ll just leave this here as a note for next time. Also, maybe if I had zipped back up in two stages, file might have recognized the final package as a jar file. To be continued : -)

Extra research

In all my research to fix the problems I had made, I found a couple of other potential ways to get stuff into flashes. So I never got to use these, but they looked useful. I record them here for future reference.

  1. You can convert a .img into a .zip for purposes of flashing, by renaming the .img file after the flash partition into which it should be flashed (eg, recovery.img) and zipping it by itself with NO compression — (use the “store” compression method). This is handy if you have an img file to install but only the sideload method will currently work for you (due to various mistakes you may or may not have made up to this point, eliminating all the other methods that might work, and/or misunderstandings about fastboot that make you think it won’t work for you).


    zip -Z store newzipfilename.zip recovery.img
  2. Figure out the flash partition device, and dd the .img file into the flash partition. eg:


    dd if=/sdcard/recovery.img of=/dev/block/platform/soc.0/f9824900.sdhci/by-name/recovery

    Obviously you have to look at your own /dev/block/platform directory to get the exact path. Mine contains (in system mode):


    bullhead:/ # ls /dev/block/platform/soc.0/f9824900.sdhci/by-name
    DDR cmnlib fsg keymasterbak modem persistent sbl1 tz
    aboot cmnlibbak grow keystore modemst1 pmic sbl1bak tzbak
    abootbak config hyp laf modemst2 pmicbak sdi userdata
    apdp devinfo hypbak limits msadp recovery sec vendor
    boot dpo imgdata metadata oem rpm ssd
    cache fsc keymaster misc persist rpmbak system

    (Well sorry about the bad formatting … one day I will revisit and format this properly.)

  3. Use TWRP’s flash_image utility from TWRP’s shell. E.g.


    flash_image misc /mnt/sdcard/_update/misc.img

For these last two, you need to put the .img file onto the phone (perhaps using adb) and then run the appropriate command on the phone, perhaps in the shell supplied by TWRP. Be careful with these — it is probably best to flash recovery while booted to system, and to flash system partitions while booted to recovery. Not least because you know you have something you can still boot to, if the flash operation goes wrong. This is why I did not continue to try upgrading the TWRP partition after the phone would not boot to system any more. Once system was working again I was able to flash TWRP (with all my newfound knowledge above) and it was super quick and successful.

Final notes

So I hope I will find this handy, next time I need to upgrade the OS, the recovery, and/or just install new software on or transfer files to/from the phone.

One last note, the xda site seems to be a fountain of knowledge on working with android devices of all kinds (hardware and os-distro), even if you do have to read a lot of individual problem reports and solution attempts to get those nuggets. This page was especially useful: xda beginners how-to guides.

Well that was my weekend … at least I got a beefy blog article out of it : -)