f2fs – booting linux on an f2fs flash drive

~ 01micko

Today I had an eureka moment. I installed a full install of Slacko Puppy Linux to an f2fs formatted flash stick, booted and surfed the net. Has anyone done that yet?

The kernel is k3.4.52 patched with f2fs patches from Now Computing. I had to hack them a bit because the developer is only supporting k’s 3.0, 3.2, 3.5 and 3.8. I also patched the kernel for AUFS as we do in Puppy.

The unique problem here is that no bootloader supports f2fs at the time of writing. The work around is to first create a small boot partition in a recognised format such as vFAT, ext2, 3 or 4. Then you can boot the stick with grub (I used grub4dos). You could probably use syslinux, grub2 or even lilo.

In the boot partition you create a /boot directory that contains the kernel image (vmlinuz) and an initrd (in my case initrd,gz). Now in my kernel configuration I have all the relevant drivers that I need as builtins however using an initrd it would work just the same to have hid, usbhid, ehci-hcd, uhci-hcd and f2fs as modules and just load them. In fact, in my init script I have those loading anyway with the error sent to /dev/null. Of course you need a tree with the appropriate modules in the appropriate places, busybox, the necessary mount points and so on.

The real trick was to find the root partition which was on /dev/sdb2, however that would not work, nor would sda2, which probably should have since I wasn’t loading any other filesystems and the only builtin is ext2 (I was using FAT32 on my boot partition). My hard drive is formatted NTFS and ext4. I achieved this by using the UUID in the init script itself. You could just as easy have a config file that init reads.  Once the root filesystem is mounted you then switch root in to the running systems as with any other initrd.

Thanks have to go out to Barry Kauler for integrating f2fs into Puppy Linux infrastructure and for some of the code from his init script. My init script uses some of that but uses a directory structure based on Slackware’s initrd and some of the code from Pat Volkerding’s init script too.

I’ll post the code and documentation at a later date.

Any questions, ask in the comments.

11 thoughts on “f2fs – booting linux on an f2fs flash drive”

  2. No idea when. But you should be able to do it yourself. Go to the “issues” link at the nowcomputing git repo and read my comments. You’ll see that you apply all the 3.5 patches in order then apply 0001, 0006, and 0007 from the 3.2 patches. I prefixed them with 2 just so I could apply all the patches in one big loop. This worked great with 3.4.52 and should work with 3.4.53. It’s been running stable for a few days on an old laptop with a stick formatted f2fs.
    Cheers!
    PS: btw, my hacks only concerned AUFS, if you don’t use it then there is no hacking of the patches.

    Like

    Reply

  3. ” my hacks only concerned AUFS, if you don’t use it then there is no hacking of the patches.”

    What do you mean??
    I intend to compile the kernel for the Allwinner A10 Arm Soc… so what exactly do I need to do?
    Or, maybe… just maybe, do you want to try it?
    Some links:
    http://linux-sunxi.org/Linux

    and a working configs:
    http://romanrm.ru/dl/a10/kernels/server-3.4/3.4.43-20130531.1334-s-rm1%2b/config-3.4.43-20130531.1334-s-rm1%2b.txt
    or
    http://romanrm.ru/dl/a10/kernels/server-3.0/3.0.62-r1-rm2/config-r1-rm2

    I will volunteer to test it!
    (if you managed to do it, this kernel can be used for a Arm Puppy)

    Thanks
    Jorge.

    Like

    Reply

  4. I would tend to think using the latest (or near to it) kernel would be better for f2fs support as it was accepted in the main tree in k3.8. The only reason I wanted to backport it was for better driver suppot in 3.4 for some older nvidia chips, which a lot of my users have.

    As for Allwinner A10 Arm Soc, I can’t help you there. I do have a Pi and will (time permitting) compile a kernel for it with f2fs at some point.

    Cheers!

    Like

    Reply

  5. Mick,

    I just tried this on Slacko 5.5.70 “test” version from http://slackotest.01micko.com/slacko-5.5.70/. Got myself an old USB stick (1 GB), then partitioned it into two:
    – /dev/sdb1 as 16MB FAT16 partition
    – /dev/sdb2 as f2fs partition (the rest of the space – that’s around 993MB)
    Then I made the stick bootable with syslinux (on /dev/sdb1). Grab the above slacko, and copy vmlinuz to sdb1; unsquashfs puppy_slacko_5.5.70.sfs to /dev/sdb2. Then I created syslinux.cfg (on sdb1) as follows:

    default slacko
    label slacko
    kernel vmlinuz
    append root=/dev/sdb2 rw rootwait

    sync everything, take the stick out, put it to eeepc — and I got it to boot, without initrd. Xorg failed to start the beginning, but not that xorgwizard can’t fix. Speed was decent.
    I tried to recognise the partition by uuid (instead of /dev/sdb2) but it didn’t work like you say, the kernel kept waiting and waiting … so I’ll just stick with /dev/sdb2 for now.

    For reference, when I tested this my eeepc’s SSD had its original four partitions (showing up as sda1..4); and I had a FAT-formatted SD Card (showing up as sdc1). Reason why I made the FAT partition 16 MB (instead of 8MB – since the kernel was only about 4MB)? In case I want to add some initrd to it 🙂

    JotaMG, I’m porting Fatdog to A10. Look it up on my site.

    Like

    Reply

  6. Forget the f2fs – I would be happy just to surf the net with Slacko. [http://www.murga-linux.com/puppy/viewtopic.php?search_id=854117456&t=86196]

    Like

    Reply

    1. It could be a driver issue. If the drivers that come with puppy are what you’re using then you might try using Ndiswrapper and the Windows drivers. If that doesn’t work, and you tried Slacko and it didn’t work after Lighthouse didn’t work, then it’s probably a hardware incompatibility issue and you should try to find the problem hardware (wifi card, ethernet card, etc.) and just look for more linux friendly hardware.

      Like

      Reply

  7. Re: root= and uuid
    The kernel’s builtin requires root=PARTUUID=….. which is a global UUID comes from: http://www.rodsbooks.com/gdisk/gdisk.html and _NOT_ the same “uuid” that you get from blkid and requires EFI support to be enabled.

    The root= UUID tutorials that do not use PARTUUID are distro specific hacks that take place in that distro’s init script

    Like

    Reply

    1. Although jamesbond pointed out that boot from fat to f2fs is indeed possible (with hard coding to the partition number), my simple initrd proves that it can be done with uuid as produced by blkid. It’s getting late now but I will post the script tomorrow. I think it’s far simpler and more universal than worrying about gpt.

      Like

      Reply

      1. Ok, here is the code…
        #!/bin/bash

        # July 2013
        # (c) Michael Amadio, 01micko@gmail.com, GPL v2 2013
        # interactive script to install to an f2fs flash drive in puppylinux
        #-----------------------------------------------------------------------------#
        # depends:
        # k >=3.8, with f2fs as builtin or module, (OR patched for f2fs kernel)
        # +gparted>=1.4.1 patched for f2fs
        # +gettext
        # +syslinux >= 4.0 for vesamenu (optional)
        #-----------------------------------------------------------------------------#

        . /etc/DISTRO_SPECS

        . /etc/rc.d/PUPSTATE
        # best thing to do is copy all this from running system, no good though if usb or
        # if it's an existing full HD install, best actually if it's live CD or frugal
        case $PUPMODE in
        2|3|6|7|13|77)
        MSG0=$(gettext "Full and USB installs are not supported for now.nExiting")
        exit 0
        ;;
        esac

        # check f2fs support
        CFIG=/etc/modules/DOTconfig*
        . $CFIG
        if [ ! "$CONFIG_F2FS_FS" ];then
        MSG1=$(gettext "f2fs is not supported in your kernel.nExiting now")
        echo -e "$MSG1"
        exit
        fi

        KERNELVER=`uname -r`

        # simple progress fn
        progress_func(){
        while [ "1" ];do
        echo -n "="
        sleep 3
        done
        }

        # put the onus on the user, not too much hand holding
        echo -en "\033[1;31m" #red
        echo $(gettext 'Please ensure you have a USB stick plugged in,')
        echo $(gettext 'with a recommended first partition of 32MB FAT32')
        echo $(gettext 'and the rest of the stick formatted in f2fs.')
        echo $(gettext 'This can be done with GParted.')
        echo $(gettext 'BE SURE TO MARK THE FAT32 PARTITION WITH THE BOOT FLAG')
        echo $(gettext 'If you have done it hit ENTER.')
        echo $(gettext 'or if not hit CTRL-C now!')
        echo -en "\033[0;39m"
        read goon

        # can't trust users... sanity check
        HASF2FS=`probepart|grep 'f2fs'`
        if [ ! "$HASF2FS" ];then
        echo -e $(gettext 'No f2fs partition found!nExiting.')
        exit 1
        fi

        DESTPART=`probepart|grep 'f2fs'|awk -F '|' '{print $1}' |cut -d / -f3`

        echo -e $(gettext 'Now mounting ')$DESTPART
        [ ! -d /mnt/$DESTPART ] && mkdir /mnt/$DESTPART
        mount -t f2fs /dev/$DESTPART /mnt/$DESTPART -o rw
        [ $? -ne 0 ] && echo -e "Failed to mount $DESTPART nExiting" && exit 1
        DESTMNTPT="`mount | grep "/dev/${DESTPART} " | tr -s " " | cut -f 3 -d " "`"

        #exit #DEBUG

        #-----------------------------------------------------------------------------#
        # MAIN FS #
        #-----------------------------------------------------------------------------#
        # this is dirctly from PUI by Barry, some comments removed
        echo "$(gettext 'Please wait, copying Puppy files to') $DESTPART..."
        progress_func &
        XPID=$! ####
        sleep 1
        # this can take a while
        [ ! "$PUP_LAYER" ] && echo "Major error, bailing now!" && exit
        cp -a -u --remove-destination /initrd${PUP_LAYER}/* $DESTMNTPT/

        # fix handling if separate zdrv file
        [ -d /initrd/pup_z/lib ] &&
        cp -a -u --remove-destination /initrd/pup_z/* $DESTMNTPT/

        # put these modules back into the normal places...
        cp -a -u --remove-destination /lib/modules/$KERNELVER/initrd/* $DESTMNTPT/lib/modules/$KERNELVER/
        # fixup some stuff if missing
        cp -af /etc/DISTRO_SPECS $DESTMNTPT/etc
        cp -af /etc/modules/modules.* $DESTMNTPT/lib/modules/$KERNELVER/ 2>/dev/null
        sync
        depmod -b $DESTMNTPT

        if [ -d /lib/keymaps ];then
        mkdir -p ${DESTMNTPT}/lib/keymaps
        cp -a -f /lib/keymaps/* ${DESTMNTPT}/lib/keymaps/
        mkdir -p ${DESTMNTPT}/lib/consolefonts
        cp -a -f /lib/consolefonts/* ${DESTMNTPT}/lib/consolefonts/
        fi

        mkdir $DESTMNTPT/sys 2>/dev/null
        rm -f $DESTMNTPT/var/log/modprobes.log 2>/dev/null
        echo '#ATADRIVES is all internal ide/pata/sata drives...' >> $DESTMNTPT/etc/rc.d/PUPSTATE
        echo "ATADRIVES='$ATADRIVES'" >> $DESTMNTPT/etc/rc.d/PUPSTATE
        sync
        kill $XPID #### killed progress_func
        echo
        # chroot to DESTPART, set DISTRO_VERSION back to $PUPVEROLD, run rc.update
        echo -n "$(gettext 'Please wait, executing rc.update script...')"

        sleep 2

        FLAGDEVMISS=0
        if [ ! -e $DESTMNTPT/dev/null ];then #130613 recent pups have empty /dev. rc.update may need these...
        cp -a /dev/console $DESTMNTPT/dev/
        cp -a /dev/null $DESTMNTPT/dev/
        FLAGDEVMISS=1
        fi
        chroot $DESTMNTPT /etc/rc.d/rc.update option2hdinstall 2>/dev/null
        if [ $FLAGDEVMISS -eq 1 ];then #130613
        rm -f $DESTMNTPT/dev/console
        rm -f $DESTMNTPT/dev/null
        fi
        #(need to tell rc.update where src files are, as passed param)
        sync
        mv -f $DESTMNTPT/etc/fstab $DESTMNTPT/etc/fstab.bak
        echo "/dev/$DESTPART / $DESTFS defaults 0 1" > $DESTMNTPT/etc/fstab
        echo "none /proc proc defaults 0 0" >> $DESTMNTPT/etc/fstab
        echo "none /sys sysfs defaults 0 0" >> $DESTMNTPT/etc/fstab
        echo "none /dev/pts devpts gid=2,mode=620 0 0" >> $DESTMNTPT/etc/fstab
        echo "/dev/fd0 /mnt/floppy auto noauto,rw 0 0" >> $DESTMNTPT/etc/fstab

        sync

        echo
        #before we bale we get the uuid of the f2fs partition
        rootdev=`blkid /dev/$DESTPART|tr ' ' 'n'|grep 'UUID'|cut -d '=' -f2|tr -d '"'`
        umount /mnt/$DESTPART

        #-----------------------------------------------------------------------------#
        # INITRD.GZ #
        #-----------------------------------------------------------------------------#

        echo -e $(gettext 'Now building initial ramdisk')
        BOOTPART=`probepart|grep -i 'fat'|awk -F '|' '{print $1}' |cut -d / -f3`
        [ ! -d /mnt/$BOOTPART ] && mkdir /mnt/$BOOTPART
        mount -t vfat /dev/$BOOTPART /mnt/$BOOTPART
        INITSHELL=`find /usr/share -type d -name 'initrd-f2fs-skeleton'`
        cp -a -u --remove-destination $INITSHELL /tmp/
        echo "rootdev=$rootdev" > /tmp/initrd-f2fs-skeleton/init.cfg
        # wait number
        echo "WAIT=5" >> /tmp/initrd-f2fs-skeleton/init.cfg

        # find existing initrd
        MAINSFS=`find /initrd/mnt/dev_save -name $DISTRO_PUPPYSFS`
        mkdir -p /tmp/oldinitrd/init-tree
        mkdir -p /mnt/$BOOTPART/boot
        # boot from CD?
        if [ ! "$MAINSFS" ];then
        echo -e $(gettext 'Did you boot off CD or DVD?')
        echo -e $(gettext 'If so, make sure it is inserted now')
        sleep 5
        echo -e $(gettext 'Good you inserted the disk. Press Enter')
        read cd
        sleep 2
        [ ! -d /mnt/cdrom ] && mkdir /mnt/cdrom
        mount -t iso9660 /dev/cdrom /mnt/cdrom -o ro
        [ $? -ne 0 ] && $(gettext 'Failed to mount the disc.nExiting') && exit
        cp -af /mnt/cdrom/initrd.gz /tmp/oldinitrd
        cp -af /mnt/cdrom/vmlinuz /mnt/$BOOTPART/boot

        else
        INITLOC=`dirname $MAINSFS`
        cp -af $INITLOC/initrd.gz /tmp/oldinitrd
        cp -a $INITLOC/vmlinuz /mnt/$BOOTPART/boot
        fi
        cd /tmp/oldinitrd/init-tree
        zcat ../initrd.gz | cpio -i -d
        # copy busybox
        cp -af bin/* /tmp/initrd-f2fs-skeleton/bin/
        # copy modules
        for module in mmc_block.ko.gz mmc_core.ko.gz sdhci-pci.ko.gz sdhci.ko.gz tifm_sd.ko.gz tifm_7xx1.ko.gz tifm_core.ko.gz ssb.ko.gz
        do MOD=`find lib -type f -name "$module"`
        MODDIR=`dirname $MOD`
        mkdir -p /tmp/initrd-f2fs-skeleton/$MODDIR 2>/dev/null
        [ "$MODDIR" ] && cp -a $MOD /tmp/initrd-f2fs-skeleton/$MODDIR/
        done

        # console fonts and keymaps
        mkdir /tmp/initrd-f2fs-skeleton/lib/consolefonts
        cp -a lib/consolefonts/* /tmp/initrd-f2fs-skeleton/lib/consolefonts
        mkdir /tmp/initrd-f2fs-skeleton/lib/keymaps
        cp -a lib/keymaps/* /tmp/initrd-f2fs-skeleton/lib/keymaps

        # build initrd
        cd /tmp/initrd-f2fs-skeleton
        find . | cpio -o -H newc | gzip -9 > ../initrd.gz
        cd /root
        ls /tmp|grep -q initrd.gz
        [ $? -ne 0 ] && echo -e $(gettext 'Building ramdisk failed.nExiting') && exit

        # copy to stick
        echo -e $(gettext 'initrd.gz is built, moving to filesystem')
        mv /tmp/initrd.gz /mnt/$BOOTPART/boot
        sync
        rm -r /tmp/*initrd* #cleanup

        #-----------------------------------------------------------------------------#
        # SYSLINUX #
        #-----------------------------------------------------------------------------#

        echo -e $(gettext 'Now installing bootloader')
        # check if syslinux supports vesamenu
        grep -q 'vesamenu' ~/.packages/builtin_files/syslinux
        BIVAL=$?
        grep -q 'vesamenu' ~/.packages/* 2>/dev/null
        INVAL=$?
        [[ "$BIVAL" = "0" || "$INVAL" = "0" ]] && VMENU="yes" || VMENU=""

        if [ "$VMENU" ];then
        # define syslinux dir
        SYSDIR=`find /usr -type f -name vesamenu.c32|sed 's%/vesamenu.c32%%'`
        else
        SYSDIR=/usr/lib/syslinux
        fi

        CHOSEN=`echo $BOOTPART|tr -d [0-9]`
        CHOSEN=/dev/$CHOSEN

        boot_copy_func(){
        if [ "$VMENU" ];then
        for b in vesamenu.c32 chain.c32
        do cp ${SYSDIR}/$b /mnt/$BOOTPART 2>/dev/null #top level
        done
        fi
        sync
        }

        new_func(){
        # image
        IMGDIR=/usr/share/img/
        IMG=slacko.jpg
        mkdir /mnt/$BOOTPART/img/
        cp -a $IMGDIR/$IMG /mnt/$BOOTPART/img/default.jpg

        LABEL="$DISTRO_NAME"

        # construct menu
        cat > /mnt/$BOOTPART/syslinux.cfg </mnt/$BOOTPART/syslinux.cfg
        }

        if [ "$VMENU" ];then
        boot_copy_func
        new_func
        extlinux -i /mnt/$BOOTPART
        else
        old_func
        syslinux /dev/$BOOTPART
        fi

        sync

        # mbr
        dd bs=440 count=1 conv=notrunc if=${SYSDIR}/mbr.bin of=${CHOSEN}

        # cleanup
        sync
        sleep 1
        umount /mnt/$BOOTPART/

        echo
        echo -e
        "$(gettext "Your installation is now complete. You may now remove USB drive") $CHOSEN"
        echo -e $(gettext "Hit Enter to close this box.")
        read getoutahere

        Like

  8. Pingback: harvey

Leave a comment