Saturday, October 18, 2014

Notes on OSX Virtalization, booting raw disks

I picked up a Mac to use for work, primarily, a couple weeks ago.  It's a late 2013 Macbook Pro Retina 13" w/ 8GB ram and 256GB SSD.  I wanted to be able to dual boot with Linux for home and dev type stuff, but being short on time needing to encrypt my primary hdd, I didn't get to figure out boot camp before finalizing my setup for work, so I'm not about to mess with my primary disk's partition/efi config and risk downtime at work.

Sooo, I'm trying to get set up to boot to boot Linux on an SD card.  This has been a bumpy road.  I purchased a Nifty SD card adapter and a Sandisk 80MBps 64G card to install to, and I'm doing the install via a VM while I work on other stuff.  This presents first problem - support for booting SD card by Parallels, Fusion, or VirtualBox.

I'm defaulting to Parallels since everyone else on mac uses it at work, but Fusion looks pretty comparable.  I've used VirtualBox a lot in the past, but I got tired of bugs and crashing at some point and switched to VMWare workstation at home.

That said, VirtualBox seems to be the most flexible and capable when it comes to booting SD cards on Mac.  It is the only virtualization solution of the three that doesn't have  "dumbed down" interface for people who expect things to just work without understanding them on mac.  It comes with command line tools for creating virtual disk files, etc.  This is what you need if you want to make a "bootcamp" disk image to use with Fusion.

Parallels and Fusion only support booting raw disks via "bootcamp".  You can't just point them to a block device and have them use it without tricking them.

  • To trick Fusion into booting to the SSD, set up a linux VM with a virtual disk that you'll replace.  Then use virtualbox command line tools to provision a raw disk vmdk and copy this over in place of the virtual disk that Fusion created.   That's it. It should boot fine. 
  • To trick Parallels, and note that I haven't found a way to do a raw disk vmdk like with Fusion, create a VM without a disk and enable efi.  You may want to remove Network from the boot order.  In the VM settings menu, attach the SD card to the VM.  Power up your VM and wait for it to drop you to the efi shell.  Then run:
    • fs0:
    • cd EFI\ubuntu
    • grubx64.efi
Getting the macbook to boot the SSD is another matter, and it's where I'll look at the actual Ubuntu build.  

Macbook Pro Dualboot Ubuntu SD Card & Virtual in OSX

This is just a quick post outlining how I finally was able to get ubuntu installed on an SD card that would boot from the option menu when I power on the Macbook and would also boot in either Fusion or Parallels in OSX.

For reference I'm on a Q4 2013 Macbook Pro 13" Retina with a SanDisk Extreme Pro 64GB SD card.  Not the fastest, but nearly the fastest.  I finally had success using VMWare Fusion Pro 7.0.  This may work with Parallels, which seems to have better EFI support, but has not been very stable for me.  This solution would not boot in Fusion 6.x.

  • Install the SD card in the laptop.
  • Use VirtualBox to create a .vmdk disk file that points to the SD card (/dev/disk2, in my case).  This effectively creates a "bootcamp" disk image to use with Parallels or Fusion. 
  • Download the rEFInd (the more recent version of rEFIt) usb stick disk image and dump it to the SD card:
    • Disk Manager to see what the device name is for the SD card.  It was /dev/disk2 for me. 
    • Use mount to see which partitions of the SD card are mounted:
      • mount | grep /dev/disk2
      • blah blah /dev/disk2s1...
    • Use diskutil to unmount:
      • diskutil unmount /dev/disk2s1
      • ...
    • Write the rEFInd image to the card:
      • dd if=rEFInd_blah.img of=/dev/disk2
    • Use diskutil info to take note of the name of the disk2s1 partition name. Something like ElTorito. 
  • Download your choice of linux distro.
  • Create a VM in VMWare Fusion and:
    • Attach a 20GB IDE hdd to it.
    • Attach the Linux install CD and set it as default boot. 
    • In the VM config directory, remove the 20G disk image vmdk files and rename the .vmdk that was created in VirtualBox to match. 
    • Add firmware="efi" to the vmx config file for the VM. 
  • Boot the new VM and when it loads into the Linux Live CD:
    • Dump /dev/disk2s1 (this is the 4.5MB partition that was created when we dumped the rEFInd image file to the card) to a tmp file:
      • dd if=/dev/sda1 of=./sda1.dd
    • Use gdisk (the GPT equivelant of fdisk) to:
      • Take note of the starting and ending sectors  and the partition type of the refind partition (sda1)
      • Wipe the GPT table (under expert optoins somewhere) and then re-open gdisk.
      • Create a new partition of for rEFInd using the same starting and ending sectors we used earlier and the same partitoin type. 
      • Set the name of this partition to the same as it was before (ElTorito?)
      • Create a linux partition using the remaining space.  You may want to leave some buffer space after the first partition in case you ever need to update refind and don't want to have to resize or rebuild the linux partition. 
      • save and exit gdisk
    • Restore our image of sda1 to the new partition we created along with the new GPT partition table:
      • dd if=./sda1.dd of=/dev/sda1
  • Reboot the VM with SD card as default and verify that it comes up in the rEFInd boot menu. 
  • Reboot the VM and boot to the CDROM again and start the install. 
    • I'd recommend using ext2 filesystem, since it doesn't use journaling and is fast, and no swap, to conserve writes to the SD card.  
    • Tell Ubuntu (or your distro) to install the bootloader to sda2!!! this is critical, so it doesn't bork your rEFInd bootloader.  
  • When the Linux install completes, reboot the VM to SD card and verify that rEFInd loads and that, from there, you can select the linux image and boot to it.  
  • if that works, then reboot the laptop and hold "option" to get the boot menu.  You should see the SD card and be able to boot Linux via rEFInd!
So Simple!    You don't want to know how many days of tinkering and repeated linux installs it took to find a setup that would work like this.  Fack!

Trying to boot to CD and install ubuntu directly to the SD card never worked for me, as the MAC bootloader wouldn't see the installation and present it in the boot menu when the laptop powers up, despite Ubuntu installing EFI boot junk on the SD card.  It wasn't too difficult to get the SD card to boot in EFI or legacy mode inside of a VM once figuring out how to do the bootcamp vmdk, which raises another important thing to note:

Fusion and Parallels are playthings only.  They really have no advanced tools included via command line or the standard UI.  There's no way to "bootcamp" to an SD card with them without hand-editing their config files and using 3rd party tools to create the disk image files.  VirtualBox is indispensable when manipulating VM images.  It comes with a bunch of great command line utilities for creating and managing disk images, and who knows what else.  I don't use it normally, though, because I used to run into issues with stability when I used it in Linux.  Meh... It worked great before Oracle bought it.  :P

Update:  Just saw the following, about creating raw disk iamges with vmware tools... might be helpful:

And this, which talks about using "tmpfs" to redirect IO-intensive filesystem activity to ramdisks and cut down on SD card activity:
He suggests:

tmpfs    /tmp    tmpfs    defaults,noatime,nosuid,size=100m    0 0
tmpfs    /var/tmp    tmpfs    defaults,noatime,nosuid,size=30m    0 0
tmpfs    /var/log    tmpfs    defaults,noatime,nosuid,mode=0755,size=100m    0 0
tmpfs    /var/run    tmpfs    defaults,noatime,nosuid,mode=0755,size=2m    0 0
tmpfs    /var/spool/mqueue    tmpfs    defaults,noatime,nosuid,mode=0700,gid=12,size=30m    0 0

Might be useful... but my system wouldn't boot with all of these enabled... will have to actually think about this and do some testing later.

Tuesday, February 04, 2014

Why I love bash - simple remote connections

Bash is my go-to language for busting out sysadmin stuff.  I've been ramping up with python for a while now, and for more complicated tasks, it's fantastic;  it should probably even be fantastic for simple tasks.  But, there is one thing that I do all of the time in Bash that I have yet to find a way to replicate without unnecessary complexity in python:

agregateResults (); { while read line; do echo "Program Output: $line"; done; }
clientCode (); { echo "I am client $(hostname)."; }
if [[ "${1}" == "client" ]]; then
    for server in  ${*}; do
        cat ${0} | ssh ${server} -s client
    done | agregateResults

This is so freaking useful.  I have servers across multiple domains and I need to run script to aggregate data from all of them, but I can't always count on network file systems being available, etc.  This lets me have a script in once place and execute the same code everywhere without fumbling with copying it ahead of time, etc.   It's so clean and easy.

When I try to find out how to do this with Python, I read all kinds of stuff about parallels, Paramiko, blah blah blah... it's all so complicated.  There has to be a five-liner example out there somewhere of how to do the same kind of thing.  

Started a code-golf at stackexchange for this one: