EFI adventures

If you are like me, you probably have heard of EFI, but don’t know what it means in practice, how to eat it, or what to mix it with. But I recently bought a new PC, and it happened to come with EFI, so I decided to enable it and then I learned many things about it.

Whole new world

First of all, EFI is not something just can just “enable”. Even if your kernel has support for EFI, that won’t make a difference unless you boot it through EFI, and you can’t seamlessly boot through EFI unless you create a boot entry, which is done with efibootmgr, which only works once you are in EFI mode. So it’s a bit of a chicken-and-egg problem.

So how do you enter this world in the first place? First of all, you would need a shell. Some BIOSes come with one, but otherwise you would need to install one, and you can find public links on this wiki page.

But where to put this “shell”? Well, the EFI system partition–yes, you need a dedicated partition (more info here). This partition should be formatted using FAT32, and marked in a special way, so the EFI BIOS would be able to identify it, and use it’s contents.

You copy the shell to this partition on /shellx64.efi (I tried UEFI Shell 2.0 (Beta), but it didn’t work for me, so I used the old one).

Your BIOS should now be able to boot this shell… Somehow. It took me a long time to figure out that in my BIOS, that’s a hidden option on the exit menu, where you can “exit” into a shell.

Congratulations, now you are in EFI mode, and probably type some commands, like in the old DOS days, but not be able to do anything really (unless you have some EFI version of Prince of Persia).

The bootloader

There are a few EFI bootloaders out there, like ELILO, and efilinux, but I decided to go with the safest and familiar choice: GRUB2 (well GRUB was familiar, GRUB2 not so much). Arch Linux’s wiki describes in great detail how to setup GRUB2 for EFI, so I’ll just mention that you need to use a different command (‘grub_efi_x86_64-install’ in the case of Arch Linux), and the files need to be installed in the aforementioned EFI system partition.

You would end up with a file /efi/arch/grubx64.efi, and this is an EFI binary that you can use in the shell. Theoretically you should be able to enter the command “\efi\arch\grubx64.efi”, but for some reason that didn’t work for me, so I had to “cd \efi\arch”, and then “grubx64.efi”. Of course, you first need to choose the right partition, but the shell makes it easier by having aliases to the EFI system partitions, so you first type “fs0:” (or something like that), like I said; very similar to DOS.

Directly, please

But going into the shell and typing those commands every time you boot is cumbersome, which is why you would want to add a boot entry. So, once you have been able to boot Linux through EFI, you should be able to load the ‘efivars’ module, and thus access ‘/sys/firmware/efi/vars/’, which is what ‘efibootmgr’ needs.

Then you can do something like:

efibootmgr --create --gpt --disk /dev/sdX --part Y --write-signature --label "Arch Linux (GRUB2)" --loader '\efi\arch\grubx64.efi'

WARNING: You shouldn’t use this on Mac’s, there’s a different process for those.

In your BIOS you should see the option of booting into “Arch Linux (GRUB2)”, like you have the option of booting into a CD-ROM, and then you can choose that as the first one in the boot order πŸ™‚

Stub

But wait a second, that starts to look like two bootloaders: the BIOS has a boot entry, and so does GRUB. Why not boot directly into Linux?

A couple of patches from Matt Fleming enable just that. They should be in version 3.3. Enable CONFIG_EFI_STUB, and copy bzImage to /efi/linux/linux.efi.

So now you can boot directly from the BIOS into Linux (or Windows) with no bootloader at all πŸ™‚

Update: echo "initrd=\efi\arch\initramfs.img root=/dev/sda3 ro quiet" | iconv -t ucs2 | efibootmgr --create --gpt --disk /dev/sda --part 1 --label "Arch Linux" --loader '\efi\arch\vmlinuz.efi' --append-binary-args -

GPT

While you are on that, why not enable this new partitioning scheme? It’s more EFI friendly, and you can’t actually install a version of Windows that works with EFI without it. For Windows 7 it’s either EFI and GPT, or non-EFI and MBR; you can’t mix them (don’t ask me why).

Fortunately it’s very easy to switch from MBR to GPT; just run gdisk (from a rescue disk, of course). It’s also easy to switch back, as long ad you haven’t used more than four partitions.

Advertisements

Holy grail of packaging; yum history

Through the years I have often found myself asking this question “Which packages have I installed?” which comes in handy when you want to reinstall your system from scratch, or want to remove cruft. Unfortunately there has never been a proper way to answer it. One way would be to save somewhere all the packages that came with the installation, and compare with the current ones. However you would also see the packages that where installed as dependencies, which might be a lot. But finally, I noticed a new feature (to me) of yum; history. Continue reading

felipec’s installation notes

I regularly get rid of my home directory in order to prune my configuration, get rid of cruft, and backup important stuff. Here I’ll try to share the important steps to get a decent linux system configuration from scratch. Some of these are specific to GNOME, and some to Fedora, but mostly are generic.

root permissions

I hate to type the root password so I add my user to the ‘wheel‘ group and edit ‘/etc/sudoers‘ to add:
%wheel ALL=(ALL) NOPASSWD: ALL

So I just need to type ‘sudo -i‘ and I log-in as root (no password).

openbox

The first thing to do is to get rid of that annoying metacity. I choose openbox because the defaults work fine, and it doesn’t need anything special, just install it, and re-login with “GNOME/openbox”; voilΓ .

Now you have a decent window manager that resizes windows with alt + right button. And also you can configure it in many ways.

However, if you decide to stay with metacity, this makes it slightly more usable:
gconftool-2 --set --type bool /apps/metacity/general/resize_with_right_button true

Update: I found XFCE to be much superior to GNOME, and the WM works as expected by default.

keyboard settings

The next annoying thing is the keyboard settings; I find the repeat rate too slow:
gconftool-2 --set --type int /desktop/gnome/peripherals/keyboard/rate 98
gconftool-2 --set --type int /desktop/gnome/peripherals/keyboard/delay 242

zsh

I find bash too conservative; zsh provides many more options and extensibility, so install the ‘zsh‘ package, copy ‘/etc/skel/.zshrc‘ to your home, and then as root:

usermod -s /bin/zsh <user_name>

However, the defaults don’t play well with gnome-terminal; each console will show “Terminal” instead of the cwd, so edit ‘~/.zshrc‘:

case $TERM in
    xterm*)
        precmd () { print -Pn "\e]0;%n@%m: %~\a" }
        ;;
esac

Unfortunately, zsh doesn’t use readline’s inputrc, but it’s easy to convert:

bindkey -e
bindkey "\e[1~" beginning-of-line
bindkey "\e[4~" end-of-line
bindkey "\e[5~" history-search-backward
bindkey "\e[6~" history-search-forward
bindkey "\e[3~" delete-char
bindkey "\e[2~" quoted-insert
bindkey "\e[5C" forward-word
bindkey "\e[5D" backward-word
bindkey "\e[1;5C" forward-word
bindkey "\e[1;5D" backward-word

# for rxvt
bindkey "\e[8~" end-of-line
bindkey "\eOc" forward-word
bindkey "\eOd" backward-word

# for non RH/Debian xterm, can't hurt for RH/DEbian xterm
bindkey "\eOH" beginning-of-line
bindkey "\eOF" end-of-line

# for freebsd console
bindkey "\e[H" beginning-of-line
bindkey "\e[F" end-of-line

After doing these changes, re-login.

Certain settings are not specific to zsh and can be shared with bash, for that I use ‘~/.profile‘, which I manually include on ‘~/.bash_profile (not needed on ‘~/.zprofile‘):, and link ‘~/.zprofile‘ to it.

test -r ~/.profile && . ~/.profile

history-search

I avoid typing as much as possible, and quite often what I want to do is already in the history, so I find history-search-* commands essential. Fortunately this is now enabled by default in Fedora 13, but for the unlucky ones here are the instructions.

First, copy ‘/etc/inputrc‘ to ‘~/.inputrc‘ and make a minor modification:
"\e[5~": history-search-backward
"\e[6~": history-search-forward

This allows nice searches through the history, like ‘cp ‘ will search for the previous command starting with ‘cp ‘, type PGUP again and will search backards again.

However, you’ll need this on your ‘~/.profile‘:

export INPUTRC=$HOME/.inputrc

(a similar change would need to be done on zsh’s bindkeys)

development

Many applications benefit from these (on ‘~/.profile‘):

export EMAIL=felipe.contreras@gmail.com
export EDITOR="gvim --nofork"

And also, set your real name (as root):
usermod -c β€œReal Name” <user_name>

GNOME vim syntax

I like to have GLib code properly highlighted (in C), so I install gtk-vim-syntax which has a lot of stuff (D-Bus, GTK+, clutter, etc.).

Create ‘.vim/after/syntax‘, copy the files you are insterested on (glib.vim, gobject.vim, and gio.vim for me), and then, on c.vim:

runtime! syntax/glib.vim
runtime! syntax/gobject.vim
runtime! syntax/gio.vim

git

These go into ‘~/.gitconfig‘.

I like colors in git:
[color]
ui = auto

I’m not going to list all the aliases, just this one which is very useful:
[alias]
l = log --oneline --decorate --graph

I find mergetool essential to resolve conflicts:
[merge]
tool = gvimdiff
[mergetool]
prompt = false

Everyone should have their own exclude file:
[core]
excludesfile = /home/felipec/.gitignore

This is mine:
.*.sw[nop]

Very useful if you use sendemail often, (for more information see this other post)
[sendemail]
aliasesfile = /home/felipec/.mutt/aliases
aliasfiletype = mutt
chainreplyto = false
confirm = auto
smtpserver = /usr/bin/msmtp
envelopesender = "auto"

And a few more:
[user]
name = Felipe Contreras
email = felipe.contreras@gmail.com
[push]
default = current
[receive]
denyCurrentBranch = warn

Look and feel

fonts

Since I have a laptop (with LCD):
gconftool-2 --set --type string /desktop/gnome/font_rendering/hinting full
gconftool-2 --set --type string /desktop/gnome/font_rendering/antialiasing rgba
gconftool-2 --set --type string /desktop/gnome/font_rendering/rgba_order rgb

Also, I like the Droid fonts, and Iconsolata.

I copy them to ‘~/fonts‘, and instead of of manually configure everything on the
system to use them, I create a ‘~/fonts.conf‘ file like this:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
  <alias>
    <family>serif</family>
    <prefer>
      <family>Droid Serif</family>
    </prefer>
  </alias>
  <alias>
    <family>sans-serif</family>
    <prefer>
      <family>Droid Sans</family>
    </prefer>
  </alias>
  <alias>
    <family>monospace</family>
    <prefer>
      <family>Inconsolata</family>
    </prefer>
  </alias>
</fontconfig>

Then re-login.

mouse cursor

I like jimmac’s DMZ cursor theme.

gconftool-2 --set --type string /desktop/gnome/peripherals/mouse/cursor_theme dmz

Re-login.

Fedora has it installed by default, but if not, you can just extract the tarball into ‘~/.icons‘.

vim

I also wrote my own color scheme for vim, just copy to ‘.vim/colors‘, and then on ‘~/vimrc‘:
colorscheme felipec

felipec's vim color scheme

My custom color scheme

extra

Before I know it, I need Flash, which I install manually by downloading the tarball and extracting ‘libflashplayer.so‘ into ‘~/.mozilla/plugins‘. You don’t need to restart your browser, but it can’t hurt.

Then when I need to do some multimedia-related stuff I configure rpmfusion. Then add mplayer, which plays pretty much everything, or gnome-mplayer if you want a fancier UI.

After doing all this, and installing a few essential packages (such as vim-X11, gcc) I consider my system usable πŸ™‚

For the rest of my configuration files, check in github.

GStreamer development in embedded with sbox2

I’ve heard many people having problems cross-compiling GStreamer on ARM systems so I decided to write a step-by-step guide describing how I do it.

This guide is meant for scratchbox 2. You can read this old post where I explain how to install it. I’m using CodeSourcery 2009q3 but any compiler would do.

Continue reading

git send-email tricks

I recently found out a few awesome tricks for git send-email that have made my life much easier, so I decided to share them here πŸ˜‰

First, git send-email is an essential tool in every git guru’s arsenal, as it’s the preferred way of submitting patches. All you need to do is generate your patches with git format-patch (preferably with a cover letter), and then use git send-email to send those patches inlined in a nicely formatted email thread with your MTA.

But do you know about the different threading formats, address-book queries, or the cc-cmd option? No? Well, let’s get started.

Continue reading

Installing scratchbox 1 and 2 for ARM cross-compilation

Hi,

I’ve tried many different cross-compilation methods and so far scratchbox is the simplest and most effective. Here I’ll try to introduce the benefits of using it, and how to get started as simply as possible.

Intro

If you are not familiar with scratchbox; it’s a cross-compilation toolkit which allows you to use native tools (gcc, binutils, perl) when possible and emulate the target through qemu when needed.

It’s needed because of autotools; have you seen these checks?
checking whether the C compiler works... yes

The configure script actually compiles a small program and then runs it to validate it’s working, and sometimes extract information, such a the size of certain structures which might be different depending on the platform. If you tell ‘configure’ that you are cross-compiling it will go through a different path (which is much less tested) and ultimately will end up with wrong information that you need to correct.

OpenEmbedded and other distributions go through each and every package, make sure cross-compilation works, sometimes patching configure.ac, and often providing some information that normally would be obtained by running some test program.

This is an example of a typical GLib cross-compilation
./configure ---host=arm-linux --build=i386-linux

You’ll see something like:

checking whether the C compiler works... yes
checking whether we are cross compiling... yes

And then:

checking for growing stack pointer... configure: error: in `glib-2.20.3':
configure: error: cannot run test program while cross compiling

Of course, ‘configure’ has no way of knowing whether the stack grows on this particular platform. So you need to tell him yourself by creating an arm-linux.cache file like this:
glib_cv_stack_grows=no

And then running:
./configure --host=arm-linux --build=i386-linux --cache-file=arm-linux.cache

Of course that is not enough, you need to specify more:
glib_cv_stack_grows=no
glib_cv_uscore=no
ac_cv_func_posix_getgrgid_r=yes
ac_cv_func_posix_getpwuid_r=yes

And then the compilation fails because I don’t have glib-genmarshal in my system, and it’s needed by the build. Normally it’s compiled and run in the build, but now we can’t do that.

All these problems disappear with scratchbox.

Installing scratchbox 1

Many people think it’s difficult to install, but it’s not. Just follow these easy steps:

install

Download the basic packages:
wget -c http://scratchbox.org/download/files/sbox-releases/apophis/tarball/scratchbox-core-1.0.14-i386.tar.gz
wget -c http://scratchbox.org/download/files/sbox-releases/apophis/tarball/scratchbox-libs-1.0.14-i386.tar.gz
wget -c http://scratchbox.org/download/files/sbox-releases/apophis/tarball/scratchbox-devkit-qemu-0.10.0-0sb5-i386.tar.gz
wget -c http://scratchbox.org/download/files/sbox-releases/apophis/tarball/scratchbox-toolchain-cs2007q3-glibc2.5-arm7-1.0.12-9-i386.tar.gz

Extract them:
sudo tar -xf /tmp/sb/scratchbox-core-1.0.14-i386.tar.gz -C /opt
sudo tar -xf /tmp/sb/scratchbox-libs-1.0.14-i386.tar.gz -C /opt
sudo tar -xf /tmp/sb/scratchbox-devkit-qemu-0.10.0-0sb5-i386.tar.gz -C /opt
sudo tar -xf /tmp/sb/scratchbox-toolchain-cs2007q3-glibc2.5-arm7-1.0.12-9-i386.tar.gz -C /opt

Setup scratchbox, and add your user:
sudo /opt/scratchbox/run_me_first.sh
sudo /opt/scratchbox/sbin/sbox_adduser $USER yes

You'll need to re-login to be in the sbox group and have proper permissions:
sudo su $USER

target

Finally, setup an armv7 target (you can have multiple targets inside scratchbox):
/opt/scratchbox/tools/bin/sb-conf setup armv7 --force --compiler="cs2007q3-glibc2.5-arm7" --devkits="qemu" --cputransp="qemu-arm-sb"
/opt/scratchbox/tools/bin/sb-conf select armv7
/opt/scratchbox/tools/bin/sb-conf install armv7 --clibrary --devkits --fakeroot --etc

That's it, you have scratchbox setup πŸ™‚ I explicitly mentioned all the commands, but instead you can run this script that I wrote.

start

Before running scratchbox you'll need to do some steps as root:
echo 0 > /proc/sys/vm/vdso_enabled
echo 4096 > /proc/sys/vm/mmap_min_addr
/opt/scratchbox/sbin/sbox_ctl start

And then as user:
/opt/scratchbox/login

This will get you to this screen:

Welcome to Scratchbox, the cross-compilation toolkit!

Use 'sb-menu' to change your compilation target.
See /scratchbox/doc/ for documentation.

[sbox-armv7: ~] > 

Now if you want to cross-compile GLib, you do it as in your PC:
./configure && make install

Much easier, now scratchbox does all the magic πŸ˜‰

Scratchbox 2

Scratchbox 1 serves it's purpose, but there are many corner-cases where things get overly complicated so people came up with a much more elegant approach: Scratchbox 2.

In sb1 you need to login to a target (e.g. armv7, armv6, fremantle, diablo, etc.) in order to do anything, you can use only one target at a time, and each target is independent, in order to share tools between targets you need a devkit. Also, toolchains must be packaged in a special way.

In sb2, you don't login, you can setup any toolchain easily, you can use multiple targets at the same time, and you can configure it to do pretty much anything you want.

QEMU

sb2 doesn't include QEMU, you must have it already, this is how I compile it:
git clone git://git.savannah.nongnu.org/qemu.git
cd qemu
git checkout -b stable v0.10.5
./configure --prefix=/opt/qemu --target-list=arm-linux-user
make install

sbox2

Compile and install like this:
git clone git://anongit.freedesktop.org/git/sbox2
cd sbox2
./configure --prefix=/opt/sb2
make install

Add sb2 to the PATH:
export PATH=/opt/sb2/bin:$PATH

target

Now it's time to configure a target, I have a CodeSourcery toolchain installed on /opt/arm-2008q3, so:
cd /opt/arm-2008q3/arm-none-linux-gnueabi/libc/
sb2-init -c /opt/qemu/bin/qemu-arm armv7 /opt/arm-2008q3/bin/arm-none-linux-gnueabi-gcc

You don't need to log-in, just prefix your commands with sb2 to do the magic:
sb2 ./configure --prefix=/opt/arm/

If you want to use a different target just use the -t option:
sb2 -t armv8 ./configure --prefix=/opt/arm/

How cool is that?

Transcoding for the Internet Tablets the smart way

A while ago I wrote the “official” video transcoding for Maemo how-to based on the wiki page and some heuristics.

The interesting thing here is the intelligent script that automatically finds the right parameters to use for the encoding.

Some of the considerations include:

  • As higher framerate as possible. For smooth videos.
  • Keep aspect-ratio. So the videos don’t look weird.
  • No cropping. You might miss something.
  • Single pass. Bigger files, but constant CPU usage.

Is this thing really smart? Let’s try it.

Crappy clip from YouTube

transcode.rb -v -i youtube.flv
* Input: 320x240 [4:3], 29.917 fps, 0 kbps.
* Output: 320x240 [4:3], 29.917 fps, 400 kbps.
mencoder youtube.flv -o youtube_it.avi -srate 44100 -oac mp3lame -lameopts vbr=0:br=128 -af volnorm -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=400 -ofps 29.917000

Nothing special, the script maintained all the parameters, so there shouldn’t be any problems. Also, the aspect-ratio is not so different from the one of the device. This one is OK.

DVD

transcode.rb -v -i dvd://1
* Input: 720x480 [3:2], 29.97 fps, 7500000 kbps.
* Output: 400x240 [5:3], 29.97 fps, 400 kbps.
mencoder dvd://1 -o 1_it.avi -srate 44100 -oac mp3lame -lameopts vbr=0:br=128 -af volnorm -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=400 -ofps 29.970000 -vf-add scale=400:240

In this case the aspect-ratio is slightly modified (less than 10%), but the target aspect-ratio is exactly the same as the one of the device, so, it will look good.

So far the script does seem intelligent πŸ™‚

High Definition

Now let’s try a non-standard resolution, high bitrate, and H.264 format.

transcode.rb -v -i starcraft.divx -f h264 -q 8
* Input: 1024x436 [256:109], 23.99 fps, 1917920 kbps.
* Output: 352x144 [22:9], 23.99 fps, 1500 kbps.
mencoder starcraft.divx -o starcraft_it.avi -srate 44100 -oac mp3lame -lameopts vbr=0:br=128 -af volnorm -ovc x264 -x264encopts bitrate=1500:nocabac -ofps 23.990000 -vf-add scale=352:144

The aspect ratio now is about 4% different, that’s good. Unfortunately the aspect-ratio is quite different from the one of the device, but that’s not too bad.

This is how it would look like:

screenshot-hd

note H.264 videos look pretty good on the device.

Conclusion

So far there hasn’t been a single video where I had to modify the parameters that this algorithm suggested; sometimes the quality is not that good but increasing it with the -q option does the trick.

Wondering if it would find the right parameter for your clips? Why don’t you try it and find out πŸ™‚

The script is here. And the garage project here.