Maemo Reconstructed

Our goal is to bring together Nokia and community developers who have an interest in the Maemo platform and reach a vision of the future Maemo system architecture that's satisfying to both Nokia and the community.

These are the low level technical system goals (use of booting process, busybox/coreutils, kernel features etc.), and not high level stuff like "Ogg support", or "open source everything". The process is called Maemo "Reconstructed", as to indicate we can remix parts from and disassemble the current Maemo platform and augment it until we get a vision that would be fitting as the future of the Maemo platform.

If you have proposals that make certain parts of this proposal impossible, please add it to the talk page. Again, this is not the place for discussion on high-level topics like A2DP support and finger-friendliness. These are low-level platform issues. If you want to shoot down part of the proposal, document your findings on why it is not possible or should not be done and discuss it on the talk page.

This proposal was initially written by Stskeeps (Carsten V. Munk), but is intended to be a unified vision that Nokia and the community will agree upon.

This discussion was sparked by:

Contents

The Maemo "Reconstructed" platform

The purpose of Maemo "Reconstructed" is to make Maemo a general platform for tablet devices (including Nokia devices), to make it more developer-friendly, and to bring it into alignment with standard Linux distributions. It is also within the philosophy that the platform should be "hackable", a main selling point with open-source friendly hardware.

We should stop seeing the tablets as strictly under-powered embedded systems, and see them for what they really are—powerful, power-efficient, economical handheld computers. Not simply hopped-up electronic calendars, but real computers that are capable of almost as much computing as a laptop.

For distribution, the methods, categories, and distributions outlined in the Maemo System Model - Part II could come to good use in Maemo "R". Maybe a part about Maemo vs Maemo + Nokia closed source components and seperation in archives?

There should exist scripts and documented methods for creating a Maemo "R" image that can be adapted to any setup or device. It should be easy to build your own jffs2 image.

Development of the Maemo "R" platform (non-device-specific and non-vendor-specfic differentiation) should be done in the open, with public SCM, bugtrackers and blueprints. This way, developers can adjust their applications to later 'stable' releases and know what is coming regarding the Maemo API.

One of the goals of the Maemo "R" platform is that it should be easy to port an existing desktop application by hildonizing it and adjusting to the form factor of a tablet, without having to deal with platform particularities like what features the busybox version of a tool uses, or other factors. A secondary goal is to have a platform that is flexible enough to encourage experimentation and development for tablets.

Kernel

For the Maemo "R" platform specific kernels does not matter, but it may be relevant to discuss certain features that would be nice to have for the platform to work correctly.

Boot process

The basic boot process of a Maemo "R" platform consists of loading a kernel from a flash area, or from a file on a partition either on flash, MMC or USB, which then mounts the chosen root partition. This could be done with uBoot for instance (as seen on Beagle Boards), but the bootloader choice is up to the vendor, with the stipulation that it should support some of the following abilities:

In the boot process of the Maemo "R" platform, there should be a way to use alternate boot configurations. This menu of alternate boot configurations should be possible to enter through a key combination while turning on the device (as with the current bootmenu and holding down the menu key). The primary use for this would be to get the device into a rescue menu for recovering the rootfs after a reboot-loop or other crash.

The behaviour of the boot process should be modelled on grub's menu.lst, as to allow different boot configurations. Possible ways to deal with this can be alike with a /etc/bootmenu.d like setting which updates the boot configuration with refresh_bootmenu.d (inspiration from grub menu.lst, http://internettablettalk.com/forums/showthread.php?t=23051).

Cloning to and booting from a flash card is a popular hack, as it offers significantly more storage space and some performance improvements. A bundled boot menu system will streamline the setup of multi-boot systems, and reduce the impact of SSU on these setups. The boot menu should be unintrusive and invisible to the "normal" user, except when activated with the boot menu key combination (but may be activated to show the menu always at boot in a control panel).

There should exist a rescue partition in standard Maemo "R" image, possibly like http://www.youtube.com/watch?v=h24f2YjzWBE (or in text form). That can both activate certain services from the rescue partition or set boot settings such as framebuffer console or other settings for the system (think F8-while-boot in Windows.)

Many users often run into the reboot loops. A rescue partition will help users recover from these loops by offering a remote shell, dumping the user's files from /home/user to a connected PC, and allowing for the creation and restoration of backups on a flash card or a connected PC. These abilities should be modelled with a "plugin" structure, to encourage innovation in this area.

Busybox+uclibc should be used to ensure minimal rescue partition size. Toolkit for the right combination should be as easy to get as Maemo SDK. Busybox shouldn't be stuck at 1.0 like it is in initfs right now.

The rescue partition should be possible to update without having to reflash, and may employ a package system, to encourage innovation in this area (referencing to the case of initfs getting reflashed completely, removing boot menus and other initfs modifications, in Diablo).

Base system

Maemo needs to be brought into alignment with "major" distributions like Ubuntu and Debian, and the base system is a good place to begin. Maemo is its own distribution, targeted toward embedded systems, but on the other hand Maemo runs on systems that are as powerful as the computers these distributions were initially developed on, and many of the typical embedded tricks may simply not be needed. It is helpful seeing Maemo targeted at power-saving mobile devices where it's fine to be able to run a traditional "desktop" distribution.

Nokia should embrace the Debian heritage of Maemo fully and put processes and practices into place that make Maemo a Debian derivative like Ubuntu, only better.

– Marius Volmer

This proposal tries to see Maemo as a deriative of Ubuntu. The reasons for this direction is:

  • Handhelds Mojo, which is sponsored by Nokia.
  • Debian, for now, targets armv4t, and Mojo has already done the hard work to target typical Maemo device processors.
  • Taking Ubuntu over Debian (for now), gives us upstart along (part of Task:Maemo roadmap/Fremantle), which has not been adapted properly (except in Debian experimental) in Debian just yet.
  • A remark however should be that Debian and Ubuntu is not very different, at base system level. The choice of Ubuntu here has been done to illustrate a base Linux system optimized for the Internet Tablets, with some of the technologies such as Upstart pre-existing in the distribution. armv5 was chosen for no particular reason except that the Hasty armv6 build has not yet been completed.

Maemo base system includes:

  • 'minbase' Mojo handhelds Ubuntu 'Hasty' (based on Hardy, or later)
    • Motivation: This is a proof of concept. How functional and how well performing is a standard minimal ubuntu base image on a N800? Our restrictions is about 250mb flash availiable to us, with ideally space to spare.
    • Details:
      • # cp /usr/share/debootstrap/scripts/hardy /usr/share/debootstrap/scripts/hasty
      • # debootstrap --arch=arm --variant=minbase hasty /target http://repository.handhelds.org/hasty-armv5el-vfp/
      • # chroot /target mount -t proc proc /proc
      • # chroot /target mount -t devpts devpts /dev/pts
      • # apt-get install apt-utils udev
      • # cd /dev; /sbin/MAKEDEV fb0 fb1 and such.
      • # apt-get autoclean; apt-get clean
      • add in a couple of boot scripts from and framebuffer console kernel drivers from Deblet, and docpurge-1.6 from Maemo.
      • Now you have a bootable minimal ubuntu image (tested on N800 with bootmenu with linuxrc ability), with full coreutils and bsdutils, etc.
    • Performance: From selecting in boot menu to boot, 12 seconds to get to framebuffer console login, with ext3 mount, udev and all.
    • Size: uncompressed 89.5mb but, mkfs.jffs2 -r /target -o maemo-r.jffs2 -e 128 -l -n , 37.0mb in JFFS2.
    • Contents: http://bsd.tspre.org/~stskeeps/minboot.txt + docpurge
    • Other examples, http://bsd.tspre.org/~stskeeps/ubuntumin.txt (69m jffs2, may have forgotten docpurge in this one.)
    • Advantages: We now have a full ubuntu base system on our tablet. We've got Upstart for for free without any development. We have a base we can easily work with upstream on, as we base on typical packages. The mojo project also gets to have instant value to the Nokia/Maemo development and Nokia gets a return from the research investment in it.
    • Discussion: Is busybox really needed on a tablet? In initfs, sure, but in the actual system? Having sane coreutils and bsdutils (and debconf) -really- helps porting. Now that dash exists, is there a reason for busybox as sh? This avoids any conflicts with coreutils, bsdutils etc if they are going to be used.
      • A simple (which may be unfair and inaccurate) experiment with dpkg-deb -x of procps, coreutils, util-linux, bsdutils into a path, and then mkjffs'ing it after stripping man pages, locales and info pages, 2.2mb flash, and with busybox getting same treatment, 300k flash.
      • Is 1.9mb flash so much to sacrifice for making porting for Maemo a matter of hildonization and platform-centric things, instead of hunting debconf, busybox incompatibilities and such for weird libraries you never heard of?
      • Are there any direct and noticable differences in performance that justifies the use of busybox, and do they justify the lessened ability of being able to port to Maemo without hassle?
    • Problems: For some reason the mojo people decided to divert from Debian arch notation and call their ARM EL distribution arch type 'arm', not 'armel', which is in wide use with Maemo.
  • DSME (Device State Management)
    • Basic device/Mobile service DSME should be open sourced, but may load modules of hardware-specific closed source modules (CAL access, watchdog kicker, etc), and DSME protocol should be documented and the modules associated with it (CAL access seems to be a touchy subject though).
    • DSME is a good part of a mobile platform (process watchdog, etc), but needs documentation of the communication protocol (even though most of it has been reverse engineered) for community OS developers to work properly with it. Can it really be true things like Advanced Backlight (http://adv-backlight.garage.maemo.org/) has to use undocumented DSME protocol to do what it should?
  • OHM (Open Hardware Manager)
    • Part of Task:Maemo roadmap/Fremantle. http://ohm.freedesktop.org/wiki/UseCases OHM may apply device-specific plugins but these are not included in the Maemo "R" base. Developers should strive towards not using sysvinit init scripts, but go for upstart job descriptions instead, which can be controlled by OHM (disable RSS feed reader download attempts while obviously offline, for instance).
  • GUPnP
  • Meta Tracker (trackerd)
  • D-Bus
    • Part of the Maemo platform as of now, - is ideal for embedded systems as it works with signals and events.
  • HAL
    • Part of the Maemo platform as of now, needed for OHM. HAL should be targeted to embedded systems but should not contain device specific things. These should be published as HAL addons (and may be closed source, example could be hald-addon-bme as it done right, and hald-addon-omap-gpio not), as to keep as close to HAL upstream as possible.
  • Generic API for connectivity (libconic comes to mind), using D-Bus. It should not have to include a ICD as some vendors may vary on that (Nokia), but libconic should be generic and usable in both Nokia ICD and NetworkManager based setups. Ideally a NM-based ICD or a stub ICD should be provided along with platform, but should be replaceable. Under this things like iptables, udhcpc, dnsmasq would be.
    • Many Maemo applications depend on libconic and this could be a showstopper for portability.
  • Udev
    • Part of Maemo platform already, and useful for desktop and embedded systems.
  • Ke-recv & Alarmd
    • Some daemons representing these (Kernel event handling + scripts, maybe OHM?), and a API for setting device wakeups. Maybe even a cron-like interface?
  • GConf
    • Central repository for system-wide configuration.

Other things like supporting libraries such as OpenSSL and such are not directly mentioned but obviously needed as dependancies.

Hardware Support

A Maemo "R" image may contain closed-source hardware specific firmware, drivers (cx3110x, etc), daemons (BME, hw watchdog kicker). This will probably be common on commercial platforms since there are trade secrets/NDAs/etc. Vendors for the Maemo platform should strive towards allowing redistribution of the binary blobs (restricting to use only on the hardware they are intended for should be okay), to encourage remixing and experimentation with the system and the device.

In case of kernel drivers these should ideally be open source, but this may not always be possible. Nokia, for instance is going in the right direction with open source wifi driver. In case of newer kernels for "old" devices, vendors should aim for giving a community developer access to kernel module source for recompiling (with possible NDA) for later versions or provide build services (insert kernel tar.gz, get out .ko).

In case of closed source daemons (BME, DSME modules) the protocols should ideally be documented (it should be possible to get remaining battery time and bars to show and such from BME, maybe even current current usage of the device.). This will satisfy most OSS OS developers, but the problem about closed source daemons is that there should optimally be updates, even for old devices (backwards compatible daemons maybe, maybe automated build services again/community/head contributors access to recompile, so Nokia wouldn't have to invest time to support "old" devices, could be a possibility.)

In case of generic devices that will exist on multiple platforms (battery for instance), it should be encouraged to work on standardization of the information (HAL battery status comes to mind).

User Experience Base

  • Xorg
    • Part of Fremantle roadmap.
    • Device-specific Xorg drivers (omap for N8x0's, other devices has other drivers, etc..). It should not be a requirement which Xorg driver is in use, but the capabilities of this.
  • GStreamer
    • Part of Maemo platform as of now, ideal for codecs, include vendor-specific propertiary ones.
  • PulseAudio
  • BlueZ
    • Part of Maemo platform as of now. Optional part of platform?
  • Hildon desktop & Matchbox WM'
    • A basic part of the Maemo platform as of now. However, the initialization of Hildon should be done through an alternatives system like in Debian (update-alternatives --config) x-session-manager, so having Hildon on top of a Maemo system is one use interface. The Maemo system should not depend on it being Hildon running on the device.
  • GTK/GDK/etc.
    • The base graphical framework of Maemo. This GTK version should be Maemo 'GTK', but the following things should hold true for the Maemo GTK.
      • Maemo GTK is designed for power-saving low-freq devices. It is designed for touchscreens and with memory conservation in mind
      • It should not alter it's behaviour in such a way that a certain window manager or desktop environment is needed to run applications using this toolkit. If this change of behaviour is needed, the toolkit should detect the WM/DE and enable the behaviour if it's Hildon+Matchbox.
      • It should not change upstream API (but may add or cause breakage that would make normal GTK applications not work with it.
  • GTKMM
    • There should be possibility to use two versions of GTKMM, one with vfuncs removed for embedded usage and one with them enabled for running GTK applications that require these to make porting easier (or simply enabling vfuncs in GTKMM for Maemo.). The replacement GTKMM may be a package that is downloadable which replaces the non-vfunc GTKMM.
  • Hildon application manager
    • A mobile system must have ability to add/remove applications..
  • Hildon control panel
    • The Maemo "R" platform can be adjusted from in here, but the configuration settings used in various applets should be fully documented on how to do programmatically.
  • Hildon input method
    • Input method for touchscreens.
  • Basic Hildon theme
    • Plankton or the likes to start building Maemo systems that look "good" from the start.
  • Other open source components currently in Maemo.
    • There are plenty other open source components currently in Maemo, which is not listed here, but considered part of the platform.

The goal of this user experience base is to have a solid open source base to build Maemo platform devices, both for Nokia and for community to remix, or use on other devices. This user experience base should be usable, but should be augmented with vendor-specific differentiations (ICD, Applications, Browser support, drivers, hardware support, home applets, etc.), but documenting the interfaces developed for these.

Developing for the Maemo platform, and not for the actual devices, should be encouraged except in cases where the applications require certain non-standardized (yet) interfaces. This will be useful for hardware vendors as well as development and change in hardware contents will be easier to achieve, with the applications still working on new hardware as they target the Maemo platform, not the device.

SDK (developer kit for the platform)

The SDK is a basic part of developing for Maemo. But there are occasional horrorstories. Based on some of the stories about the SDK and other trends in the mobile device world, this is a vision of a proper SDK for Maemo "R".

Maemo "R" SDK, should consist of four parts:

Scratchbox (cross-compilation environment)

Build programs for another architecture/target than on the one compiling.

Maemo "R" rootstrap - Maemo base OS with libraries and dev packages to cross-compile any applications for Maemo "R", not to run them.

Cross-compiliation certainly speeds up compilation and allows a proper developer environment for developing for Maemo "R". "Not to run them" is meant as, CPU transparency is okay, but the rootstrap should not try to be a fullly deployed Maemo "R" system. The rootstrap shouldn't try to act as an tablet - we have the emulators for that.

A Maemo "R" rootstrap may have Nokia/device specific dev packages but developers should be encouraged to develop for the defined Maemo "R" platform. Nokia/other hardware vendors specific dev packages should strive towards non-device-specific standardization of the interfacing if possible (a BME dev package may not be relevant on other devices).

Maemo has great possibility to become a general platform for tablet like devices. The possibility of additional device platforms which also embrace the Maemo minimal base + build-essential + dev packages direction will cause every Maemo device to have additional applications as well when they appear for other devices.

QEMU appliances included with target device(s) emulated

The appliances would be shipped with a device-specific Maemo "R" installed on it. This is where built applications should be run, not inside the SDK (no Xephyr, not full hildon desktop, etc. in Scratchbox). These should be configured with sbrsh for cpu transparency from initial setup. N8x0 emulation is already seen (http://blog.haerwu.biz/2008/04/11/nokia-n800-emulation/). Qemu appliances will allow the user to instantly test how their applications work on the actual device or an emulated version.

Scripts to ease deployment of and debugging built applications inside Scratchbox+rootstrap to emulated and actual devices.

It's much better when you can test on an emulated device instead of dealing with a rootstrap trying to act like it's a device. Easy development ability is needed - it's a question that's often seen when people start developing for Maemo (how do we transfer to the device? Try see Visual Studio's deployment ways)

Work could be done on easy deployment of developed packages (# sb-copy-to-tablet -i usb, -i emu, etc) or running in similar manner to sbrsh with NFS and the likes.


Packages for Maemo should not expect Scratchbox to be where they're compiled (There's been seen examples of this amongst Nokia code.). Tis increases portability of Maemo platform. A package that builds in Scratchbox should also build on a tablet.

Maemo "R" SDK should be easy to install. It should be a matter of apt-get'ing a package or installing a VMware appliance for non-Linux users. It's alright for device-specific dev packages & emulator images to require a license to download, but the base Maemo "R" SDK should be easily installable and allow you to start developing for the base Maemo "R" platform immediately. You shouldn't have to go through hoops to get Scratchbox + Rootstrap + Generic Maemo device emulator image and such, - this attracts developers.

Embrace VMware Applicances/VirtualBox with SDK + Emu inside, for windows / other OS users to get developing for Maemo "R". Linux isn't the only platform people use for development.

Retrieved from "http://bugs.maemo.org/Maemo_Reconstructed"