Bug 7019 - Release source code of "getbootstate"
: Release source code of "getbootstate"
Status: RESOLVED WONTFIX
Product: Licensing Change Requests
General
: 5.0/(3.2010.02-8)
: All Maemo
: High enhancement with 28 votes (vote)
: ---
Assigned To: Quim Gil
: licensing-requests
:
:
:
:
  Show dependency tree
 
Reported: 2009-12-16 04:26 UTC by Jeff Moe
Modified: 2012-05-22 01:46 UTC (History)
9 users (show)

See Also:


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description Jeff Moe (reporter) 2009-12-16 04:26:30 UTC
SOFTWARE VERSION:
5.0
1.2009.42-11

getbootstate 1.0.35+0m5

EXACT STEPS LEADING TO PROBLEM: 
1. Brick stock system by filling up to 100%
2. Recover
3. Rebuild kernel with framebuffer console enabled so you can see WTF is going
on.
4. Happily use this kernel for days.
5. Re-brick system, likely due to filesystem 98% full.
6. Boot up and see on console WTF is going on.
7. See error:

getbootstate: Unexpected reset count 50 exceeds maximum (51)
getbootstate: Entering state 'MALF'.
getbootstate: Houston, we have a problem, powering off...

8. Go over to SDK
9. apt-get source getbootstate


EXPECTED OUTCOME:
Download getbootstate source code so I can investigate further.


ACTUAL OUTCOME:
The source is unavailable:
[sbox-FREMANTLE_ARMEL: ~] > apt-get source getbootstate
Reading package lists... Done
Building dependency tree... Done
E: Unable to find a source package for getbootstate

I assume it is closed source, but I don't know this for sure. I can't seem to
find the code anywhere.

REPRODUCIBILITY:
100%

OTHER COMMENTS:
The "second dot lit" hang at boot has been mentioned by a few people. Just
while filling out this ticket someone had it and was busy reflashing (talking
on IRC). They said they had 65M full at the time.

Anyway, it would be nice to have the source to the full stack. Please make the
code public.

Let us help you for free. :)

User-Agent:       Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.5)
Gecko/20091105 Fedora/3.5.5-1.fc12 Firefox/3.5.5
Comment 1 Jeff Moe (reporter) 2009-12-16 04:28:56 UTC
s/They said they had 65M full at the time./They said they had 65M FREE at the
time./
Comment 2 Andre Klapper maemo.org 2009-12-16 04:34:27 UTC
Confirming that this is closed source.

This is a very small application that has only a few lines of code.
For general info, also see http://wiki.maemo.org/Why_the_closed_packages
Comment 3 Jeff Moe (reporter) 2009-12-16 05:38:59 UTC
After going through the DSM code that you *did* release
(https://garage.maemo.org/projects/dsm/), I was able to glean a bit of info.

I decided to give this a shot:
sudo ./flasher-3.5 --enable-rd-mode

With this, I was able to recover my device *WITHOUT REFLASHING*. It also showed
that it wasn't a full / filesystem that kept my system from booting.

Note to children playing at home: DO NOT LEAVE YOUR SYSTEM IN R&D MODE. It will
cause weird side effects (such as making your keyboard flash), so you should
disable it. To disable it run:

sudo ./flasher-3.5 --disable-rd-mode

(If you don't have sudo set up, just run the command as root.)

I have a feeling running that "--enable-rd-mode" is going to save a lot of
people from reflashing. I *only* thought of that because I was able to see the
DSM code you did release. So release ALL of it, please and it will help you
even more. Please.

With sugar on top. Plz plz plz.  :)


P.S. You can see images of my hung system, pre --enable-rd-mode here:
http://www.freemoe.org/users/jebba/MALF
I can upload them to bz if you care as that URL may not be there forever.
Comment 4 Andre Klapper maemo.org 2009-12-16 18:16:48 UTC
Bug 6350 had its underlying root issue in getbootstate.
This would have been found probably earlier if there had been an option for the
community to inspect this small piece of code - See bug 6350 comment 36.
Comment 5 Eero Tamminen nokia 2009-12-17 19:14:25 UTC
> 3. Rebuild kernel with framebuffer console enabled so you can see WTF is going on.

Thanks for debugging this!!!

I don't think this would have been found in our internal investigation as the
first step to debug these kind of things would most likely have been to enable
RD mode (to get serial working), at which point the bug would have
disappeared...


Now that the bug root cause is known, fixing it doesn't actually require
sources, you can just edit the device /sbin/preinit script (see bug 6350
comment 39).
Comment 6 Zack Morris 2009-12-17 20:30:20 UTC
My bug is related because I am starting up in RD mode and I checked the
filesystem and it is not full. When I power down the device, it goes right back
to the repetitive boot state. Need code to debug.
Comment 7 Eero Tamminen nokia 2009-12-18 14:54:52 UTC
(In reply to comment #6)
> My bug is related because I am starting up in RD mode and I checked the
> filesystem and it is not full. When I power down the device, it goes right
> back to the repetitive boot state.

If you want to debug the bootup, it's better if you start with a kernel like
what Jeff used.

Getbootstate just provides information about device boot state.  The actual
decisions on what to do based on the boot state are done in the scripts calling
getbootstate (/sbin/preinit and /etc init scripts).  Those scripts you can
already read and modify.
Comment 8 Josh Triplett 2010-01-24 21:51:29 UTC
Poking this bug as the discussion seems to have died off and I don't see any
indication of a forward to an internal bug-tracker.
Comment 9 Eero Tamminen nokia 2010-01-25 14:28:28 UTC
(In reply to comment #8)
> Poking this bug as the discussion seems to have died off and I don't see any
> indication of a forward to an internal bug-tracker.

Open sourcing something is not really a bug from our internal bug tracker point
of view.  Quim (whom this is assigned to) looks after that kind of stuff.
Comment 10 Andre Klapper maemo.org 2010-02-16 19:57:39 UTC
(Moving all Licensing Change Requests to a separate Product in Bugzilla.
Sorry for the bugmail noise)
Comment 11 Carsten Munk maemo.org 2010-02-18 16:49:06 UTC
Temporarily setting this to priority unspecified while it gets evaluated.
Comment 12 Carsten Munk maemo.org 2010-02-19 10:09:11 UTC
Jeff, could I ask you to fill in the form at
http://wiki.maemo.org/Open_development/Licensing_change_requests and comment it
to this bug report? I figured if I wrote it myself my evaluation would be
rather biased.

After that is done, I'll do an immediate evaluation to get this on the road -
this is a prime case where we will need to negotiate a bit/seek compromise as I
forsee some problems.
Comment 13 Jeff Moe (reporter) 2010-02-23 13:40:32 UTC
Per: http://wiki.maemo.org/Open_development/Licensing_change_requests

What component(s) or source packages/etc is the licensing change request about? 

getbootstate

What component area is the component in if you know? (See the openness reports
at
http://mer-project.blogspot.com/2010/02/mapping-openness-of-maemo-50-pr11-and.html)

Perhaps Core/core-bootstrap

What is the current licensing of the component?

Exact license unknown, but something proprietary.

What licensing would you like it to be and why?

GPL and publicly developed. This is a critical small application which has
played a part in one of the worst bugs possible (bricking of device). It also
plays a key role in the bootup process. It needs to be vetted for more bugs and
possibly optimized to speed up booting.

What project(s) would have benefit from this licensing change request?

Maemo & MeeGo.

What technical purpose do you/your project(s) have for wanting the licensing
change?

1) Shed more light on what is actually going on when the device boots up.

2) Possibly speed up bootup.

3) Check for other bugs.
Comment 14 Carsten Munk maemo.org 2010-02-23 14:22:48 UTC
Evaluation:

Verification not a duplicate request: OK
Technical purpose: 

"This is a critical small application": 

OK, for reasons seen below (critical, small)

"which has played a part in one of the worst bugs possible (bricking of device) 

OK: was a problem for community to help find the problem mentioned in bug
report MB7019.

"It also plays a key role in the bootup process." - OK, preinit from
getbootstate package is the first thing in the bootup process. getbootstate
decides what branch (USER, LOCAL, ACTDEAD, etc) the bootup process takes.

"It needs to be vetted for more bugs" - This is a subjective opinion, but
having further openness in the bootup process and ability to pass around
patches legally in community and look at preinit and getbootstate would help
these things.

"and possibly optimized to speed up booting." - OK, community can not currently
pass around patches legally for optimizing the preinit script even though it is
readable on devices.

Evaluating for priority.

No open source replacements existing: Positive, none exists that covers both
preinit and getbootstate. 

Aiding open source replacements: Positive, aids community development in the
bootup process and ability to build this package on their own and modify it.

Closed package reasoning: 

Brand: Positive, no brand involved beyond behaviour (the actual branching
happens in open source package 'upstart')

Differentiation: Positive, no appearant reason for differentiation.

Legacy: Positive, getbootstate is still maintained.

IPR & licensing issues: Negative, please see comment below

Security: Negative: getbootstate has code that accesses battery information
(not alteration of state). This is a bit touchy subject (see BME reasoning). Re
IPR, code accesses battery information through ADC interface which I am not
certain if it is a protected 

Third party: Positive, but not sure about issues mentioned above.

Relicensing reasons: 

   1. Fixing a bug: N/A not relevant in this licensing request
   2. Nurturing application development: Positive, ability to edit early bootup
process may help developing interesting applications ('make a full snapshot of
my N900' and other things). Such a patch to preinit was made with regards to
bootmenu.sh and there's a lot of interest in this patch.
   3. Spread of Maemo driven technologies to other platforms: Positive, but on
basis of helping alternative OS'es on Nokia devices as preinit and getbootstate
helps developing proper bootup on Nokia devices.
   4. Community maintenance: Component is actively maintained but may help to
have more hands involved. Suggesting gitorious.org placement for the component.
   5. Better architecture: Closed source package in the very beginning of the
bootup process (the very first, too)

One or more projects: Mer, MeeGo and most alternative OS'es on the device, as
well as Maemo-on-OMAP - we have to use a 'fake getbootstate' with closed
licensing there.

OK, so, conclusion:

There are some potential security and IPR issues.

There's two ways this can be fixed:

My preferred (since it opens getbootstate too):

Move battery information related methods into a seperate library (closed
source, maybe non-free but redistributable to aid alternative OS'es. Similar
licensing to libspeexdsp, maybe. BSD licensed but closed source)

Build-dep/depend on that library in getbootstate package and GPL it.

Not so preferred (keeps getbootstate closed and I believe it is valuable to
know how getbootstate decides what state to take, as the original ):

Seperate preinit script into 'preinit' package in order to help early bootup

Keep getbootstate binary closed.


Final opinion:

I'm setting priority HIGH as it has high relevancy and it is worth getting an
opinion on system level things such as this. It is a closed component right at
the startup and decides a lot about how the system will proceed in the bootup
sequence. It contains information that is valuable to be able to use in
alternative OS'es on the Nokia devices and may even help Maemo-on-OMAP (beagle,
zoom2). 

This priority comes with the reservation about security and IPR as we need to
look into that. From a technical point of view, separation to avoid these
issues should be trivial in terms of packaging and build. 

Proposal for licensing:

getbootstate, preinit: GPLv2
library for battery information: re-distributable in order to encourage use of
it on alternative OSes.
Comment 15 Jeff Moe (reporter) 2010-07-13 23:56:36 UTC
NOK8
Comment 16 Quim Gil nokia 2010-08-25 17:27:50 UTC
I have been told that, from an architectural point of view, opening
getbootstate requires opening libcal in the first place. This makes things a
bit more complicated. Still asking around.
Comment 17 Quim Gil nokia 2010-08-26 21:03:31 UTC
More info: "libcal dependency comes from checking the R&D mode (called only
once and the real need for that is questionable). Might be worth considering
removing this dependency (since it's trivial)."
Comment 18 Jonathan Wilson 2011-01-19 10:10:43 UTC
Is it possible to open this or is there still sensitive cal-related information
not contained in the libcal-dev package?
Comment 19 Quim Gil nokia 2011-01-19 10:38:24 UTC
At this point I don't expect any Maemo 5 specific opening since all the open
source development is focusing on MeeGo. 

At least the problem of the original reporter was solved without needing the
source code of this component.
Comment 20 Jeff Moe (reporter) 2011-01-19 16:44:57 UTC
(In reply to comment #19)
> At this point I don't expect any Maemo 5 specific opening since all the open
> source development is focusing on MeeGo. 

Has Nokia ever opened *any* code that was placed in the "Licensing Change
Request Queue"? I'm not aware of *anything ever* being opened via this
distraction.

http://wiki.maemo.org/Open_development/Licensing_change_requests
Comment 21 Quim Gil nokia 2011-01-19 16:50:23 UTC
Every step has been done with the best of the intentions of the people
involved. Then things turn in directions unexpected. I know this doesn't change
the end result but just to let you know.
Comment 22 Josh Triplett 2011-01-20 11:19:26 UTC
(In reply to comment #21)
> Every step has been done with the best of the intentions of the people
> involved. Then things turn in directions unexpected. I know this doesn't change
> the end result but just to let you know.

(Happy to take this to a more appropriate forum or to email, but this seemed
like the best place to respond to this particular comment for now.)

Given that Maemo 5 has more or less become end-of-life, does any fundamental
reason exist to *not* release more of its code openly?  Most of the reasons
previously given for not opening code don't seem to apply anymore, with the
exception of "it might take work on Nokia's part".  I suspect that the Maemo
community has no shortage of people willing to do the work of reviewing things
and looking for what could get released without causing any problems with
third-party IP and such, if that could be arranged, saving Nokia the time and
trouble.  Any particular barriers that would prevent something like this?
Comment 23 Andre Klapper maemo.org 2011-01-20 11:23:27 UTC
See http://talk.maemo.org/showpost.php?p=784443&postcount=32
Comment 24 Josh Triplett 2011-01-20 11:38:22 UTC
(In reply to comment #23)
> See http://talk.maemo.org/showpost.php?p=784443&postcount=32

Ah.  Thanks for the pointer; that helps.
Comment 25 Pali Rohár 2011-05-01 14:01:25 UTC
I found source of getbootstate in meego dsme tree:
https://meego.gitorious.org/meego-middleware/dsme/blobs/master/getbootstate/getbootstate.c

Can somebody confirm if this is same code as in maemo framanle?
Comment 26 Jeff Moe (reporter) 2011-05-02 18:14:48 UTC
(In reply to comment #25)
> I found source of getbootstate in meego dsme tree:
> https://meego.gitorious.org/meego-middleware/dsme/blobs/master/getbootstate/getbootstate.c
> 
> Can somebody confirm if this is same code as in maemo framanle?

I searched the git logs for December 2009 and following months for a fix
related to this bug, but I don't see anything. The getboostate.c code does
appear related though.

Had issues like this been properly addressed in their proper time, perhaps NOK
wouldn't have lost $5 billion market capitalization. Just sayin'. NOK9.
Comment 27 Quim Gil nokia 2011-05-02 22:50:23 UTC
(In reply to comment #25)
> Can somebody confirm if this is same code as in maemo framanle?

I just sent an email to the authors specified in the source code link above
(thanks for the URL).
Comment 28 Eero Tamminen nokia 2011-05-04 17:06:39 UTC
(In reply to comment #26)
> (In reply to comment #25)
> > I found source of getbootstate in meego dsme tree:
> > https://meego.gitorious.org/meego-middleware/dsme/blobs/master/getbootstate/getbootstate.c
> > 
> > Can somebody confirm if this is same code as in maemo framanle?

At least it looks familiar.

> I searched the git logs for December 2009 and following months for a fix
> related to this bug, but I don't see anything. The getboostate.c code does
> appear related though.

I guess the code for Harmattan/MeeGo could have been branched before that...
Comment 29 Pali Rohár 2011-05-04 17:26:34 UTC
So it is possible to replace version in n900 with this code?
And if this is not same, what is difference between n900 version and this
(meego) code?
Comment 30 Eero Tamminen nokia 2011-05-05 18:50:24 UTC
(In reply to comment #29)
> So it is possible to replace version in n900 with this code?

No idea.

> And if this is not same, what is difference between n900 version and this
> (meego) code?

The meego one seems almost a rewrite:

$ wc -l fremantle-2010-01/getbootstate.c meego/getbootstate.c
  524 fremantle-2010-01/getbootstate.c
  511 meego/getbootstate.c

diff -ub fremantle-2010-01/getbootstate.c meego/getbootstate.c|grep '^[-]'|wc
-l
380
diff -ub fremantle-2010-01/getbootstate.c meego/getbootstate.c|grep '^[+]'|wc
-l
367
Comment 31 Eero Tamminen nokia 2011-05-06 11:01:18 UTC
Quim got more information on this:

"Maemo5 and Harmattan getbootstate are quite similar but with one major 
difference: in maemo5, getbootstate accesses the battery information. In 
Harmattan, battery checks are moved to the boot loader (NOLO)."

So unfortunately the Harmattan getbootstate cannot be used as a drop-in
replacement in Fremantle but you can get hints how the bootstate logic works
from the open source version. The rest you could find out using strace & other
debugging tools, if you need to figure out how it works.
Comment 32 Pali Rohár 2011-05-06 13:05:49 UTC
And what fremantle getbootstate chacking with battery? Now we have open source
bq27x00_battery kernel module which can access to battery information.

bq27x00_battery is backported to kernel-power v47, but can be compiled with
stock nokia kernel too. See:
http://talk.maemo.org/showpost.php?p=925980&postcount=674
Comment 33 Eero Tamminen nokia 2011-05-06 16:45:15 UTC
(In reply to comment #32)
> And what fremantle getbootstate chacking with battery?

To shutdown the device for safety reasons if battery is of incompatible type.
Comment 34 Eero Tamminen nokia 2011-05-06 16:48:45 UTC
Type seems to be in /sys/power/battery_type file.
Comment 35 Pali Rohár 2011-05-11 22:58:14 UTC
File /sys/power/battery_type does not exist in default stock Nokia kernel (and
not in power kernel too)
Comment 36 Pali Rohár 2011-06-07 15:51:45 UTC
Patched harmattan open source version which is compatible with fremantle on
n900 is here:
http://talk.maemo.org/showthread.php?t=73792
Comment 37 Eero Tamminen nokia 2011-06-07 16:23:46 UTC
(In reply to comment #36)
> Patched harmattan open source version which is compatible with fremantle on
> n900 is here:
> http://talk.maemo.org/showthread.php?t=73792

"And last if it not set R&D mode (yes, only here it is used) set state to
SHUTDOWN and write error."

One of the purposes of RD mode is to be able to boot devices with fake
batteries used on development boards.  AFAIK with normal batteries this
condition might be dangerous, so that's why the device is shutdown on this
condition unless you've explicitly enabled RD mode (= "I know what I'm doing
and this isn't some el-cheapo exploding battery").
Comment 38 Pali Rohár 2011-06-07 16:28:39 UTC
Yes, this sounds good. In madde qemu is enabled R&D mode (same reason). When i
disabled R&D mode in madde qemu I was not able boot it.
Comment 39 Jeff Moe (reporter) 2012-05-22 01:46:33 UTC
NOK3