Bug 6720

Summary: Add the ability for Flasher and NOLO to download data
Product: [Maemo Official Platform] Flasher Reporter: Darryl L. Miles <darryl-mailinglists>
Component: otherAssignee: unassigned <nobody>
Status: RESOLVED WONTFIX QA Contact: flasher-other-bugs
Severity: enhancement    
Priority: Low CC: andre_klapper, darryl-mailinglists, david.hagood, eero.tamminen, maemo.org
Version: 5.0/(1.2009.42-11)   
Target Milestone: ---   
Hardware: N900   
OS: Linux   

Description Darryl L. Miles (reporter) 2009-12-08 16:59:35 UTC
I use the term NOLO as I think that is the name of the software component
inside platform which communicates with flasher.


This facility would add command line options to Flasher to:
 * List/enumerate all ubifs volume names and their current sizes.

 * List/enumerate all other flash storage available and their current sizes
(mandatory covering: internal eMMC, optionally covering: microSD port)

 * Request download of contents from a location (optionally specifying the
offset and length, if they are omitted obvious behavior occurs, like offset is
the start/base/0 location and the length is the remainder of the data left in
that storage area/volume/segment).



Suggested example of usage (fictitious numbers and values shown) :

$ ./flasher --list-ubifs-volumes
ubifs:        size=256Mb  (raw whole image data view, ubifs internals visible)
ubifs:kernel  length=0x00023c00 raw_offset=0x00000000
ubifs:initfs  length=0x000f0000 raw_offset=0x00023c00
ubifs:rootfs  length=0x00bc0000 raw_offset=0x0011ec00
ubifs:swap    length=0x00c00000 raw_offset=0x00cdec00

$ ./flasher --list-flash-storage
ubifs:        size=256Mb
emmc:         size=32Gb
microsd0:     size=16Gb

# --download-spec has <storage_area>:<volume_label>:<offset>:<length>
# --outfile-file is the host file to write the data to
# --outfile-offset causes a seek into that host file
./flasher --download-spec ubifs:rootfs:0x0:0x00bc0000  --outfile-file
/tmp/mydata.dat --outfile-offset 0

# example for emmc, the spec "emmc:::" is the same as "emmc", there is
#  never any volume id (unless NOLO can understand partitioning)
#  having the meaning downloading the entire internal eMMC to file.
./flasher --download-spec emmc:::  --outfile-file /tmp/mydata.dat


The purpose of this is to allow:
 * Data recovery (yes someone will then sort out the ubifs recovery tooling!)
 * Postmortem analysis of so called "bricked" devices
Comment 1 Andre Klapper maemo.org 2009-12-08 18:55:47 UTC
Thanks for reporting this.

Sounds a bit related to bug 2984 to me.
Comment 2 Darryl L. Miles (reporter) 2009-12-11 15:32:54 UTC
(In reply to comment #1)
> Sounds a bit related to bug 2984 to me.

I agree it is similar, I believe the feature request contained here is more
complete (i.e. general and broader) and contains everything asked in that other
bug.  This bug is (hopefully) easier to understand what is being requested.


This request attempts to encapsulate which parts of the closed system could be
improved so that self-help is pushed back into the hands of developers and away
from Nokia's "Return Merchandise Authorization" department (which has an
obvious cost to Nokia).

I think developers would appreciate knowing they can get their data back out of
their device (no matter what state the flash is in).  Many developers only have
access to one device (which they paid for with their own money).  So its not
like a Nokia employee who probably might have one device to use and one device
to crash-test.



I have also digested the thoughts in the other bugs on data locking.  My
thoughts on that are that other mechanisms should be used to provide that
protection (such as storage encryption).  The thought that denying the ability
to download data in anyway addresses that security problem is misguided.

Discussion on that problem is best made in its own problem domain "Security
Storage API" this is something else I have collected my thoughts to make an API
request.
Comment 3 Eero Tamminen nokia 2009-12-15 18:29:35 UTC
(In reply to comment #2)
> I think developers would appreciate knowing they can get their data back out
> of their device (no matter what state the flash is in).

Except for some settings, the user data is on eMMC.  AFAIK accessing that (e.g.
flashing it) is not done/supported by the bootloader.

I don't think you necessarily need specific support for something like this in
the bootloader, it could be enough just to tell to flasher to put given kernel
+ initrd to RAM which does the rescue stuff, and tell flasher to boot that
that, like one could use Oprofile in Diablo:
  http://maemo.org/development/tools/doc/diablo/oprofile/

(I think it should work also with Fremantle.)
Comment 4 Josh Triplett 2009-12-20 00:23:38 UTC
*** Bug 2984 has been marked as a duplicate of this bug. ***
Comment 5 Darryl L. Miles (reporter) 2010-01-13 16:00:56 UTC
(In reply to comment #3)
> (In reply to comment #2)
> Except for some settings, the user data is on eMMC.  AFAIK accessing that (e.g.
> flashing it) is not done/supported by the bootloader.
> 
> I don't think you necessarily need specific support for something like this in
> the bootloader, it could be enough just to tell to flasher to put given kernel
> + initrd to RAM which does the rescue stuff, and tell flasher to boot that
> that, like one could use Oprofile in Diablo:

Just to come back on this point.

I do believe the current flasher/bootloader does access the eMMC since there
are at least 2 kinds of flashing options.  UBIFS only (i.e. to the OneNAND) and
UBIFS+eMMC (which restores factory default settings and themes etc...)  So this
is why I do not think eMMC should pose a significant problem to implement.

I do not however know the details of exactly how the eMMC is re-flashed, maybe
some technical clarification could chime in.  Does the bootloader have a
"driver" to directly access it built into it already ?   Alternately it is
possible the UBIFS+eMMC is achieved by a re-flash of the OneNAND, then the
device is rebooted and the kernel allowed to run.  The flasher tool then talks
to the kernel to access and re-flash the eMMC.   I do not know the specifics.


I appreciate if you are saying that the "NOLO bootloader" does not currently
have a driver capable of accessing the eMMC (same comment applies to accessing
internal microSD port) and therefore in order to fulfill this bug dealing with
eMMC directly from bootloader would create a large amount of work (since a
driver would need to be added).

If this is the case then it would be appreciated if successive releases over
time would be made:
 * Provide full support for OneNAND/ubifs
 * Provide full support for eMMC (in a later release)
 * Provide full support for microSD (in an even later release, maybe a
threshold can be set, if after implementing the above 2 points this feature can
not be added with less than the amount of work it takes to create 250 new lines
of code then cut the feature, this is really a "nice to have")


The goal is to make recovery of data from the device close to single step (i.e.
as simple as possible).  So while I talk of "developers" making use of this
feature initially I would like to leave to door open as much as possible to
users being able to at least get a recovered image onto their PC workstation
which with the help of forums/community they might themselves be able to
recover their data.  Bootstrapping a temporary rescue kernel would be
undesirable.
Comment 6 Eero Tamminen nokia 2010-01-13 16:47:29 UTC
(In reply to comment #5)
> Bootstrapping a temporary rescue kernel would be undesirable.

Why?  That at least sounds doable, unlike the unrealistic request of
duplicating again large amount of currently kernel code in the bootloader (now
that we finally got rid of that duplication in Fremantle, the code is only in
kernel and user space).
Comment 7 Ax 2010-02-11 14:42:40 UTC
Hi's Guys - a use case scenario re this thread. 
I just got my n900, have had it for two weeks, I went NUTS on this device- way
cool happy with it on this level, it suites me in so many ways. Unfortunately I
did a boo boo which has put my n900 in a reboot loop and, based on my
knowledge, I believe I know exactly what I did you cause it, I also kept
backups of the files I change which caused the problem (and its way
embarrassing and I'm not going to say what I did). 
My fix, if I could access the rootfs would be to do a mv command and I'd be
back online again, but I can't. I don't want to re-flash, because I'll have to
start again, and I don't want to- the fix is too easy, If I could access the
rootfs.

Bug 2984 discusses security, but if the device was in an operable state I could
install a backdoor in seconds that would give me total access in minutes and
would be much easy than what I'm attempting now, so I don't believe that is a
real concern. 

If this was a full blown computer I'd be booting off a CD and back on track
very easyily, now I'm trying to achieve the same resolution (but of course
without the CD drive. 

If I could just pull and push data/filesystems that would easily solve my woes.
My current approach tonight will be to try and tell the kernel to use the SD
card as the rootfs and gain access to the real rootfs that way, and then do the
fix. If anyone would care to point me towards doco explaining why this wont
work, please do - as stated before, I've had this device for two weeks now. Any
other pointers towards the data structures/boot process/nolo would also be
appreciated. I'm assuming the base linux kernel parms are available I hope that
the modules required for the SD card are available via initrd.... 

Appologies if my reasoning is flawed, Maemo and the n900 are new to me, and I
thought I'd add my two cents before I move on from this thread
Comment 8 Andre Klapper maemo.org 2012-03-24 11:44:37 UTC
The Maemo 5 User Interface and Maemo 5 platform components (e.g. libraries)
used for the N900 are considered stable by Nokia and it seems that there are no
plans for official updates currently, hence nobody plans to work on this
enhancement/wishlist request. 
(And in case you feel like discussing this situation: Nokia Customer Care or
http://talk.maemo.org would be the place to do so as you will not reach Nokia
officials in this community bugtracker - though all of this is really no news.)

Reflecting this status by setting RESOLVED WONTFIX for this
enhancement/wishlist request (see
https://bugs.maemo.org/page.cgi?id=fields.html#status for status explanations).

There is a small chance for issues in those Maemo components that are open
source: Contributed patches could be included and made available in the Maemo 5
Community CSSU updates. 
The Maemo CSSU project is run by a small team of volunteers; see
http://wiki.maemo.org/CSSU for more information.
So in case that you can provide a patch that fixes the reported problem, please
feel encouraged to file a request under
https://bugs.maemo.org/enter_bug.cgi?product=Maemo%205%20Community%20SSU .
Please note: The Maemo CSSU project is not related in any way to Nokia.


( Tag for mass-deleting bugmail: [cleanup20120324] )