Bug 838 - Opera draws fonts as blocks when booting from mmc with ext2 fs
: Opera draws fonts as blocks when booting from mmc with ext2 fs
Status: RESOLVED WONTFIX
Product: X-Discontinued
Browser Opera engine (770/N800)
: 2.1
: All All
: Medium normal (vote)
: ---
Assigned To: unassigned
: opera-engine-bugs
:
:
:
:
  Show dependency tree
 
Reported: 2006-11-06 18:28 UTC by Frantisek Dufka
Modified: 2008-12-06 15:41 UTC (History)
2 users (show)

See Also:


Attachments
screenshots (418.17 KB, application/x-zip-compressed)
2006-11-07 16:58 UTC, Frantisek Dufka
Details
output of ls -lR (384.70 KB, application/x-zip-compressed)
2006-11-07 16:58 UTC, Frantisek Dufka
Details


Note

You need to log in before you can comment on or make changes to this bug.


Description Frantisek Dufka (reporter) maemo.org 2006-11-06 18:28:00 UTC
This is pretty weird but when cloning rootfs to mmc card using rsync and
booting
via bootmenu everything seems to work fine except Opera browser shows few
specific pages with all characters being blocks. 
see thread here for details and screenshots
http://www.internettablettalk.com/forums/showthread.php?p=24946#post24946

example URLs:

http://www.mono-project.com
http://www.ubuntu.com

99.9% of other pages work fine.

I understand the whole thing of booting system from mmc is unsupported but this
should be at least examined because it is very bizarre and should not happen
even in theory :-) Both filesystem contain exactly same data but when system is
booted from internal flash those pages work, yet when exactly same system is
booted from mmc it shows blocks instead of characters.

Is the Opera browser somewhat hardcoded that it expects the root filesystem to
be jffs2 or root device to be /dev/mtdblock4 or does it expect some specific
features of the filesystem?

Way to clone system and dual boot is described here
http://maemo.org/maemowiki/SardineDistro

I'm using cloned system to mmc for many months and never found any difference
in
behaviour of any application except this. Also like already said - tons of
other
sites work fine in opera when booted from mmc so this is really something
obscure. 

happens both in 1.2006.25-8 and 2.2006.39-14 but only when booting from mmc
card

I tried various filesystem mount options for ext2 filesystem like
noatime,nodiartime to make ext2 filesystem behaviour similar to jffs2 but there
is no difference. I double checked and cloned filesystem more times and even
reformatted partition and cloned to empty filesystem (rsync -avH --delete
source/ destination/)to make sure data is same but it still happens.
Comment 1 Marius Vollmer nokia 2006-11-06 20:53:20 UTC
Just some random questions for debugging:

 - What happens when you save the broken page to flash and let opera show it
   from there?

 - If you still see blocks, what happens when you boot to internal flash
   and use that opera to show the previously saved page?

 - What happens when you download the page with wget or on your PC and
   use the MMC opera to view it?

One difference between JFFS2 and ext2 is that JFFS2 does not support writable
mmap, but ext2 does.  Sounds far fetched, true....
Comment 2 Frantisek Dufka (reporter) maemo.org 2006-11-07 11:02:31 UTC
found workaround/solution, looks like rsync is the problem

When the filesystem is copied via GNU tar it does not happen. When done via
rsync it happens.

Details here http://maemo.org/pipermail/maemo-developers/2006-November/006113.html

Still it would be interensting to know where exactly is the difference and what
confuses Opera to do what is does.

Like mentioned in the mailing list I tried it multiple times both with rsync and
tar and tar makes it right and rsync doesn't. Yet the data in ordinary files in
both filesystems is same. I checked with something like

compare.sh:
#!/bin/sh
cmp $1/$3 $2/$3 || echo ERR $3

cd /opt
find -type f . | xargs -n 1 compare.sh /opt /floppy 

and there is no difference even with rsync so it has to be something in metadata
(permissions,ownership ... ?)

any tips how to check properly for metadata difference (except by using rsync)?
Comment 3 Tapani Pälli 2006-11-07 13:20:21 UTC
my guess is that somehow font associated files get corrupted and browser cannot
open (correct) font properly. Not all fonts contain all characters and usually
when printing character that is unknown/out_of_range in a font you get '[]' box.
Comment 4 Frantisek Dufka (reporter) maemo.org 2006-11-07 16:52:20 UTC
In addition to comparing file contens (which is identical) I compared output or
ls -lR and found no significant difference (output in attachment).

insignificant differences:
- both rsync and tar do not preserve symlink time so there are differences in
symlink timestamps
- jffs2 directories report size of 0 bytes
- rsync causes directories with lots of filenames to take more space bacause it
uses temporary files a lot (--inplace turns this off and does not help)
- tar doesn't copy socket /var/run/sdp but when removed after rsync copy it does
not make difference
Comment 5 Frantisek Dufka (reporter) maemo.org 2006-11-07 16:58:11 UTC
Created an attachment (id=131) [details]
screenshots
Comment 6 Frantisek Dufka (reporter) maemo.org 2006-11-07 16:58:55 UTC
Created an attachment (id=132) [details]
output of ls -lR
Comment 7 Frantisek Dufka (reporter) maemo.org 2006-11-09 09:25:35 UTC
As suggested in mailing list
http://maemo.org/pipermail/maemo-developers/2006-November/006122.html
I also tried to use tar for finding differences between filesystems cloned with
tar and rsync. The result is - there is no difference in any filesystem object
(file/directory/device/..)!

The only idea I have now is that the order in which files are created in
filesystem matters. Both tar and mkfs.jffs2 create somewhat 'sorted' filesystem
- files are created alphabetically. Rsync may not do it like this.

My bold theory is that there is a bug in Opera when enumerating available fonts.
Since the file enumeration in directory in filesystem probably does no guarantee
any order in which files are returned, there may be a bug which fails to open
specific font (like off by one error - last font is not read or something
similar). It can be that both mkfs.jffs2 and tar creates filesystem so that the
order of the files in directory is specific (alphabetical). Filesystem created
by rsync may change that order so Opera (silently) fails with different font and
the page looks OK. Opinions?
Comment 8 Jakub Pavelek nokia 2006-11-13 19:07:56 UTC
(In reply to comment #7)
Nice theory but I would look for the problem somewhere else. I remember trying
out different font sets earlier and there was no such issue with the Opera browser.

I do not work with the Opera browser but I'd consider looking into some Opera
cache files that might have gone missing/corrupt when you did this exercise and
it could cause the page to fail during rendering. It is hard to label this a
"bug" really ;-)
Comment 9 Frantisek Dufka (reporter) maemo.org 2006-11-13 19:51:20 UTC
(In reply to comment #8)
> I do not work with the Opera browser but I'd consider looking into some Opera
> cache files that might have gone missing/corrupt when you did this exercise and
> it could cause the page to fail during rendering. It is hard to label this a
> "bug" really ;-)

Also Tapali mentioned filesystem corruption. But as I mentioned previously it
looks like both filesystems are identical as for data content and metadata
(permissions,..) so please try to think about the problem taking this as a
precondition.

Filesystem corruption means both cmp and tar is wrong about reporting
differences and also that rsync is buggy. While this may be true I'd guess it is
highly unlikely. If you have some other suggestions how to test filesystem
differences if cmp and tar --diff (and also rsync) cannot be trusted, let me know.

Also (to prevent further questions) all those test are not done on 'live'
filesystem. I have two bootable partions on mmc so all tests are done when
booted from mmcblk0p2 and transferring data from /dev/mtdblock4 (jffs2) to
/dev/mmcblk0p3 (ext2). After cloning there is no filesystem difference with both
ways (tar vs rsync). After unmounting and rebooting to mmcblk0p3 clone made by
tar works, clone made by rsync (proved to be identical by both cmp at tar --diff
with original) fails (to show the page correctly).

As for corrupted opera cache I also tried to remove whole /home/user/.opera. It
was re-created after next Opera run but it didn't help. Is the cache somewhere else?
Comment 10 Frantisek Dufka (reporter) maemo.org 2006-11-13 21:38:48 UTC
> Just some random questions for debugging:
> 
>  - What happens when you save the broken page to flash and let opera show it
>    from there?

No difference.

> 
>  - If you still see blocks, what happens when you boot to internal flash
>    and use that opera to show the previously saved page?

Yes, that works. Saved pages are same when saved both when booted from mmc and
from internal falsh. Both show fine when booted from flash. Both show blocks
when displayed while booted from mmc. So the problem is while displaying. The
stored file is OK.
Comment 11 timeless 2006-12-21 12:14:14 UTC
i'd suggest playing w/ strace+ldd. normally you'd never hear me make that 
suggestion, but oh well.
Comment 12 Frantisek Dufka (reporter) maemo.org 2007-03-25 14:37:37 UTC
Got this issue again even with tar. The thing that broke it this time is the
dir_index attribute of ext2/ext3 filesystem. I've been finetuning system on mmc.
First I switched to ext3 which was fine. Then found about -O dir_index mkfs.ext3
option which should speed up enumeration of files in directories. Tried it and
fonts in Opera are wrong again. This time it is readable but only different font
is used, can be seen on http://www.internettablettalk.com site (again).

Looks like the hashing method implemented with dir_index changes order of files
again so the problem really appears to be related to enumeration order of files
in directory. 

When removing the flag when creating filesystem and cloning it in exactly the
same way, it works fine.

I guess it is really a bug to have dependency on such (undefined) order. If
order is needed then it should be sorted by hand.
Comment 13 Eero Tamminen nokia 2007-03-28 14:47:08 UTC
Maybe the different timestamps cause wierd behaviour with fontconfig?
It has a lot of symlinks in "/etc/fonts/conf.d" and might be doing
things differently depending in which order it reads them.
Comment 14 Frantisek Dufka (reporter) maemo.org 2007-03-28 15:14:52 UTC
(In reply to comment #13)
> Maybe the different timestamps cause wierd behaviour with fontconfig?

It does not look like it is related to filesystem metadata i.e. timestamps
(proved by some experiments mentioned above). dir_index flag of ext2/3 system is
enough to break it. Like said in comment #12 - "When removing the flag when
creating filesystem and cloning it in exactly the same way, it works fine."

One thing the dir_index does is changing order of directory enumeration.

http://www.redhat.com/archives/ext3-users/2004-September/msg00029.html
"This is because readdir() has to return
the directory entries in hash sort order.  In contrast, in a vanilla
ext3 filesystem, normally directory entries are added in the order
that they were created, and inodes are created in sequential order."


> It has a lot of symlinks in "/etc/fonts/conf.d" and might be doing
> things differently depending in which order it reads them.

Does Opera use system font architecture (freetype,fontconfig etc.) or does it
have its own (different,buggy?) version compiled in statically? I'm asking
because this bug seems to be specific to opera browser, fonts seem to be ok in
everything else, and also bug #263 suggests it is not affected by some font
configuration changes.
Comment 15 Eero Tamminen nokia 2007-03-28 17:30:25 UTC
> Does Opera use system font architecture (freetype,fontconfig etc.) or does it
> have its own (different,buggy?) version compiled in statically?

I think Opera uses the lower level stuff (fontconfig etc), but not the higher
level ones (Pango) as just adding Chinese font doesn't make it to show Chinese
text like happens with Gtk widgets.  You can check what the libopera library
links against with "/lib/ld-2.3.6.so --list /path/lib.so" (the device doesn't
have ldd).
Comment 16 Tilman Vogel 2007-09-14 00:12:25 UTC
I can add some detail: It seems to me that for these rare websites, opera for
some strange reason just picks the physically first font in /usr/share/fonts.
You can get the physical order by

find /usr/share/fonts/

On the flash rootfs, this gives

~ $ find /usr/share/fonts.good/
/usr/share/fonts.good/
/usr/share/fonts.good/SwaBI4nh.ttf
/usr/share/fonts.good/nokia
/usr/share/fonts.good/nokia/fonts.cache-1
/usr/share/fonts.good/nokia/DeviceSymbols.ttf
/usr/share/fonts.good/nokia/nosnb.ttf
/usr/share/fonts.good/nokia/nosnr.ttf
/usr/share/fonts.good/nokia/nscnr.ttf
/usr/share/fonts.good/nokia/nokiasmiley.ttf
/usr/share/fonts.good/truetype
/usr/share/fonts.good/truetype/fonts.cache-1
/usr/share/fonts.good/truetype/ttf-bitstream-vera
/usr/share/fonts.good/truetype/ttf-bitstream-vera/VeraSeBd.ttf
/usr/share/fonts.good/truetype/ttf-bitstream-vera/Vera.ttf
/usr/share/fonts.good/truetype/ttf-bitstream-vera/fonts.cache-1
/usr/share/fonts.good/truetype/ttf-bitstream-vera/VeraMono.ttf
/usr/share/fonts.good/truetype/ttf-bitstream-vera/VeraBI.ttf
/usr/share/fonts.good/truetype/ttf-bitstream-vera/VeraBd.ttf
/usr/share/fonts.good/truetype/ttf-bitstream-vera/VeraIt.ttf
/usr/share/fonts.good/truetype/ttf-bitstream-vera/VeraSe.ttf
/usr/share/fonts.good/truetype/ttf-bitstream-vera/VeraMoBI.ttf
/usr/share/fonts.good/truetype/ttf-bitstream-vera/VeraMoBd.ttf
/usr/share/fonts.good/truetype/ttf-bitstream-vera/VeraMoIt.ttf
/usr/share/fonts.good/SwaRI4nh.ttf
/usr/share/fonts.good/fonts.cache-1
/usr/share/fonts.good/SwaBR4nh.ttf
/usr/share/fonts.good/SwaRR4nh.ttf
/usr/share/fonts.good/NcrBI4nh.ttf
/usr/share/fonts.good/NtmBI4nh.ttf
/usr/share/fonts.good/NcrRI4nh.ttf
/usr/share/fonts.good/NtmRI4nh.ttf
/usr/share/fonts.good/NcrBR4nh.ttf
/usr/share/fonts.good/NtmBR4nh.ttf
/usr/share/fonts.good/NcrRR4nh.ttf
/usr/share/fonts.good/NtmRR4nh.ttf

Rsync copies subdirectories first, so this gives:

~ $ find /usr/share/fonts.broken/
/usr/share/fonts.broken/
/usr/share/fonts.broken/nokia
/usr/share/fonts.broken/nokia/DeviceSymbols.ttf
/usr/share/fonts.broken/nokia/fonts.cache-1
/usr/share/fonts.broken/nokia/nokiasmiley.ttf
/usr/share/fonts.broken/nokia/nosnb.ttf
/usr/share/fonts.broken/nokia/nosnr.ttf
/usr/share/fonts.broken/nokia/nscnr.ttf
/usr/share/fonts.broken/truetype
/usr/share/fonts.broken/truetype/ttf-bitstream-vera
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/VeraBI.ttf
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/Vera.ttf
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/fonts.cache-1
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/VeraBd.ttf
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/VeraIt.ttf
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/VeraMoBI.ttf
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/VeraMoBd.ttf
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/VeraMoIt.ttf
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/VeraMono.ttf
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/VeraSe.ttf
/usr/share/fonts.broken/truetype/ttf-bitstream-vera/VeraSeBd.ttf
/usr/share/fonts.broken/truetype/fonts.cache-1
/usr/share/fonts.broken/fonts.cache-1
/usr/share/fonts.broken/NcrBI4nh.ttf
/usr/share/fonts.broken/NcrBR4nh.ttf
/usr/share/fonts.broken/NcrRI4nh.ttf
/usr/share/fonts.broken/NcrRR4nh.ttf
/usr/share/fonts.broken/NtmBI4nh.ttf
/usr/share/fonts.broken/NtmBR4nh.ttf
/usr/share/fonts.broken/NtmRI4nh.ttf
/usr/share/fonts.broken/NtmRR4nh.ttf
/usr/share/fonts.broken/SwaBI4nh.ttf
/usr/share/fonts.broken/SwaBR4nh.ttf
/usr/share/fonts.broken/SwaRI4nh.ttf
/usr/share/fonts.broken/SwaRR4nh.ttf
~ $

So, in this case, the logo font which has lots of blocks instead of characters
is first instead of SwissA. ;-)

That's easy to fix: Make a new fonts.new dir, copy in your favourite font
first, then the rest. Rename fonts to fonts.old, then fonts.new to fonts.
Enjoy. No reboot, only opera restart needed.

BTW ubuntu.com is not a test case anymore, but mono still is.
Comment 17 timeless 2008-05-30 13:29:30 UTC
sadly, this bug was never migrated internally, so it wasn't actually reported
by us to Opera. If you want to help them, you could grab Opera 8/8.5 for x86
and try to reproduce the problem with a desktop environment. If you can, they
should accept bug reports at https://bugs.opera.com/

as we're no longer shipping opera this bug will not be fixed.