Reducing JFFS2 mount time

When it comes to choosing a file system for flash memory JFFS2 is top of the list for most projects. It is robust and works with most flash chips using NOR or NAND technologies. The problem is that JFFS2 partitions can be slow to mount and as flash memory sizes increase, so does the issue of the tine to mount.

There is a solution: "erase block summary" or EBS. It has been in Linux kernels since 2.6.15.

How it works

JFFS2 is a log-based file system, meaning that each change to a file is written to the flash memory as a node. There is no directory in the normal sense, but by reading all the nodes in sequence you can deduce one. Doing things this way makes the file system robust - either the last node was written or it wasn't, but in both cases the data seen by the user is consistent. But, it also makes mounting slow because of the time taken to read the nodes and construct a directory in memory.

EBS helps by writing a "summary" node at the end of each complete flash block so that at mount time you only have to read that node and not the entire block. If the summary node is missing, maybe because of a system power-down before it could be written to flash, nothing bad happens - JFFS2 just falls back to scanning the whole block as it would have done before.

The result is a reduction in mount time by a factor of 4 or 5. There are a few downsides as well, but in most cases they are acceptable

  • More RAM used to hold the summary information for the current erase block
  • Less storage space in the flash because the summary node takes some room

Seeing it work in practice

First you need a kernel with CONFIG_JFFS2_SUMMARY enabled. It is one of the options in the JFFS2 configuration in Filesystems->Miscellaneous filesystems. With that turned on, all filled ease blocks in your jffs2 file system will have a summary node.

Seeing is believing, so I wrote a program called nand_check to scan a flash partition and display a character for each erase block:

  B        Bad block
  .        Empty
  -        Partially filled
  =        Full, no summary node
  S        Full, summary node

Here is an example of freshly erased partition, with, as it happens, one bad block:

# /nand_check /dev/mtd4
Flash type is 4
Block size 131072, page size 2048, OOB size 64
16777216 bytes, 128 blocks
...........................B=...........
........................................
........................................
........

Now, copy some files and scan again

# mount -t jffs2 /dev/mtdblock4 /mnt
# cp -a /usr/lib /mnt
# /nand_check /dev/mtd4
Flash type is 4
Block size 131072, page size 2048, OOB size 64
16777216 bytes, 128 blocks
...........................B=...........
........................................
..................................-SSSSS
SSSSSSSS

You can see that blocks at the end of the partition have been written and that they all have summary nodes. There is one partially filled block (marked with a '-'), and of course that does not have summary information.

Preparing JFFS2 images: sumtool

It is quite common to create a jffs2 image file to "pre-load" flash with a set of files. The tools that does that, mkfs.jffs2, does not do anything about summary nodes, or to put it another way, by default you will end up with an image that takes a long time to mount. The secret is that you have to use a second tool from mtdutils: sumtool. Here is an example

mkfs.jffs2 -n -e 0x20000 -p 0x20000 -d rootdir -o rootdir.jffs2
sumtool -n -e 0x20000 -p -i rootdir.jffs2 -o rootdir-sum.jffs2

In both cases, I use '-n' to suppress cleanmarker nodes since this is for NAND flash and the cleanmarkers were created in the OOB area by flash_eraseall -j. The eraseblock size for this flash chip is 128 KB, so I add -e 0x20000. The -p option means pad the last block to the eraseblock size, -d is the directory to use for input files, -o is the output file.

Now lets write the first one to flash and take a look:

# flash_eraseall -j /dev/mtd4
# nandwrite /dev/mtd4 rootdir.jffs2
# /nand_check /dev/mtd4
Flash type is 4
Block size 131072, page size 2048, OOB size 64
16777216 bytes, 128 blocks
==================.........B=...........
........................................
........................................
........

Same thing using the second image

# flash_eraseall -j /dev/mtd4
# nandwrite /dev/mtd4 rootdir-sum.jffs2
# /nand_check /dev/mtd4
Flash type is 4
Block size 131072, page size 2048, OOB size 64
16777216 bytes, 128 blocks
SSSSSSSSSSSSSSSSSS=........B=...........
........................................
........................................
........

Pretty much as you would expect: the first has no summary nodes but the second does.

Mount times

These measurements were made using a Digi development board, with an ARM926EJ-S core running at 155 MHz, and a single 1 GiBit (128 MiByte) ST NAND flash chip with 2KB pages and 128KB erase blocks. I used three file systems

Size (MiB) No. files No. dirs Image size(MiB) Image size with summary (MiB) Overhead
Empty 0 0 0 N/A N/A N/A
Small 4.4 444 40 2.250 2.375 5.5%
Large 56 556 2515 22.125 23.125 4.5%

Here, Size is the size of the files and directories as reported by du -sh. The Image Size is that produced by mkffs.jffs2 using the default zlib compressor. The Overhead column shows the additional percentage of storage needed for the summary nodes.

Time to mount

No summary Summary Improvement
Empty 2.83 N/A N/A
Small 4.55 3.03 150%
Large 19.5 3.6 540%

It seems that there is a constant overhead of almost 3s just to do the mount. With the small file system this dominates so the mount time with EBS is only one and a half times faster. With the large file system the time spent scanning the blocks dominates and so the improvement is more than five fold.

References

http://www.linux-mtd.infradead.org/doc/jffs2.html has a brief description of JFFS2 summary nodes.
http://www.inf.u-szeged.hu/jffs2/mount.php also discusses EBS.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

uncorrectable error while running nand_check

Thanks for your ideas about jffs2 boot time.
Although I think I did everything right, I get uncorrectable error.
It's only in this partition (and mdt0 which uses HWECC).

I did a
flash_eraseall -j /dev/mtd8
Erasing 128 Kibyte @ cc0000 -- 30 % complete. Cleanmarker written at cc0000.
Skipping bad block at 0x00ce0000
Erasing 128 Kibyte @ 2960000 -- 99 % complete. Cleanmarker written at 2960000.

and then a

./nand_check /dev/mtd8
Flash type is 4
Block size 131072, page size 2048, OOB size 64
43515904 bytes, 332 blocks
........................................
........................................
uncorrectable error :
uncorrectable error :
..
uncorrectable error : .......................B=...............
........................................
........................................
........................................
........................................
........................................
............

BTW: I get no error mounting this erased partition as jffs2 ?!? What does this mean? Do I have another Bad Block which is not marked? How would you handle that?

Re: uncorrectable error while running nand_check

I'm pretty sure that message is coming from your kernel and not the nand_check program. If you look in drivers/mtd/nand/nand_ecc.c you will see that there is a function __nand_correct_data() which prints that error string when it is not able to correct an ecc error. Possibly you have some bad blocks that have not been marked as such yet, but since they are not being used to store any data they do not cause errors at mount time.

jffs2 padding

this is wrong:

mkfs.jffs2 -n -e 0x20000 -p 0x20000 -d rootdir -o rootdir.jffs2

you should not pad the jffs2 image to the eraseblock size.
The remaining pages in the last eraseblock will be written with 0xff, and when the jffs2 image is mounted, the jffs2 fs driver will assume those pages can be written to without first erasing the block. As a result, when you modify something in the filesystem, and those changes end up in the padded eraseblock, you will get errors.
Depending on the quality of your NAND, you might get away with it. Real SLC should allow a few writes between erase, but most SLC's nowadays immediately give you bit errors when you do a second write to a page.

No summary nodes afer a file copy

Hi,

I don't understand how to pass through my current situation:

I have a root fs partition of 62259200 bytes length:
1) from uboot I erase all the sectors from that partition (seems the block are not really cleared as I don't see any '.' character using your check program).
2) from uboot I load the root fs image (built with EBS), that image's size is 5614592 bytes
3) I copy a 13Mb file
4) I delete that file
5) I copy that file again

Here is that result of your program (modified to work with NOR flash):

# /sbin/nandcheck /dev/mtd4
Flash type is 3
Block size 131072, page size 131072, OOB size 0
62259200 bytes, 475 blocks
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS===================================
================================================================================
================================================================================
================================================================================
======================----------------------------------------------------------
---------------------------------------------------------------------------

Here is the effect I wanted to fix and why I wanted to have solution with EBS:

1) I erase my root fs from uboot (59Mb)
2) from uboot I tftp the root fs image (5Mb)
3) device boots up: 25sec
4) I copy 2 files of 14MB each to user space
5) device boots up : 45 sec !!
6) I delete those 2 files, still boots in 45sec !

How can I fix this behavior?

Thanks in advance

jffs2 NAND mounting time

Hi

I am using jffs2 fs for NAND partition (64MB) for the use of small data rw (single file size is 512 bytes).
I tested with EBS (CONFIG_JFFS2_SUMMARY), it significantly reduces the mount time from 30 minutes to 4 to 5 minutes for 99% filled 64MB partition. Thanks. Then I was expecting further reduced mount time after removing files for about 20M, but mounting time remain same.

For further iteration tests, I waited for the jffs2_gcd_mtd to optimize the total partition usage and did umount and mount. Even then mount remain same.

I tried with BCH_ECC kernel option also for the same result.

I did NAND erase and started filling for about 10%, 20%, 30% ... the mount time increase according to the total size.
So the problem is mount time not reducing if the files are reduced even after proper umount, reboot and mount.

How to fix this problem ?

Thanking You
syed

Re: jffs2 NAND mounting time

The thing is that when you delete a file with JFFS2 it only appends a node recording that transaction, but it does not modify the previous nodes. That is how log structured files systems like JFFS2 work. So, at mount time, there are the same number of nodes to be processed and the mount time remains the same. At some point, the garbage collector thread (jffs2_gcd_mtd) will attempt to free up some flash erase blocks, which should reduce the number of summary nodes to process at mount time. I'm not sure what the algorithm is, but it looks like it just keeps a small number of free blocks and stops when it has satisfied that criterion, and once again the mount time is largely unaffected.

How to fix the problem: don't use JFFS2 with such (comparatively) large partitions. Either divide the storage in to smaller (16 MB?) partitions, or better still use a different file system such as UBIFS instead.

Chris.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Youtube and Vimeo video links are automatically converted into embedded videos.
  • Lines and paragraphs break automatically.
  • Web page addresses and e-mail addresses turn into links automatically.
  • You may post code using <code>...</code> (generic) or <?php ... ?> (highlighted PHP) tags.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.