YAFFS2

Summary

The original motivation for YAFFS 2 was to add support for the new NAND with 2kB pages instead of 512-byte pages and strictly sequential page writing order.

To achieve this, a new design is used which also realises the following benefits:

• zero page rewrites, which means faster operation. (YAFFS1 uses a single rewrite in the spare area to delete a page).

• ability to exploit simultaneous page programming on some chips.

• improves performance relative to YAFFS1 speed(write:1.5x to 5x, delete: 4x, garbage collection: 2x)

• lower RAM footprint (approx. 25% to 50% of YAFFS1, depending on chunk size used).

• Can support Toshiba/Sandisk MLC parts.

• Runtime selection between various chunk sizes.

Most of YAFFS and YAFFS2 code is common, therefore the code will likely be kept common with YAFFS vs YAFFS2 being run-time selected.

Method

The main philosophical difference between YAFFS and YAFFS2 is how discarded status is tracked. Since we can't do any re-writing, we can't use the "discarded" flags in YAFFS2.

Instead YAFFS2 uses two mechanisms to resolve data state.

• YAFFS2 chunks have more tag information, including a block sequence Id. From that we can determine the chunk sequence Id since the chunks are allocated sequentially in a block. Thus we always know the patch order for all chunks in the system.

• The above helps us track stale -vs- fresh data, but does not help determine when a file/object is deleted. Deletion is achieved by moving the object to the "unlinked" directory. We also keep track of the number of chunks (both stale and current) in the system for each object. While this number indicates that there are still chunks associated with this object we keep the deletion record. When the last trace of the object has been really erased from NAND, we can forget about the deletion record too.

• Since there is no deletion, a resize (shrinking) of a file will still have valid data chunks past the end of file on the NAND. However, we write a new ObjectHeader at the time of the resize, therefore this shows the shrunken file size. The ObjectHeader also carries information to say that this is a shrink, for some special handling in the garbage collector.

This changes erasure slightly:

• During garbage collection we can't just look at chunk state flags, instead we must read the tags of each chunk to determine which object's chunk count we must decrement. This step must also be performed when a block is erased (as part of deletion). This is already done in YAFFS by soft deletion.

This makes erasure & garbage collection more expensive (by adding reads), but remember that ion YAFFS2 we don't need to do page deletions which are much more expensive operations. Thus, all-up YAFFS2 wins.

Resize Handling

In YAFFS, soft deletion is ued for everything but resizing (shrinking) a file which has some particularly ugly cases that can complicate garbage collection.

As mentioned before, we write a new ObjectHeader to indicate the shrinking. However, it is important that this ObjectHeader does not get destroyed (erased) before the data chunks that were discarded during the shrink are destroyed (erased). If this precaution is not taken then it is possible that the deleted chunks might be brought back to life.

The modification to the garbage collector is as follows:

• The block info flags are expanded as follows:

• containsShrinkObjectHeader: Indicates that one or more of the chunks in the block are object headers to indicate a file shrink.

• containsShrinkDataChunks: Indicates that one or more of the chunks in the block are data chunks discarded by a file shrink.

• Before we allow a block with containsShrinkObjectHeader set to be erased (garbage collected), we must first ensure that all the earlier blocks (ie according to the block sequence number) that have containsShrinkDataChunks set are erased (garbage collected) which ensures that the shrunk data chunks are deleted. The mechanisms to do this are as follows:

• If the garbage collector attempts to select a block with containsShrinkObjectHeader set, we check that the above criterion in met before selecting the block.

• Periodically select the oldest block with containsShrinkDataChunks for garbage collection.

Tag structure

Each chunk in YAFFS2 has the following information:

To get enough spare bytes for this tagging structure requires a chunk-size of at least 1kB. YAFFS1 is still used for 512-byte chunk sizes.

The blockSequence increments each time a block is allocated. (ie. the first block allocated is block 1, and so on).

Scanning

The only reason we need to keep track of data status on NAND is to be able to recreate the file system state during scanning. Since we no longer have chunk deletion status flags we use a slightly different process for scanning a YAFFS2 system.

In effect, YAFFS2 recreates its state by "replaying the tape". ie. it scans the chunks in their allocation order (block sequence Id order) rather than in their order on the media. This implies that at start up, the blocks must be read and their block sequence determined.

Performance

These numbers are indicative of relative performance. These only apply to the NAND data transfer and do not include other overheads.

As an example, read/write cycle times of 100nS are used (though NAND can typically do 50nS), "seek time" of 10uS and program time of 200uS. Mileage will vary.

NB x16 means using a 16-bit bus. Clearly this cuts down on data transfer time relative to an 8-bit bus.

Times for 2kB read(units of 1uS).

Times for 2kB writes(units of 1uS).

Times for 1MB delete (units of 1uS).

Times for 1MB of garbage collection at 50% dirty (units of 1uS).

MTD Interface

As mentioned before, YAFFS2 requires a chunk-size of at least 1kB to get a large enough spare area to support the increased size of the tags. This is not really a disadvantage, but should rather be viewed as an opportunity to exploit the NAND hardware more effectively. In particular:

• Newer, larger, NAND with 2kB pages can be used in chunks of 2kB. Keeping the relationship of one chunk per page improves robustness and performance (rather than say trying to "fake" 512byte pages). A block comprises 64x2kB pages.

• Some devices have 512byte pages, but are arranged as multiple "bit planes" which can be programmed and erased in parallel. For example, the Samsung K9K1G08U0M can support 4 simultaneous operations. YAFFS2 can exploit this by using 2kB chunks by using groups of 4 pages - one on each bitplane. Virtual blocks would be built which comprise 32x2kB chunks.

To this end, yaffs_guts is being re-crafted to support arbitrary chunk size (power of 2, >= 1024), with arbitrary block size.

Currently, YAFFS also makes some other assumptions which will need to be changed:

• Spare layout. Might need to be changed. This is relatively simple to shuffle around.

• Bad block detection. Currently YAFFS uses the SmartMedia detection (checks first two pages for bad block markers). Needs to be more flexible. eg. Sandisk/Toshiba MLC parts mark the last two pages in a block.

• ECC was on this list, but now seems flexible enough. Thanx Thomas.

Some of these differences can be absorbed in the yaffs_mtd layer. Some will need to be handles inside the mtd itself.

$Id: yaffs2.html,v 1.3 2003-09-16 06:48:38 charles Exp $