Logs in Flash

Logging to flash has advantages over logging to RAM, such as surviving power loss. There are also disadvantages, such as performance impact.

Logging to flash is an interesting middle ground between logging into RAM and logging into a file. Certainly, flash can be the medium behind a file system, but you can also manage raw data. At least two blocks would need to be allocated for logging in flash to maintain an A-B pair to alternate between.

Advantages:

• Vast storage area

• Persistent Memory

Disadvantages:

• Complexity

• Speed

• Requires logging task

• Require flash interface bandwidth

• Reduced MTBF of flash with increased erase cycles

Managing flash blocks for logging

Flash has a critical limitation. Though you can write in small chunks, you can only erase in blocks. Depending on the technology, the smallest memory you can erase is 16KB in NOR flash and 128KB in NAND flash. As you can see, you will not want to perform a read/modify/write every time you append a log to the current block. The usual method is to write to the flash incrementally.

Once you have filled the first block you are logging in to, you switch to the second one. Typically, when statically allocating blocks, you will use them sequentially, wrapping at the end. Eventually, you will erase the oldest block and start logging new data there. Such circular FIFO schemes are common in software. Still, flash memory adds a "lumpiness" in performance due to the coarse granularity of the minimum erase block size limits and how long erase takes.

Erasing a block is slow. It would be best to plan to erase the oldest log block as a background action, sending the erase command but not waiting for the completed state. Such asynchronous commands allow the main CPU to work on another task while the flash's internal controller erases for you. Starting the erase at a high watermark, rather than a full condition, will prevent a log write from becoming a blocking transaction.

The flash device used for logging will likely be a shared resource. You must have thread-aware code that supports multiple users of the flash resource. The flash manager may be one of several users of the bus on which the flash resides. The bus will be SPI or I2C and might be a shared resource.

This brief overview shows the complexity of logging into flash and how it is very invasive on the system. If you are using the right development platform, a logging framework and flash and bus control software will be available. If the SW stack is in place, your life as a client of the logging system is relatively easy.

Reading the log from flash

There are several ways to retrieve logs from flash storage. You may support flash log retrieval in your code. You may require external tools to pull the image from flash. Tools, like STM32_Programmer from ST, provide a mechanism to access external flash attached to an ST MCU. You may connect directly to your flash with an external SPI or I2C controller and software. While you hold the MCU in reset, the external tools can retrieve the logs. TotalPhase provides the tool Flash Center to work with their Cheetah and Promira devices. You may also create your own external flash tool, a lightweight application that you can load into your MCU RAM and XIP (execute in place) to retrieve logs. The Binho is another versatile host protocol adaptor that provides similar functionality.

You will have to identify the flash blocks that contain the log. You also have to determine the current, in-use block. If you are using a simple A-B block allocation (or A-B-C, etc.), the current block has erased state at the end (typically all data = 0xFFFFFFFF). The last non-erase data is the newest record. With a pre-emptive erase, you will find two blocks that are wiped at the end but only one that has data at the beginning.

If you have been using dynamically selected blocks chained together, you must find the block headers and re-assemble the chain. Again, the top of the partial block will be erased.

Whether chains have fore- and back-references or are uni-directional is implementation-dependent. When the oldest block is erased, you might not clean up the next block's back pointer. A whole book could be written about this. You will be familiar with linked lists and chained data structures as a software engineer.

Flash erase cycle limit

There is only a finite number of times you can erase flash before it becomes unusable. We need to limit the frequency at which we erase the device. The more blocks you use for the logs, the less often you erase.

As a best practice, resume logging where the previous session left off; don't erase at the beginning. You can always add a session begin message as the first write after booting.

Be conservative in how many logging statements you place in your code. You will need plenty of log messages as you start development, but clean out the stale output as a code module matures.

Wear leveling

Wear leveling techniques can be applied to reduce problems with erase cycles. Leveling may be built in when logs are maintained within a flash file system. If you are allocating blocks for writing directly to flash, a round-robin usage of several blocks is a typical method. These blocks are pre-allocated for logging. You can use a more complex flash manager to allocate random blocks to the log. Your logging system then has to make sure discarded blocks are released. You will lose some of the log space to descriptor blocks showing how the log is chained through the flash.

Systems with constrained flash typically have a boot sector and two executable partitions to ping pong between firmware updates. There may be some space allocated for configuration. In a system like this, you are unlikely to have more than two blocks for logging, and wear leveling may not be credible.