A minimalist file system

I needed a file system for a small web server managed by an STM32G071 microcontroller. After some research I decided to write my own minimalist file system to meet my needs: the MFS library.

For this server on microcontroller I need to access web pages which are read-only files. Indeed the pages are created on a PC and gathered in a folder, this folder is then treated to constitute only one file containing the file system to be downloaded in the flash managed by the microcontroller.

Download MFS sources from GitHub

To download the MFS to the flash you need an external loader

Which File System (FS)

It is often interesting to use software available and maintained by a community. After searching on the net I came to the conclusion that littlefs is a good candidate.

The littlefs github page says:

It’s tasty, so I gave it a try.

The first compilation indicates an occupation of the code in flash of 24 kB. It’s more than the RTOS I use, and more than the server code.

There is a read-only mode, so I’m testing it: code size 12 kB. That’s still a lot.

So a little thought:

• Power-loss resilience: this helps to manage power loss during a write. I don’t need it since I never write to the FS.
• Dynamic wear leveling: No need either, site updates are rare. I’m sure I’ll do less than 100,000, the number of erase/program cycles supported by the flash I’m using.
• Bounded RAM/ROM: RAM usage is limited to a few cache areas, which is a good thing.

Littlefs is a response to problems introduced by the writability in the FS and it responds to them quite well. In the end, the assets of littlefs are not useful to me and have a significant cost.

Therefore I decided to develop my own minimalist file system.

General specifications

• As simple as possible, without useless functionality that would complicate the code, hence the minimalist qualification attributed to this FS.
• Minimal flash and RAM usage.
• Since the system must be read only, we must use all the simplifications that this makes possible.
• Have an architecture based on directories, to be closer to the original architecture on PC.
• No limitation in number of files or level of directories.
• Be able to use fairly long, but reasonable filenames.
• To be able to allow the use of several FS at the same time, and the use of an FS by several tasks at the same time (thread safe).

Block size

The FS support memory allocation unit is a fixed size block.

There is no need to worry about the flash erase block size which is often 4 kB, because there are never any partial erases. We systematically erase all the necessary blocks when copying the FS in flash. Using 512 byte blocks saves a lot of space on small files or blocks used by directories.

A file uses an integer number of blocks. This militates in the direction of having small blocks.

The size of the blocks is configurable during the construction of the FS, and registered in the super block of the FS. This allows multiple FSs to be used with blocks of different sizes.

Directories

A directory block contains a list of entries corresponding to files and subdirectories. These entries contain, among other things, the name of the file or directory. In a generic FS these entries are of fixed size, because it is necessary to be able to replace a name by another without having to touch the other entries.

For a read-only FS, one can have entries whose size is variable and adjusted during the construction of the FS to the size of the names. In this way the entries are “compacted”: one can use files with long names without wasting space, and place more entries in a block.

Files

In a directory, the entry for a file has a pointer to a sequence of contiguous blocks containing the file’s data. This allows for a simple structure for the FS, and faster reading.

Files have no attributes. In a directory entry a single bit indicates whether it is a file or a directory.

Memory occupancy

The library uses a RAM scratchpad buffer the size of a directory entry which is the largest object in the architecture. There is never a reading of an entire block for example.

By default, the size of an entry is 96 bytes, which makes it possible to have a name of 87 bytes at most. This size can be modified by a #define, but then it will be necessary to be careful to use the same size when building the FS on PC and its use on microcontroller.

To use the FS (mount it) the user provides a 24 byte mfsCtx_t configuration structure.

For each open file the user provides a 20 byte mfsFile_t structure.

The code uses less than 1.5 KB of flash.

From my point of view, the objective of a reduced use of RAM and flash memories has been achieved.

Physical memory access

The MFS library must have read access to the memory supporting the FS, and therefore use a low-level driver.

Access to this driver is provided by the user through the read function provided to the configuration structure when mounting the FS.

Concurrent access

There are several types of coincident to consider:

• The same library can be used to access several FS
• Several tasks of an RTOS can concurrently access the same FS
• Concurrent access to FS support. For example the FS can be placed on a flash memory accessed by SPI, and this same SPI can be connected to a screen.

In the configuration structure provided when mounting the FS the user has entered two functions lock and unlock. They are used at the entry and exit of each MFS API function.

If the user is sure that there is no concurrency, these functions can be set to NULL.

Otherwise these functions must for example take a semaphore, and configure the SPI for their use (it may be that another device on the SPI is not using the same mode or the same speed).

The library provides the means to manage concurrency, but the lock and unlock functions are provided by the user.

The MFS Library API

mfsError_t	mfsMount	(mfsCtx_t * pCtx) ;
mfsError_t	mfsUmount	(mfsCtx_t * pCtx) ;
mfsError_t	mfsStat		(mfsCtx_t * pCtx, const char * path, mfsStat_t * pStat) ;

mfsError_t	mfsOpen		(mfsCtx_t * pCtx, mfsFile_t * pFile, const char * path) ;
mfsError_t	mfsClose	(mfsFile_t * pFile) ;

mfsError_t	mfsRead		(mfsFile_t * pFile, void * pBuffer, int32_t size) ;
mfsError_t	mfsSeek		(mfsFile_t * pFile, int32_t offset, mfsWhenceFlag_t whence) ;
int32_t		mfsSize		(mfsFile_t * pFile) ;
void		mfsRewind	(mfsFile_t * pFile) ;

mfsError_t	mfsDirOpen	(mfsCtx_t * pCtx, const char * path, mfsDir_t * pDir) ;
mfsError_t	mfsDirRead	(mfsCtx_t * pCtx, mfsDir_t * pDir, mfsDirEntry_t * pEntry);

The mfsDirOpen and mfsDirRead functions are not normally useful, but can be used to check the consistency of the FS

Complete code example

#include "mfs.h"

//--------------------------------------------------------------------------------
// The user provided function to set in the MFS context

static int mfsDevRead (void * userData, uint32_t address, void * pBuffer, uint32_t size)
{
	W25Q_Read (pBuffer, address + (uint32_t) userData, size) ;
	return MFS_ENONE ;
}

static	void mfsDevLock (void * userData)
{
	(void) userData ;
	W25Q_SpiGet () ;
}

static	void mfsDevUnlock (void * userData)
{
	(void) userData ;
}

//--------------------------------------------------------------------------------

 void	dumpFile (const char * filePath)
{
	mfsCtx_t	mfsCtx ;
	mfsFile_t	mfsFile ;
	mfsStat_t	fileStat ;
	mfsError_t	err ;
	uint32_t	nn, fileSize ;
	uint8_t	readBuffer [80] ;

	// Initialize the MFS context
	// userData is used to provide the starting address
	// of the file system in the FLASH
	mfsCtx.userData = (void *) 0x100000 ;
	mfsCtx.lock     = mfsDevLock ;
	mfsCtx.unlock   = mfsDevUnlock ;
	mfsCtx.read     = mfsDevRead ;

	// Mount the MFS
	err = mfsMount (& mfsCtx) ;
	if (err != MFS_ENONE)
	{
		printf ("mfsMount error: %d\nAbort\n", err) ;
		return ;
	}

	// Test if the file exist
	err = mfsStat (& mfsCtx, filePath, & fileStat) ;
	if (err != MFS_ENONE)
	{
		printf ("mfsStat error: %d\nAbort\n", err) ;
		return ;
	}
	printf ("Information from: mfsStat (\"%s\")\n", filePath) ;
	printf ("  Type %s\n", ((fileStat.type & MFS_DIR) != 0) ?
				 "DIR" : ((fileStat.type & MFS_FILE) != 0) ? "FILE" : "???") ;
	printf ("  Size %d\n\n", fileStat.size) ;

	if ((fileStat.type & MFS_DIR) == MFS_DIR)
	{
		printf ("Can't dump a directory\n") ;
		return ;
	}

	// Open the MFS file
	err = mfsOpen (& mfsCtx, & mfsFile, filePath) ;
	if (err != MFS_ENONE)
	{
		printf ("msfOpen error: %d\nAbort\n", err) ;
		return ;
	}

	// Dump the file
	printf ("File content:\n") ;
	fileSize = 0 ;
	printf ("<") ;	// Start marker
	while (0 != (nn = mfsRead (& mfsFile, readBuffer, sizeof (readBuffer))))
	{
		for (uint32_t ii = 0 ; ii < nn ; ii++)
		{
			putchar (readBuffer [ii]) ;
		}
		fileSize += nn;
	}
	printf (">\nBytes read: %d\n", fileSize) ;	// End marker

	mfsClose  (& mfsFile) ;
	mfsUmount (& mfsCtx) ;
}

Making the file system

The file system is built offline on a PC using the mfsBuild utility whose syntax is:

mfsBuid -i <source_dir> -o <output_image_file> -b <block_size>

source_dir
The path of the directory which will be the root directory of the FS, by default webpages

output_image_file
The path to the file that will contain the FS, by default image.bin

block_size
The block size to use, default 512. Depending on the tool used to write the FS in the flash, the extension of the image file may be imposed. If STM32CubeProgrammer is used, a .bin extension is required.

File system check

The mfsExtract utility makes it possible to extract the contents of an FS and to reconstruct its tree structure on the PC. Then it is possible to compare the original tree structure used to build the FS and the tree structure extracted from the FS. If all went well, the directories and files are identical.

mfsExtract -i <input_image_file> -o <output_folder_to_create>

input_image_file
The path to the file containing the FS, by default image.bin

output_folder_to_create
The name of the directory to create to contain the files extracted from the FS, by default imgfs.

Download

Download sources from GitHub