DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Software Design and Architecture
  3. Cloud Architecture
  4. File System Implementation on TI CC3200

File System Implementation on TI CC3200

There were some issues around the file system implementation on TI CC3200, and this article shows how it was resolved.

Anatoly Lebedev user avatar by
Anatoly Lebedev
·
Jun. 11, 16 · Tutorial
Like (3)
Save
Tweet
Share
4.61K Views

Join the DZone community and get the full member experience.

Join For Free

We have been working with TI CC3200 a lot over the past months to make sure that it is fully compatible with our IoT platform Smart.js. We came up against some issues around the file system implementation and wanted to share how we solved this.

TI CC3200 File System: What’s Hot and What’s Not so Much

TI does include file system support in its SimpleLink SDK: there’s sl_FsOpen,sl_FsRead, sl_FsWrite, and sl_FsDel. In fact, this file system (which we, for lack of a better name, will call TIFS) is the only way to access SPI Flash in CC3200, as it is connected to an SPI interface behind the integrated Network Processor.

TIFS works for very basic file access and actually provides something unique: transparent file encryption and redundant files (at the expense of 2x space usage). This is a really nice to have, however, it has two crucial shortcomings.

Firstly, TIFS doesn’t allow you to list your files. So you can create, write and read them by specific names, but you cannot see all files and select the one you need. Needless to say, there aren’t such things as directories, but even listing files in the one and only directory is not supported at all.

The second issue is around having to predeclare the size of the file. TIFS pre-allocates space and the file size is limited by the size declared at the time the file is created. This means you may not be able to write to a file if it ends up larger than you expected (even if the file system as a whole has space available) or, if it ends up smaller, the reserved space will be wasted.

In Smart.js, we aim to support a POSIX compatible file API. Unfortunately, TIFS does not map well onto that. Especially the lack of file listing was deemed critical, so here is what we did next.

SPIFFS

We examined suitable full feature file systems that would work on Flash. One of them is the SPI Flash File System (SPIFFS). This system is designed for raw Flash access and we use it on other platforms where no file system interface is provided by the SDK.

This is where we came up against the hurdle that TI doesn’t provide access to raw Flash. The only way to access Flash storage is through the TI file system. At this point a shout-out to the TI forum engineers who always provide answers to our queries! Unfortunately, in his response, a TI engineer confirmed that there is no way to access the SPI flash directly.

In light of this, we decided to try to implement SPIFFS in a TIFS file container. This way, a single large TIFS file would contain a whole SPIFFS file system and we’d have POSIX API mapped to SPIFFS file functions. This is how it would look like:

IoT Platform for TI CC3200

For SPIFFS to work, one needs to implement its block storage access API, which consists of just 3 functions: read block, write block and erase block (reset to all 1’s, the NOR Flash logic). We decided to look closer at the semantics of TIFS and it turned out, that, crucially, TIFS allows random write access to regions of a file and that the regions that were not written before read as all 1’s. Repeated writes to previously written regions can only reset bits to 0. In other words, the semantics are that of the underlying NOR flash (which may be confusing to the end users of TIFS who are not aware of the Flash behavior). This suits SPIFFS well, as it is designed for this exact purpose. There is a major downside, however: it is impossible to reopen a file for writing without erasing the contents (in terms of POSIX open modes, it’s not possible to open for “r+”, only “w”). This presents a major inconvenience, and one that, we think, could be addressed with a very simple change on the TI side, but they said no. There is also no explicit erase function, so it’s not possible to reset regions of a file to all 1’s.

Not being able to re-open an existing container for writing will have significant performance implications for writing files, but it did not completely discourage us. Assuming mostly-read access pattern, but still supporting writes (albeit at a heavy cost), we can still make it work.

The idea is to employ two container files, only one which of which is active at a time (this obviously also doubles the space requirements). Read-only access is possible from the start and no performance penalty, but whenever a mutation is requested (user write or SPIFFS block erase), a container switch operation is performed.

Container Switch

A container switch operation is basically a file copy operation that may optionally skip a specific region, thereby leaving it in all 1’s state.

There are two cases when we need to perform container switch:

  1. Preparing for write operation. We need to have TIFS descriptor that is open for writing and, as mentioned above, it is impossible to do so without truncating the file. Keeping in mind our optimization for reads, we start by mounting SPIFFS effectively read-only, but on the first block write we will have to perform a container switch with no skipping.
  2. SPIFFS block erase request. When SPIFFS decides to erase a block, we will perform a container switch copy and skip the specified range.

There are a number of issues associated with the multi-container setup and the switch operation.

Firstly, on boot, we must be able to identify which container of the two, is the most recent one. This is because after container switch they contain mostly identical copies of the filesystem.

Secondly, we must make sure that an interrupted container copy operation (e.g. due to power failure) does not result in data loss or corrupted file system.

We address both of these by keeping a short metadata structure at the end of the container file. In this structure, in addition to SPIFFS creation parameters (page and block size), we also record a sequencer. Sequencer is a 64-bit number that starts at “all 1’s minus 1” (0xFFFFFFFFFFFFFFFE) and counts down. On every container switch, we decrement the sequencer, and the file system container with the lowest sequencer is considered more recent. When performing a container switch, we write updated metadata last, and because the empty container starts as all 1’s and we count our sequencers down rather than up, an interrupted container switch (which has not written metadata yet) will result in an image with the highest sequencer value (and thus lowest priority), even higher than the initial one for a valid image (which is one lower). There are no provisions made for sequencer wraparound, as even if a program is doing nothing other than writing, that will occur if not after, then not shortly before the heat death of the Universe. The chip will certainly wear out sooner.

After all that, the final picture will look something like this:

IoT Platform for TI CC3200


So the end result is a full feature POSIX file system on TI CC3200 and it’s all used in Smart. js. 

Since Smart.js is open source, you can find all the source implementing this on GitHub, under the CC3200 platform directory (look for cc3200_fs_* files).

Future Work

There is a possibility of using smaller container files to store parts of the SPIFFS image. This will help alleviate the performance impact of container switch on larger filesystem images. It’s at idea stage at the moment. We’ll let you know when it comes to implementation.

File system Container Database Implementation Blocks

Published at DZone with permission of Anatoly Lebedev, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Implementing Adaptive Concurrency Limits
  • How to Quickly Build an Audio Editor With UI
  • 2023 Software Testing Trends: A Look Ahead at the Industry's Future
  • How to Deploy Machine Learning Models on AWS Lambda Using Docker

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: