From ec498d266026513066907dd63d97612018a8259c Mon Sep 17 00:00:00 2001 From: Gregory Nutt Date: Mon, 24 Sep 2018 18:05:09 -0600 Subject: [PATCH] This commit brings in an inital port of the SPIFFS flash file system into NuttX. The file system is still untested at this point (and subject to some additional review). It is, however, marked EXPERIMENTAL should this should not cause a problem for anyone. Squashed commit of the following: fs/spiffs: Fix last compilation issue. Now compiles without error. It is still not quite ready for testing as there is additional code review that must be be performed. It is now marked as EXPERIMENTAL so that it can be brought onto the master branch with little risk. fs/spiffs: Remove some dead code. fs/spiffs: Weak start of analysis of spiffs_nucleus.c. Renamed to spiffs_core.c fs/spiffs: Rename spiffs_nucleus.c to spiffs_core.c fs/spiffs: Remove spiffs_config.h. All configuration settings are now available in the SPIFFS Kconfig options. fs/spiffs: Finished review, update, and repartitioning of spiffs_check.c. Added spiffs_check.h. fs/spiffs: Finished review, update, and repartitioning of spiffs_cache.c. Added spiffs_cache.h. fs/spiffs: Clean up some defines used in debug output statements. fs/spiffs: Finished review, update, and repartitioning of spiffs_gc.c. Added spiffs_gc.h. fs/spiffs: Now that VFS interface is completed, I have begun the long march of repartitioning the remaining functionality, reviewing logic, identifying dead code, and cleaning up loose ends. fs/spiffs: Initial integration of MTD interface, replacing the SPIFFS native flash interface. Lots of open issues such as the use of pages vs. blocks vs. erase blocks and units of addresses, offsets, and lengths that are passed in function calls. Remove SPIFFS_USE_MAGIC support. That option (which default to OFF anyway), wrote a magic value at the beginning of every sector and support verifiable identification of the file system. It was not being and used and removing it makes life simpler. fs/spiffs: Remove semaphore lock on the file object structure. Ultimately, the file access must modify the volume and access the volume structue which also has a exclusivity lock. So use of the volume lock alone should be sufficient. Integrated the SPIFFS rename logic into the NuttX VFS. Removed non-standard application calls or convert them to IOCTL commands. These were converted to IOCTL commands: (1) integrity check, (2) garbage collection, and (3) format flash. These were removed: (1) Integrity check callback. These provided a lot of good information about the state of the file system, but such callbacks are not compatible with a POSIX compliant file system. (2) Index maps. The index maps were a performance improvement feature. The user could provide the memory and request that a region of a a file use that memory for improved lookup performance when accessing parts of the file. The fallback is the less performance lookup by traversing the FLASH memory. (3) Removed the quick garbage collection interface (the code is still used internally). Only the full garbage collection is available to the user application via IOCTL. configs/sim/spiffs: A simulator configuration to use for testing SPIFFS. fs/spiffs: Integrate SPIFFS logic into NuttX VFS bind() and unbind() methods. fs/mount/fs_mount.c: Add SPIFFS to the list of drivers that require MTD vs block drivers. fs/spiffs: Trivial changes, mostly from analysis of how to integrate the rename() VFS method. fs/spiffs: Connect NuttX VFS unlink method to the SPIFFS_remove() function. Lots of name-changing. fs/spiffs: Remove non-standard errno support. Remove bogus SPIFFS_LOCK() and SPIFFS_UNLOCK() macros. fs/spiffs: Add NuttX VFS implementation for statfs() method. Clean up some of the accumulating compilation problems. fs/spiffs: Add stat(), truncate() methods. Dummy out unsupport mkdir() and rmdir() methods. fs/spiffs: Replace some of the custom error numbers with standard error numbers. fs/spiffs: Hooks read(), write(), fstat(), ioctl(), opendir(), closedir(), rewindif(), and readdir() into the NuttX VFS. fs/spiffs: Beginning the organization to work with the NuttX VFS. Lots of things are get broken! fs/spiffs: Add spiffs.c which will be the interface between SPIFFS and NuttX. No very close at present, however. fs/spiffs: Clean up some compile problems introduced by coding standard changes. fs/spiffs: A little closer to NuttX coding standard. fs/spiffs: Ran tools/indent.sh against all files. Closer to NuttX coding standard, but needs a lot more effort to be fully compliant. fs/spiffs: This commit brings in version 0.3.7 of Peter Anderson's SPIFFS. The initial commit includes the core FS files (with some definitions destributed to their correct header files) and hooks into the build system. --- COPYING | 24 + Documentation/README.html | 2 + README.txt | 2 + configs/sim/README.txt | 9 +- configs/sim/spiffs/defconfig | 29 + configs/sim/src/sim_bringup.c | 11 + fs/Kconfig | 1 + fs/Makefile | 1 + fs/mount/fs_mount.c | 11 +- fs/spiffs/Kconfig | 151 ++ fs/spiffs/Make.defs | 49 + fs/spiffs/README.md | 215 +++ fs/spiffs/docs/TECH_SPEC | 239 +++ fs/spiffs/docs/TODO | 15 + fs/spiffs/src/spiffs.h | 349 ++++ fs/spiffs/src/spiffs_cache.c | 716 ++++++++ fs/spiffs/src/spiffs_cache.h | 282 ++++ fs/spiffs/src/spiffs_check.c | 2027 ++++++++++++++++++++++ fs/spiffs/src/spiffs_check.h | 154 ++ fs/spiffs/src/spiffs_core.c | 2989 +++++++++++++++++++++++++++++++++ fs/spiffs/src/spiffs_core.h | 495 ++++++ fs/spiffs/src/spiffs_gc.c | 1223 ++++++++++++++ fs/spiffs/src/spiffs_gc.h | 133 ++ fs/spiffs/src/spiffs_mtd.c | 189 +++ fs/spiffs/src/spiffs_mtd.h | 131 ++ fs/spiffs/src/spiffs_vfs.c | 1817 ++++++++++++++++++++ fs/spiffs/src/spiffs_volume.c | 470 ++++++ include/fcntl.h | 1 + include/nuttx/fs/dirent.h | 13 + include/nuttx/fs/ioctl.h | 15 +- include/sys/statfs.h | 1 + 31 files changed, 11760 insertions(+), 4 deletions(-) create mode 100644 configs/sim/spiffs/defconfig create mode 100644 fs/spiffs/Kconfig create mode 100644 fs/spiffs/Make.defs create mode 100644 fs/spiffs/README.md create mode 100644 fs/spiffs/docs/TECH_SPEC create mode 100644 fs/spiffs/docs/TODO create mode 100644 fs/spiffs/src/spiffs.h create mode 100644 fs/spiffs/src/spiffs_cache.c create mode 100644 fs/spiffs/src/spiffs_cache.h create mode 100644 fs/spiffs/src/spiffs_check.c create mode 100644 fs/spiffs/src/spiffs_check.h create mode 100644 fs/spiffs/src/spiffs_core.c create mode 100644 fs/spiffs/src/spiffs_core.h create mode 100644 fs/spiffs/src/spiffs_gc.c create mode 100644 fs/spiffs/src/spiffs_gc.h create mode 100644 fs/spiffs/src/spiffs_mtd.c create mode 100644 fs/spiffs/src/spiffs_mtd.h create mode 100644 fs/spiffs/src/spiffs_vfs.c create mode 100644 fs/spiffs/src/spiffs_volume.c diff --git a/COPYING b/COPYING index 757738bfe46..3ad34872694 100644 --- a/COPYING +++ b/COPYING @@ -159,6 +159,30 @@ fs/nfs: "This product includes software developed by the University of California, Berkeley and its contributors." +fs/spiffs +^^^^^^^^^ + + The MIT License (MIT) + + Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976gmail.com) + + Permission is hereby granted, free of charge, to any person obtaining a copy of + this software and associated documentation files (the "Software"), to deal in + the Software without restriction, including without limitation the rights to + use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of + the Software, and to permit persons to whom the Software is furnished to do so, + subject to the following conditions: + + The above copyright notice and this permission notice shall be included in all + copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER + IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + strtod(): ^^^^^^^^ diff --git a/Documentation/README.html b/Documentation/README.html index 95039b35387..1c45307fc18 100644 --- a/Documentation/README.html +++ b/Documentation/README.html @@ -386,6 +386,8 @@ nuttx/ | | `- README.txt | |- procfs/ | | `- README.txt + | |- spiffs/ + | | `- README.md | `- unionfs/ | `- README.txt |- graphics/ diff --git a/README.txt b/README.txt index 2f0d18ac32f..0fcf9072114 100644 --- a/README.txt +++ b/README.txt @@ -2058,6 +2058,8 @@ nuttx/ | | `- README.txt | |- procfs/ | | `- README.txt + | |- spiffs/ + | | `- README.md | `- unionfs/ | `- README.txt |- graphics/ diff --git a/configs/sim/README.txt b/configs/sim/README.txt index a6b306136fc..7a763c7a60b 100644 --- a/configs/sim/README.txt +++ b/configs/sim/README.txt @@ -785,8 +785,8 @@ nx11 nxffs - This is the apps/examples/nxffs test using a MTD RAM driver to - simulate the FLASH part. + This is a test of the NXFFS file system using the apps/examples/nxffs + test with an MTD RAM driver to simulate the FLASH part. nxlines @@ -892,6 +892,11 @@ sixlowpan See also the 'pktradio' configuration. +spiffs + + This is a test of the SPIFFS file system using the apps/examples/fstest + test with an MTD RAM driver to simulate the FLASH part. + touchscreen This configuration uses the simple touchscreen test at diff --git a/configs/sim/spiffs/defconfig b/configs/sim/spiffs/defconfig new file mode 100644 index 00000000000..f280ff5751d --- /dev/null +++ b/configs/sim/spiffs/defconfig @@ -0,0 +1,29 @@ +CONFIG_ARCH="sim" +CONFIG_ARCH_BOARD="sim" +CONFIG_ARCH_BOARD_SIM=y +CONFIG_ARCH_SIM=y +CONFIG_DEBUG_ASSERTIONS=y +CONFIG_DEBUG_FEATURES=y +CONFIG_DEBUG_SYMBOLS=y +CONFIG_DISABLE_MQUEUE=y +CONFIG_DISABLE_POLL=y +CONFIG_DISABLE_POSIX_TIMERS=y +CONFIG_DISABLE_PTHREAD=y +CONFIG_DISABLE_SIGNALS=y +CONFIG_EXAMPLES_FSTEST=y +CONFIG_EXPERIMENTAL=y +CONFIG_FS_SPIFFS=y +CONFIG_IDLETHREAD_STACKSIZE=4096 +CONFIG_LIB_RAND_ORDER=3 +CONFIG_MAX_TASKS=64 +CONFIG_MTD=y +CONFIG_NFILE_DESCRIPTORS=32 +CONFIG_PTHREAD_STACK_DEFAULT=8192 +CONFIG_RAMMTD=y +CONFIG_RAMMTD_FLASHSIM=y +CONFIG_SDCLONE_DISABLE=y +CONFIG_SPIFFS_CHECK_ONMOUNT=y +CONFIG_START_DAY=24 +CONFIG_START_MONTH=9 +CONFIG_USERMAIN_STACKSIZE=4096 +CONFIG_USER_ENTRYPOINT="fstest_main" diff --git a/configs/sim/src/sim_bringup.c b/configs/sim/src/sim_bringup.c index af5ef61420a..2b4db1952af 100644 --- a/configs/sim/src/sim_bringup.c +++ b/configs/sim/src/sim_bringup.c @@ -170,6 +170,17 @@ int sim_bringup(void) */ smart_initialize(0, mtd, NULL); + +#elif defined(CONFIG_FS_SPIFFS) + /* Register the MTD driver so that it can be accessed from the VFS */ + + ret = register_mtddriver("/dev/rammtd", mtd, 0755, NULL); + if (ret < 0) + { + syslog(LOG_ERR, "ERROR: Failed to register MTD driver: %d\n", + ret); + } + #elif defined(CONFIG_FS_NXFFS) /* Initialize to provide NXFFS on the MTD interface */ diff --git a/fs/Kconfig b/fs/Kconfig index aa5dd9f70ce..f2bfe381c97 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -87,6 +87,7 @@ source fs/tmpfs/Kconfig source fs/smartfs/Kconfig source fs/binfs/Kconfig source fs/procfs/Kconfig +source fs/spiffs/Kconfig source fs/unionfs/Kconfig source fs/userfs/Kconfig source fs/hostfs/Kconfig diff --git a/fs/Makefile b/fs/Makefile index e77f76c8b5f..86b809b1ceb 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -73,6 +73,7 @@ include nfs/Make.defs include smartfs/Make.defs include binfs/Make.defs include procfs/Make.defs +include spiffs/Make.defs include unionfs/Make.defs include userfs/Make.defs include hostfs/Make.defs diff --git a/fs/mount/fs_mount.c b/fs/mount/fs_mount.c index b938120e2e6..191b5405efe 100644 --- a/fs/mount/fs_mount.c +++ b/fs/mount/fs_mount.c @@ -77,7 +77,9 @@ /* These file systems require MTD drivers */ -#undef MDFS_SUPPORT +#ifdef CONFIG_FS_SPIFFS +# define MDFS_SUPPORT 1 +#endif /* These file systems do not require block or MTD drivers */ @@ -133,8 +135,15 @@ static const struct fsmap_t g_bdfsmap[] = #ifdef MDFS_SUPPORT /* File systems that require MTD drivers */ +#ifdef CONFIG_FS_SPIFFS +extern const struct mountpt_operations spiffs_operations; +#endif + static const struct fsmap_t g_mdfsmap[] = { +#ifdef CONFIG_FS_SPIFFS + { "spiffs", &spiffs_operations }, +#endif }; #endif /* MDFS_SUPPORT */ diff --git a/fs/spiffs/Kconfig b/fs/spiffs/Kconfig new file mode 100644 index 00000000000..f21e6f1b710 --- /dev/null +++ b/fs/spiffs/Kconfig @@ -0,0 +1,151 @@ +# +# For a description of the syntax of this configuration file, +# see the file kconfig-language.txt in the NuttX tools repository. +# + +config FS_SPIFFS + bool "SPIFFS File System" + default n + select FS_READABLE + select FS_WRITABLE + depends on !DISABLE_MOUNTPOINT && EXPERIMENTAL + ---help--- + Build the SPIFFS file system. This is a port of version 0.3.7 of + the SPIFFS file system by Peter Andersson. It was originally + released under the MIT license. + +if FS_SPIFFS + +comment "In-memory Cache Options" + +config SPIFFS_CACHE_SIZE + int "Size of the cache" + default 8192 + +config SPIFFS_CACHE_HITSCORE + int "Cache Hit Score" + default 4 + range 1 255 + ---help--- + Temporal file cache hit score. Each time a file is opened, all + cached files will lose one point. If the opened file is found in + cache, that entry will gain CONFIG_SPIFFS_CACHE_HITSCORE points. + One can experiment with this value for the specific access patterns + of the application. However, it must be between 1 (no gain for + hitting a cached entry often) and 255. + +config SPIFFS_CACHEDBG + bool "Enable cache debug output" + default n + depends on DEBUG_FS_INFO + +comment "Garbage Collection (GC) Options" + +config SPIFFS_GC_MAXRUNS + int "Max GC runs" + default 5 + ---help--- + Define maximum number of gc runs to perform to reach desired free + pages. + +config SPIFFS_GC_DELWGT + int "Deleted pages weight" + default 5 + ---help--- + Garbage collecting examines all pages in a block which and sums up + to a block score. Deleted pages normally gives positive score and + used pages normally gives a negative score (as these must be moved). + To have a fair wear-leveling, the erase age is also included in + score, whose factor normally is the most positive. The larger the + score, the more likely it is that the block will picked for garbage + collection. + + This option provides the weight used for deleted pages. + +config SPIFFS_GC_USEDWGT + int "Used pages weight" + default -1 + ---help--- + Garbage collecting examines all pages in a block which and sums up + to a block score. Deleted pages normally gives positive score and + used pages normally gives a negative score (as these must be moved). + To have a fair wear-leveling, the erase age is also included in + score, whose factor normally is the most positive. The larger the + score, the more likely it is that the block will picked for garbage + collection. + + This option provides the weight used for used pages. + +config SPIFFS_GC_ERASEAGEWGT + int "Used pages weight" + default 50 + ---help--- + Garbage collecting examines all pages in a block which and sums up + to a block score. Deleted pages normally gives positive score and + used pages normally gives a negative score (as these must be moved). + To have a fair wear-leveling, the erase age is also included in + score, whose factor normally is the most positive. The larger the + score, the more likely it is that the block will picked for garbage + collection. + + This option provides the weight used weight used for time between + last erased and erase of this block. + +config SPIFFS_GCDBG + bool "Enable garbage collection debug output" + default n + depends on DEBUG_FS_INFO + +comment "Consistency Check Options" + +config SPIFFS_CHECK_ONMOUNT + bool "Consistency check on mount" + default n + select SPIFFS_CHECK_OUTPUT + ---help--- + Perhaps a full, extended consistency check on mount. By default, a + simple scan of the file system only is performed. + +config SPIFFS_CHECK_OUTPUT + bool "Enable consistency check SYSLOG output" + default n + +comment "SPIFFS Core Options" + +config SPIFFS_NO_BLIND_WRITES + bool "No blind writes" + default n + ---help--- + By default SPIFFS in some cases relies on the property of NOR flash + that bits cannot be set from 0 to 1 by writing and that controllers + will ignore such bit changes. This results in fewer reads as SPIFFS + can in some cases perform blind writes, with all bits set to 1 and + only those it needs reset set to 0. Most of the chips and + controllers allow this behavior, so the default is to use this + technique. If your controller is one of the rare ones that don't, + turn this option on and SPIFFS will perform a read-modify-write + instead. + +config SPIFFS_PAGE_CHECK + bool "Check Page Headers" + default y + ---help--- + Always check header of each accessed page to ensure consistent state. + If enabled it will increase number of reads, will increase flash. + +config SPIFFS_NAME_MAX + int "Maximum Name Length" + default 32 + ---help--- + Object name maximum length. Note that this length include the zero + termination character, meaning maximum string of characters. + +config SPIFFS_COPYBUF_STACK + int "Max Stack Allocation" + default 64 + ---help--- + Size of buffer allocated on stack used when copying data. Lower + value generates more read/writes. No meaning having it bigger + than logical page size. + +endif # FS_SPIFFS diff --git a/fs/spiffs/Make.defs b/fs/spiffs/Make.defs new file mode 100644 index 00000000000..b02efb44e8b --- /dev/null +++ b/fs/spiffs/Make.defs @@ -0,0 +1,49 @@ +############################################################################ +# fs/spiffs/Make.defs +# +# Copyright (C) 2018 Gregory Nutt. All rights reserved. +# Author: Gregory Nutt +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# 3. Neither the name Nuttx nor the names of its contributors may be +# used to endorse or promote products derived from this software +# without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS +# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE +# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, +# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, +# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS +# OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED +# AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN +# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +# POSSIBILITY OF SUCH DAMAGE. +# +############################################################################ + +ifeq ($(CONFIG_FS_SPIFFS),y) +# Files required for spiffs file system support + +ASRCS += +CSRCS += spiffs_vfs.c spiffs_volume.c spiffs_core.c spiffs_gc.c +CSRCS += spiffs_cache.c spiffs_check.c spiffs_mtd.c + +# Include spiffs build support + +DEPPATH += --dep-path spiffs/src +VPATH += :spiffs/src +CFLAGS += ${shell $(INCDIR) $(INCDIROPT) "$(CC)" $(TOPDIR)$(DELIM)fs$(DELIM)spiffs$(DELIM)src} + +endif diff --git a/fs/spiffs/README.md b/fs/spiffs/README.md new file mode 100644 index 00000000000..e75ec5c30fe --- /dev/null +++ b/fs/spiffs/README.md @@ -0,0 +1,215 @@ +# SPIFFS (SPI Flash File System) +**V0.3.7** + +[![Build Status](https://travis-ci.org/pellepl/spiffs.svg?branch=master)](https://travis-ci.org/pellepl/spiffs) + +Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976 at gmail.com) + +For legal stuff, see [LICENSE](https://github.com/pellepl/spiffs/blob/master/LICENSE). Basically, you may do whatever you want with the source. Use, modify, sell, print it out, roll it and smoke it - as long as I won't be held responsible. + +Love to hear feedback though! + + +## INTRODUCTION + +Spiffs is a file system intended for SPI NOR flash devices on embedded targets. + +Spiffs is designed with following characteristics in mind: + - Small (embedded) targets, sparse RAM without heap + - Only big areas of data (blocks) can be erased + - An erase will reset all bits in block to ones + - Writing pulls one to zeroes + - Zeroes can only be pulled to ones by erase + - Wear leveling + + +## BUILDING + +`mkdir build; make` + +Otherwise, configure the `builddir` variable towards the top of `makefile` as something opposed to the default `build`. Sanity check on the host via `make test` and refer to `.travis.yml` for the official in-depth testing procedure. See the wiki for [integrating](https://github.com/pellepl/spiffs/wiki/Integrate-spiffs) spiffs into projects and [spiffsimg](https://github.com/nodemcu/nodemcu-firmware/tree/master/tools/spiffsimg) from [nodemcu](https://github.com/nodemcu) is a good example on the subject. + + +## FEATURES + +What spiffs does: + - Specifically designed for low ram usage + - Uses statically sized ram buffers, independent of number of files + - Posix-like api: open, close, read, write, seek, stat, etc + - It can run on any NOR flash, not only SPI flash - theoretically also on embedded flash of a microprocessor + - Multiple spiffs configurations can run on same target - and even on same SPI flash device + - Implements static wear leveling + - Built in file system consistency checks + - Highly configurable + +What spiffs does not: + - Presently, spiffs does not support directories. It produces a flat structure. Creating a file with path *tmp/myfile.txt* will create a file called *tmp/myfile.txt* instead of a *myfile.txt* under directory *tmp*. + - It is not a realtime stack. One write operation might last much longer than another. + - Poor scalability. Spiffs is intended for small memory devices - the normal sizes for SPI flashes. Going beyond ~128Mbyte is probably a bad idea. This is a side effect of the design goal to use as little ram as possible. + - Presently, it does not detect or handle bad blocks. + - One configuration, one binary. There's no generic spiffs binary that handles all types of configurations. + +## NOTICE + +0.4.0 is under construction. This is a full rewrite and will change the underlying structure. Hence, it will not be compatible with earlier versions of the filesystem. The API is the same, with minor modifications. Some config flags will be removed (as they are mandatory in 0.4.0) and some features might fall away until 0.4.1. If you have any worries or questions, it can be discussed in issue [#179](https://github.com/pellepl/spiffs/issues/179) + +## MORE INFO + +See the [wiki](https://github.com/pellepl/spiffs/wiki) for [configuring](https://github.com/pellepl/spiffs/wiki/Configure-spiffs), [integrating](https://github.com/pellepl/spiffs/wiki/Integrate-spiffs), [using](https://github.com/pellepl/spiffs/wiki/Using-spiffs), and [optimizing](https://github.com/pellepl/spiffs/wiki/Performance-and-Optimizing) spiffs. + +For design, see [docs/TECH_SPEC](https://github.com/pellepl/spiffs/blob/master/docs/TECH_SPEC). + +For a generic spi flash driver, see [this](https://github.com/pellepl/spiflash_driver). + +## HISTORY + +### 0.3.7 +- fixed prevent seeking to negative offsets #158 +- fixed file descriptor offsets not updated for multiple fds on same file #157 +- fixed cache page not closed for removed files #156 +- fixed a lseek bug when seeking exactly to end of a fully indexed first level LUT #148 +- fixed wear leveling issue #145 +- fixed attempt to write out of bounds in flash #130, +- set file offset when seeking over end #121 (thanks @sensslen) +- fixed seeking in virgin files #120 (thanks @sensslen) +- Optional file metadata #128 (thanks @cesanta) +- AFL testing framework #100 #143 (thanks @pjsg) +- Testframe updates + +New API functions: +- `SPIFFS_update_meta, SPIFFS_fupdate_meta` - updates metadata for a file + +New config defines: +- `SPIFFS_OBJ_META_LEN` - enable possibility to add extra metadata to files + +### 0.3.6 +- Fix range bug in index memory mapping #98 +- Add index memory mapping #97 +- Optimize SPIFFS_read for large files #96 +- Add temporal cache for opening files #95 +- More robust gc #93 (thanks @dismirlian) +- Fixed a double write of same data in certain cache situations +- Fixed an open bug in READ_ONLY builds +- File not visible in SPIFFS_readdir #90 (thanks @benpicco-tmp) +- Cache load code cleanup #92 (thanks @niclash) +- Fixed lock/unlock asymmetry #88 #87 (thanks @JackJefferson, @dpruessner) +- Testframe updates + +New API functions: +- `SPIFFS_ix_map` - map index meta data to memory for a file +- `SPIFFS_ix_unmap` - unmaps index meta data for a file +- `SPIFFS_ix_remap` - changes file offset for index metadata map +- `SPIFFS_bytes_to_ix_map_entries` - utility, get length of needed vector for given amount of bytes +- `SPIFFS_ix_map_entries_to_bytes` - utility, get number of bytes a vector can represent given length + +New config defines: +- `SPIFFS_IX_MAP` - enable possibility to map index meta data to memory for reading faster +- `SPIFFS_TEMPORAL_FD_CACHE` - enable temporal cache for opening files faster +- `SPIFFS_TEMPORAL_CACHE_HIT_SCORE` - for tuning the temporal cache + +### 0.3.5 +- Fixed a bug in fs check +- API returns actual error codes #84) (thanks @Nails) +- Fix compiler warnings for non-gcc #83 #81 (thanks @Nails) +- Unable to recover from full fs #82 (thanks @rojer) +- Define SPIFFS_O_* flags #80 +- Problem with long filenames #79 (thanks @psjg) +- Duplicate file name bug fix #74 (thanks @igrr) +- SPIFFS_eof and SPIFFS_tell return wrong value #72 (thanks @ArtemPisarenko) +- Bunch of testframe updates #77 #78 #86 (thanks @dpreussner, @psjg a.o) + +### 0.3.4 +- Added user callback file func. +- Fixed a stat bug with obj id. +- SPIFFS_probe_fs added +- Add possibility to compile a read-only version of spiffs +- Make magic dependent on fs length, if needed (see #59 & #66) (thanks @hreintke) +- Exposed SPIFFS_open_by_page_function +- Zero-size file cannot be seek #57 (thanks @lishen2) +- Add tell and eof functions #54 (thanks @raburton) +- Make api string params const #53 (thanks @raburton) +- Preserve user_data during mount() #51 (thanks @rojer) + +New API functions: +- `SPIFFS_set_file_callback_func` - register a callback informing about file events +- `SPIFFS_probe_fs` - probe a spi flash trying to figure out size of fs +- `SPIFFS_open_by_page` - open a file by page index +- `SPIFFS_eof` - checks if end of file is reached +- `SPIFFS_tell` - returns current file offset + +New config defines: +- `SPIFFS_READ_ONLY` +- `SPIFFS_USE_MAGIC_LENGTH` + +### 0.3.3 +**Might not be compatible with 0.3.2 structures. See issue #40** +- Possibility to add integer offset to file handles +- Truncate function presumes too few free pages #49 +- Bug in truncate function #48 (thanks @PawelDefee) +- Update spiffs_gc.c - remove unnecessary parameter (thanks @PawelDefee) +- Update INTEGRATION docs (thanks @PawelDefee) +- Fix pointer truncation in 64-bit platforms (thanks @igrr) +- Zero-sized files cannot be read #44 (thanks @rojer) +- (More) correct calculation of max_id in obj_lu_find #42 #41 (thanks @lishen2) +- Check correct error code in obj_lu_find_free #41 (thanks @lishen2) +- Moar comments for SPIFFS_lseek (thanks @igrr) +- Fixed padding in spiffs_page_object_ix #40 (thanks @jmattsson @lishen2) +- Fixed gc_quick test (thanks @jmattsson) +- Add SPIFFS_EXCL flag #36 +- SPIFFS_close may fail silently if cache is enabled #37 +- User data in callbacks #34 +- Ignoring SINGLETON build in cache setup (thanks Luca) +- Compilation error fixed #32 (thanks @chotasanjiv) +- Align cand_scores (thanks @hefloryd) +- Fix build warnings when SPIFFS_CACHE is 0 (thanks @ajaybhargav) + +New config defines: +- `SPIFFS_FILEHDL_OFFSET` + +### 0.3.2 +- Limit cache size if too much cache is given (thanks pgeiem) +- New feature - Controlled erase. #23 +- SPIFFS_rename leaks file descriptors #28 (thanks benpicco) +- moved dbg print defines in test framework to params_test.h +- lseek should return the resulting offset (thanks hefloryd) +- fixed type on dbg ifdefs +- silence warning about signed/unsigned comparison when spiffs_obj_id is 32 bit (thanks benpicco) +- Possible error in test_spiffs.c #21 (thanks yihcdaso-yeskela) +- Cache might writethrough too often #16 +- even moar testrunner updates +- Test framework update and some added tests +- Some thoughts for next gen +- Test sigsevs when having too many sectors #13 (thanks alonewolfx2) +- GC might be suboptimal #11 +- Fix eternal readdir when objheader at last block, last entry + +New API functions: +- `SPIFFS_gc_quick` - call a nonintrusive gc +- `SPIFFS_gc` - call a full-scale intrusive gc + +### 0.3.1 +- Removed two return warnings, was too triggerhappy on release + +### 0.3.0 +- Added existing namecheck when creating files +- Lots of static analysis bugs #6 +- Added rename func +- Fix SPIFFS_read length when reading beyond file size +- Added reading beyond file length testcase +- Made build a bit more configurable +- Changed name in spiffs from "errno" to "err_code" due to conflicts compiling in mingw +- Improved GC checks, fixed an append bug, more robust truncate for very special case +- GC checks preempts GC, truncate even less picky +- Struct alignment needed for some targets, define in spiffs config #10 +- Spiffs filesystem magic, definable in config + +New config defines: +- `SPIFFS_USE_MAGIC` - enable or disable magic check upon mount +- `SPIFFS_ALIGNED_OBJECT_INDEX_TABLES` - alignment for certain targets + +New API functions: +- `SPIFFS_rename` - rename files +- `SPIFFS_clearerr` - clears last errno +- `SPIFFS_info` - returns info on used and total bytes in fs +- `SPIFFS_format` - formats the filesystem +- `SPIFFS_mounted` - checks if filesystem is mounted diff --git a/fs/spiffs/docs/TECH_SPEC b/fs/spiffs/docs/TECH_SPEC new file mode 100644 index 00000000000..b4755a6dbc6 --- /dev/null +++ b/fs/spiffs/docs/TECH_SPEC @@ -0,0 +1,239 @@ +* USING SPIFFS + +TODO + + +* SPIFFS DESIGN + +Spiffs is inspired by YAFFS. However, YAFFS is designed for NAND flashes, and +for bigger targets with much more ram. Nevertheless, many wise thoughts have +been borrowed from YAFFS when writing spiffs. Kudos! + +The main complication writing spiffs was that it cannot be assumed the target +has a heap. Spiffs must go along only with the work ram buffer given to it. +This forces extra implementation on many areas of spiffs. + + +** SPI flash devices using NOR technology + +Below is a small description of how SPI flashes work internally. This is to +give an understanding of the design choices made in spiffs. + +SPI flash devices are physically divided in blocks. On some SPI flash devices, +blocks are further divided into sectors. Datasheets sometimes name blocks as +sectors and vice versa. + +Common memory capacaties for SPI flashes are 512kB up to 8MB of data, where +blocks may be 64kB. Sectors can be e.g. 4kB, if supported. Many SPI flashes +have uniform block sizes, whereas others have non-uniform - the latter meaning +that e.g. the first 16 blocks are 4kB big, and the rest are 64kB. + +The entire memory is linear and can be read and written in random access. +Erasing can only be done block- or sectorwise; or by mass erase. + +SPI flashes can normally be erased from 100.000 up to 1.000.000 cycles before +they fail. + +A clean SPI flash from factory have all bits in entire memory set to one. A +mass erase will reset the device to this state. Block or sector erasing will +put the all bits in the area given by the sector or block to ones. Writing to a +NOR flash pulls ones to zeroes. Writing 0xFF to an address is simply a no-op. + +Writing 0b10101010 to a flash address holding 0b00001111 will yield 0b00001010. + +This way of "write by nand" is used considerably in spiffs. + +Common characteristics of NOR flashes are quick reads, but slow writes. + +And finally, unlike NAND flashes, NOR flashes seem to not need any error +correction. They always write correctly I gather. + + +** Spiffs logical structure + +Some terminology before proceeding. Physical blocks/sectors means sizes stated +in the datasheet. Logical blocks and pages is something the integrator choose. + + +** Blocks and pages + +Spiffs is allocated to a part or all of the memory of the SPI flash device. +This area is divided into logical blocks, which in turn are divided into +logical pages. The boundary of a logical block must coincide with one or more +physical blocks. The sizes for logical blocks and logical pages always remain +the same, they are uniform. + +Example: non-uniform flash mapped to spiffs with 128kB logical blocks + +PHYSICAL FLASH BLOCKS SPIFFS LOGICAL BLOCKS: 128kB + ++-----------------------+ - - - +-----------------------+ +| Block 1 : 16kB | | Block 1 : 128kB | ++-----------------------+ | | +| Block 2 : 16kB | | | ++-----------------------+ | | +| Block 3 : 16kB | | | ++-----------------------+ | | +| Block 4 : 16kB | | | ++-----------------------+ | | +| Block 5 : 64kB | | | ++-----------------------+ - - - +-----------------------+ +| Block 6 : 64kB | | Block 2 : 128kB | ++-----------------------+ | | +| Block 7 : 64kB | | | ++-----------------------+ - - - +-----------------------+ +| Block 8 : 64kB | | Block 3 : 128kB | ++-----------------------+ | | +| Block 9 : 64kB | | | ++-----------------------+ - - - +-----------------------+ +| ... | | ... | + +A logical block is divided further into a number of logical pages. A page +defines the smallest data holding element known to spiffs. Hence, if a file +is created being one byte big, it will occupy one page for index and one page +for data - it will occupy 2 x size of a logical page on flash. +So it seems it is good to select a small page size. + +Each page has a metadata header being normally 5 to 9 bytes. This said, a very +small page size will make metadata occupy a lot of the memory on the flash. A +page size of 64 bytes will waste 8-14% on metadata, while 256 bytes 2-4%. +So it seems it is good to select a big page size. + +Also, spiffs uses a ram buffer being two times the page size. This ram buffer +is used for loading and manipulating pages, but it is also used for algorithms +to find free file ids, scanning the file system, etc. Having too small a page +size means less work buffer for spiffs, ending up in more reads operations and +eventually gives a slower file system. + +Choosing the page size for the system involves many factors: + - How big is the logical block size + - What is the normal size of most files + - How much ram can be spent + - How much data (vs metadata) must be crammed into the file system + - How fast must spiffs be + - Other things impossible to find out + +So, chosing the Optimal Page Size (tm) seems tricky, to say the least. Don't +fret - there is no optimal page size. This varies from how the target will use +spiffs. Use the golden rule: + + ~~~ Logical Page Size = Logical Block Size / 256 ~~~ + +This is a good starting point. The final page size can then be derived through +heuristical experimenting for us non-analytical minds. + + +** Objects, indices and look-ups + +A file, or an object as called in spiffs, is identified by an object id. +Another YAFFS rip-off. This object id is a part of the page header. So, all +pages know to which object/file they belong - not counting the free pages. + +An object is made up of two types of pages: object index pages and data pages. +Data pages contain the data written by user. Index pages contain metadata about +the object, more specifically what data pages are part of the object. + +The page header also includes something called a span index. Let's say a file +is written covering three data pages. The first data page will then have span +index 0, the second span index 1, and the last data page will have span index +2. Simple as that. + +Finally, each page header contain flags, telling if the page is used, +deleted, finalized, holds index or data, and more. + +Object indices also have span indices, where an object index with span index 0 +is referred to as the object index header. This page does not only contain +references to data pages, but also extra info such as object name, object size +in bytes, flags for file or directory, etc. + +If one were to create a file covering three data pages, named e.g. +"spandex-joke.txt", given object id 12, it could look like this: + +PAGE 0 + +PAGE 1 page header: [obj_id:12 span_ix:0 flags:USED|DATA] + + +PAGE 2 page header: [obj_id:12 span_ix:1 flags:USED|DATA] + + +PAGE 3 page header: [obj_id:545 span_ix:13 flags:USED|DATA] + + +PAGE 4 page header: [obj_id:12 span_ix:2 flags:USED|DATA] + + +PAGE 5 page header: [obj_id:12 span_ix:0 flags:USED|INDEX] + obj ix header: [name:spandex-joke.txt size:600 bytes flags:FILE] + obj ix: [1 2 4] + +Looking in detail at page 5, the object index header page, the object index +array refers to each data page in order, as mentioned before. The index of the +object index array correlates with the data page span index. + + entry ix: 0 1 2 + obj ix: [1 2 4] + | | | + PAGE 1, DATA, SPAN_IX 0 --------/ | | + PAGE 2, DATA, SPAN_IX 1 --------/ | + PAGE 4, DATA, SPAN_IX 2 --------/ + +Things to be unveiled in page 0 - well.. Spiffs is designed for systems low on +ram. We cannot keep a dynamic list on the whereabouts of each object index +header so we can find a file fast. There might not even be a heap! But, we do +not want to scan all page headers on the flash to find the object index header. + +The first page(s) of each block contains the so called object look-up. These +are not normal pages, they do not have a header. Instead, they are arrays +pointing out what object-id the rest of all pages in the block belongs to. + +By this look-up, only the first page(s) in each block must to scanned to find +the actual page which contains the object index header of the desired object. + +The object lookup is redundant metadata. The assumption is that it presents +less overhead reading a full page of data to memory from each block and search +that, instead of reading a small amount of data from each page (i.e. the page +header) in all blocks. Each read operation from SPI flash normally contains +extra data as the read command itself and the flash address. Also, depending on +the underlying implementation, other criterions may need to be passed for each +read transaction, like mutexes and such. + +The veiled example unveiled would look like this, with some extra pages: + +PAGE 0 [ 12 12 545 12 12 34 34 4 0 0 0 0 ...] +PAGE 1 page header: [obj_id:12 span_ix:0 flags:USED|DATA] ... +PAGE 2 page header: [obj_id:12 span_ix:1 flags:USED|DATA] ... +PAGE 3 page header: [obj_id:545 span_ix:13 flags:USED|DATA] ... +PAGE 4 page header: [obj_id:12 span_ix:2 flags:USED|DATA] ... +PAGE 5 page header: [obj_id:12 span_ix:0 flags:USED|INDEX] ... +PAGE 6 page header: [obj_id:34 span_ix:0 flags:USED|DATA] ... +PAGE 7 page header: [obj_id:34 span_ix:1 flags:USED|DATA] ... +PAGE 8 page header: [obj_id:4 span_ix:1 flags:USED|INDEX] ... +PAGE 9 page header: [obj_id:23 span_ix:0 flags:DELETED|INDEX] ... +PAGE 10 page header: [obj_id:23 span_ix:0 flags:DELETED|DATA] ... +PAGE 11 page header: [obj_id:23 span_ix:1 flags:DELETED|DATA] ... +PAGE 12 page header: [obj_id:23 span_ix:2 flags:DELETED|DATA] ... +... + +Ok, so why are page 9 to 12 marked as 0 when they belong to object id 23? These +pages are deleted, so this is marked both in page header flags and in the look +up. This is an example where spiffs uses NOR flashes "nand-way" of writing. + +As a matter of fact, there are two object id's which are special: + +obj id 0 (all bits zeroes) - indicates a deleted page in object look up +obj id 0xff.. (all bits ones) - indicates a free page in object look up + +Actually, the object id's have another quirk: if the most significant bit is +set, this indicates an object index page. If the most significant bit is zero, +this indicates a data page. So to be fully correct, page 0 in above example +would look like this: + +PAGE 0 [ 12 12 545 12 *12 34 34 *4 0 0 0 0 ...] + +where the asterisk means the msb of the object id is set. + +This is another way to speed up the searches when looking for object indices. +By looking on the object id's msb in the object lookup, it is also possible +to find out whether the page is an object index page or a data page. + diff --git a/fs/spiffs/docs/TODO b/fs/spiffs/docs/TODO new file mode 100644 index 00000000000..c947316a86e --- /dev/null +++ b/fs/spiffs/docs/TODO @@ -0,0 +1,15 @@ +* When mending lost pages, also see if they fit into length specified in object index header + +SPIFFS2 thoughts + +* Instead of exact object id:s in the object lookup tables, use a hash of span index and object id. + Eg. object id xor:ed with bit-reversed span index. + This should decrease number of actual pages that needs to be visited when looking thru the obj lut. + +* Logical number of each block. When moving stuff in a garbage collected page, the free + page is assigned the same number as the garbage collected. Thus, object index pages do not have to + be rewritten. + +* Steal one page, use as a bit parity page. When starting an fs modification operation, write one bit + as zero. When ending, write another bit as zero. On mount, if number of zeroes in page is uneven, a + check is automatically run. \ No newline at end of file diff --git a/fs/spiffs/src/spiffs.h b/fs/spiffs/src/spiffs.h new file mode 100644 index 00000000000..d5b2de43a1b --- /dev/null +++ b/fs/spiffs/src/spiffs.h @@ -0,0 +1,349 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs.h + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * This is a port of version 0.3.7 of SPIFFS by Peter Andersion. That + * version was originally released under the MIT license but is here re- + * released under the NuttX BSD license. + * + * Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976@gmail.com) + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +#ifndef __FS_SPIFFS_SRC_SPIFFS_H +#define __FS_SPIFFS_SRC_SPIFFS_H + +#if defined(__cplusplus) +extern "C" +{ +#endif + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include +#include + +#include +#include + +/**************************************************************************** + * Pre-processor Definitions + ****************************************************************************/ + +#define SPIFFS_ERR_DELETED -10004 +#define SPIFFS_ERR_NOT_FINALIZED -10005 +#define SPIFFS_ERR_NOT_INDEX -10006 +#define SPIFFS_ERR_IS_INDEX -10011 +#define SPIFFS_ERR_IS_FREE -10012 +#define SPIFFS_ERR_INDEX_SPAN_MISMATCH -10013 +#define SPIFFS_ERR_DATA_SPAN_MISMATCH -10014 +#define SPIFFS_ERR_INDEX_REF_FREE -10015 +#define SPIFFS_ERR_INDEX_REF_LU -10016 +#define SPIFFS_ERR_INDEX_REF_INVALID -10017 +#define SPIFFS_ERR_INDEX_FREE -10018 +#define SPIFFS_ERR_INDEX_LU -10019 +#define SPIFFS_ERR_INDEX_INVALID -10020 +#define SPIFFS_ERR_INTERNAL -10050 + +/* Flags on open file/directory options */ + +#define SFO_FLAG_UNLINKED (1 << 0) + +/* Re-entrant semaphore definitions */ + +#define SPIFFS_NO_HOLDER ((pid_t)-1) + +/**************************************************************************** + * Public Types + ****************************************************************************/ + +enum spiffs_check_e +{ + SPIFFS_CHECK_LOOKUP = 0, + SPIFFS_CHECK_INDEX, + SPIFFS_CHECK_PAGE +}; + +/* file system check callback report type */ + +enum spiffs_check_report_e +{ + SPIFFS_CHECK_PROGRESS = 0, + SPIFFS_CHECK_ERROR, + SPIFFS_CHECK_FIX_INDEX, + SPIFFS_CHECK_FIX_LOOKUP, + SPIFFS_CHECK_DELETE_ORPHANED_INDEX, + SPIFFS_CHECK_DELETE_PAGE, + SPIFFS_CHECK_DELETE_BAD_FILE +}; + +/* spi read call function type */ + +typedef int32_t(*spiffs_read_t)(uint32_t addr, uint32_t size, uint8_t * dst); + +/* spi write call function type */ + +typedef int32_t(*spiffs_write_t)(uint32_t addr, uint32_t size, uint8_t * src); + +/* spi erase call function type */ + +typedef int32_t(*spiffs_erase_t)(uint32_t addr, uint32_t size); + +/* Re-entrant semaphore */ + +struct spiffs_sem_s +{ + sem_t sem; /* The actual semaphore */ + pid_t holder; /* Current older (-1 if not held) */ + uint16_t count; /* Number of counts held */ +}; + +/* spiffs SPI configuration struct */ + +/* This structure represents the current state of an SPIFFS volume */ + +struct spiffs_file_s; /* Forward reference */ + +struct spiffs_s +{ + struct mtd_geometry_s geo; /* FLASH geometry */ + struct spiffs_sem_s exclsem; /* Supports mutually exclusive access */ + dq_queue_t objq; /* A doubly linked list of open file objects */ + FAR struct mtd_dev_s *mtd; /* The contained MTD interface */ + FAR uint8_t *lu_work; /* Primary work buffer, size of a logical page */ + FAR uint8_t *work; /* Secondary work buffer, size of a logical page */ + FAR void *cache; /* Cache memory */ +#ifdef CONFIG_HAVE_LONG_LONG + off64_t phys_size; /* Physical size of the SPI flash */ +#else + off_t phys_size; /* Physical size of the SPI flash */ +#endif + int free_entry; /* Cursor for free blocks, entry index */ + int lu_entry; /* Cursor when searching, entry index */ + uint32_t free_blocks; /* Current number of free blocks */ + uint32_t stats_p_allocated; /* Current number of busy pages */ + uint32_t stats_p_deleted; /* Current number of deleted pages */ +#ifdef CONFIG_SPIFFS_GCDBG + uint32_t stats_gc_runs; +#endif + uint32_t cache_size; /* Cache size */ +#ifdef CONFIG_SPIFFS_CACHEDBG + uint32_t cache_hits; /* Number of cache hits */ + uint32_t cache_misses; /* Number of cache misses */ +#endif + uint32_t config_magic; /* Config magic */ + int16_t free_blkndx; /* cursor for free blocks, block index */ + int16_t lu_blkndx; /* Cursor when searching, block index */ + int16_t max_erase_count; /* Max erase count amongst all blocks */ +}; + +/* This structure represents the state of an open file */ + +struct spiffs_cache_page_s; /* Forward reference */ + +struct spiffs_file_s +{ + dq_entry_t entry; /* Supports a doubly linked list */ + FAR struct spiffs_cache_page_s *cache_page; + int16_t crefs; /* Reference count */ + int16_t objid; /* Unique ID of the file object */ + uint8_t flags; /* See SFO_FLAG_* definitions */ + int16_t objhdr_pgndx; /* Cached object index header page index */ + int16_t objndx_pgndx; /* Cached offset object index page index */ + int16_t objndx_spndx; /* Cached offset object index span index */ + uint16_t oflags; /* File object open flags */ + off_t size; /* Size of the file */ + off_t offset; /* Current absolute offset */ +}; + +/**************************************************************************** + * Public Function Prototypes + ****************************************************************************/ + +struct stat; /* Foward reference */ + +/**************************************************************************** + * Name: spiffs_stat_pgndx + * + * Description: + * Checks if there are any cached writes for the object ID associated with + * given file object. If so, these writes are flushed. + * + * Input Parameters: + * fobj - A reference to the file object to flush + * + * Returned Value: + * On success, then number of bytes flushed is returned. A negated errno + * value is returned on any failure. + * + ****************************************************************************/ + +int spiffs_stat_pgndx(FAR struct spiffs_s *fs, int16_t pgndx, int16_t objid, + FAR struct stat *buf); + +/**************************************************************************** + * Name: spiffs_find_fobj_bypgndx + * + * Description: + * Given the page index of the object header, find the corresponding file + * object instance. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * pgndx - The page index to match + * ppfobj - A user provided location in which to return the matching file + * file object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_find_fobj_bypgndx(FAR struct spiffs_s *fs, int16_t pgndx, + FAR struct spiffs_file_s **ppfobj); + +/**************************************************************************** + * Name: spiffs_find_fobj_byobjid + * + * Description: + * Given a object ID, find the corresponding file object instance + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * objid - The object ID to match + * ppfobj - A user provided location in which to return the matching file + * file object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_find_fobj_byobjid(FAR struct spiffs_s *fs, int16_t objid, + FAR struct spiffs_file_s **ppfobj); + +/**************************************************************************** + * Name: spiffs_fflush_cache + * + * Description: + * Checks if there are any cached writes for the object ID associated with + * given file object. If so, these writes are flushed. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * fobj - A reference to the file object to flush + * + * Returned Value: + * On success, then number of bytes flushed is returned. A negated errno + * value is returned on any failure. + * + ****************************************************************************/ + +ssize_t spiffs_fflush_cache(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj); + +/**************************************************************************** + * Name: spiffs_fobj_write + * + * Description: + * Write to a file object + * + * Input Parameters: + * fs - A reference to the volume structure + * fobj - A reference to the file object to write to + * buffer - The data to be written + * offset - The FLASH offset to be written + * len - The number of bytes to be written + * + * Returned Value: + * On success, then number of bytes written is returned. A negated errno + * value is returned on any failure. + * + ****************************************************************************/ + +ssize_t spiffs_fobj_write(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, + FAR const void *buffer, off_t offset, size_t len); + +/**************************************************************************** + * Name: spiffs_fobj_read + * + * Description: + * Read from a file object + * + * Input Parameters: + * fs - A reference to the volume structure + * fobj - A reference to the file object to read from + * buffer - The location that the data is read to + * offset - The FLASH offset to be read + * len - The number of bytes to be read + * fpos - The file position to read from + * + * Returned Value: + * On success, then number of bytes written is returned. A negated errno + * value is returned on any failure. + * + ****************************************************************************/ + +ssize_t spiffs_fobj_read(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, FAR void *buffer, + size_t buflen, off_t fpos); + +/**************************************************************************** + * Name: spiffs_fobj_free + * + * Description: + * Free all resources used by a file object + * + * Input Parameters: + * fs - A reference to the volume structure + * fobj - A reference to the file object to be removed + * + * Returned Value: + * None + * + ****************************************************************************/ + +void spiffs_fobj_free(FAR struct spiffs_s *fs, FAR struct spiffs_file_s *fobj); + +#if defined(__cplusplus) +} +#endif + +#endif /* __FS_SPIFFS_SRC_SPIFFS_H */ diff --git a/fs/spiffs/src/spiffs_cache.c b/fs/spiffs/src/spiffs_cache.c new file mode 100644 index 00000000000..3ecddfaee5c --- /dev/null +++ b/fs/spiffs/src/spiffs_cache.c @@ -0,0 +1,716 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_cache.c + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * This is a port of version 0.3.7 of SPIFFS by Peter Andersion. That + * version was originally released under the MIT license but is here re- + * released under the NuttX BSD license. + * + * Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976@gmail.com) + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include + +#include + +#include "spiffs.h" +#include "spiffs_mtd.h" +#include "spiffs_core.h" +#include "spiffs_cache.h" + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_cache_page_get + * + * Description: + * Returns cached page for give page index, or null if no such cached page + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * pgndx - Page index to get + * + * Returned Value: + * Reference to struct spiffs_cache_page_s instance. + * + ****************************************************************************/ + +FAR static FAR struct spiffs_cache_page_s * + spiffs_cache_page_get(FAR struct spiffs_s *fs, int16_t pgndx) +{ + FAR struct spiffs_cache_s *cache; + FAR struct spiffs_cache_page_s *cp; + int i; + + cache = spiffs_get_cache(fs); + if ((cache->cpage_use_map & cache->cpage_use_mask) == 0) + { + return 0; + } + + for (i = 0; i < cache->cpage_count; i++) + { + cp = spiffs_get_cache_page_hdr(fs, cache, i); + + if ((cache->cpage_use_map & (1 << i)) != 0 && + (cp->flags & SPIFFS_CACHE_FLAG_TYPE_WR) == 0 && + cp->pgndx == pgndx) + { + cp->last_access = cache->last_access; + return cp; + } + } + + return NULL; +} + +/**************************************************************************** + * Name: spiffs_cache_page_free + * + * Description: + * Frees cached page + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * cpndx - Cache page index + * write_back - True: Write dirty pages back to hardware + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +static int spiffs_cache_page_free(FAR struct spiffs_s *fs, int cpndx, + bool write_back) +{ + FAR struct spiffs_cache_s *cache; + FAR struct spiffs_cache_page_s *cp; + int ret = OK; + + cache = spiffs_get_cache(fs); + cp = spiffs_get_cache_page_hdr(fs, cache, cpndx); + + if (cache->cpage_use_map & (1 << cpndx)) + { + if (write_back && + (cp->flags & SPIFFS_CACHE_FLAG_TYPE_WR) == 0 && + (cp->flags & SPIFFS_CACHE_FLAG_DIRTY) != 0) + { + FAR uint8_t *mem; + + mem = spiffs_get_cache_page(fs, cache, cpndx); + + spiffs_cacheinfo("Write cache page=%d pgndx %04x\n", + cpndx, cp->pgndx); + + ret = spiffs_mtd_write(fs, SPIFFS_PAGE_TO_PADDR(fs, cp->pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), mem); + } + + if (cp->flags & SPIFFS_CACHE_FLAG_TYPE_WR) + { + spiffs_cacheinfo("Free cache page=%d objid=%04x\n", + cpndx, cp->objid); + } + else + { + spiffs_cacheinfo("Free cache page %d pgndx %04x\n", + cpndx, cp->pgndx); + } + + cache->cpage_use_map &= ~(1 << cpndx); + cp->flags = 0; + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_cache_page_remove_oldest + * + * Description: + * Removes the oldest accessed cached page + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * cpndx - Cache page index + * write_back - True: Write dirty pages back to hardware + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +static int spiffs_cache_page_remove_oldest(FAR struct spiffs_s *fs, + uint8_t flag_mask, uint8_t flags) +{ + FAR struct spiffs_cache_s *cache = spiffs_get_cache(fs); + uint32_t oldest_val = 0; + int cpndx = -1; + int ret = OK; + int i; + + /* Don't remove any cache pages unless there are no free cache pages */ + + if ((cache->cpage_use_map & cache->cpage_use_mask) != cache->cpage_use_mask) + { + /* At least one free cpage */ + + return OK; + } + + /* All busy, scan through all to find the cache page which has oldest + * access time. + */ + + for (i = 0; i < cache->cpage_count; i++) + { + FAR struct spiffs_cache_page_s *cp; + + cp = spiffs_get_cache_page_hdr(fs, cache, i); + if ((cache->last_access - cp->last_access) > oldest_val && + (cp->flags & flag_mask) == flags) + { + oldest_val = cache->last_access - cp->last_access; + cpndx = i; + } + } + + /* Then free that cache page */ + + if (cpndx >= 0) + { + ret = spiffs_cache_page_free(fs, cpndx, true); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_cache_page_allocate + * + * Description: + * Allocates a new cached page and returns it, or null if all cache pages + * are busy. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * cpndx - Cache page index + * write_back - True: Write dirty pages back to hardware + * + * Returned Value: + * A reference to the allocated cache page. NULL is returned if we were + * unable to allocated a cache page. + * + ****************************************************************************/ + +static FAR struct spiffs_cache_page_s * + spiffs_cache_page_allocate(FAR struct spiffs_s *fs) +{ + FAR struct spiffs_cache_s *cache; + int i; + + /* Check if any cache pages are available */ + + cache = spiffs_get_cache(fs); + if (cache->cpage_use_map == 0xffffffff) + { + /* No.. Out of cache memory */ + + return NULL; + } + + /* Search for a free cache page */ + + for (i = 0; i < cache->cpage_count; i++) + { + if ((cache->cpage_use_map & (1 << i)) == 0) + { + FAR struct spiffs_cache_page_s *cp; + + /* We found one */ + cp = spiffs_get_cache_page_hdr(fs, cache, i); + cache->cpage_use_map |= (1 << i); + cp->last_access = cache->last_access; + return cp; + } + } + + /* Out of cache entries */ + + return NULL; +} + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_cache_initialize + * + * Description: + * Initializes the cache + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * cp - The cache page to be released + * + * Returned Value: + * None + * + ****************************************************************************/ + +void spiffs_cache_initialize(FAR struct spiffs_s *fs) +{ + FAR struct spiffs_cache_s *cp; + struct spiffs_cache_s cache; + uint32_t cache_mask = 0; + uint32_t sz; + int i; + int cache_entries; + + sz = fs->cache_size; + cache_entries = (sz - sizeof(struct spiffs_cache_s )) / + (SPIFFS_CACHE_PAGE_SIZE(fs)); + + if (fs->cache == 0) + { + return; + } + + if (cache_entries <= 0) + { + return; + } + + for (i = 0; i < cache_entries; i++) + { + cache_mask <<= 1; + cache_mask |= 1; + } + + memset(&cache, 0, sizeof(struct spiffs_cache_s )); + cache.cpage_count = cache_entries; + cache.cpages = (FAR uint8_t*) + ((FAR uint8_t *)fs->cache + sizeof(struct spiffs_cache_s )); + + cache.cpage_use_map = 0xffffffff; + cache.cpage_use_mask = cache_mask; + memcpy(fs->cache, &cache, sizeof(struct spiffs_cache_s )); + + cp = spiffs_get_cache(fs); + memset(cp->cpages, 0, cp->cpage_count * SPIFFS_CACHE_PAGE_SIZE(fs)); + + cp->cpage_use_map &= ~(cp->cpage_use_mask); + for (i = 0; i < cache.cpage_count; i++) + { + spiffs_get_cache_page_hdr(fs, cp, i)->cpndx = i; + } +} + +/**************************************************************************** + * Name: spiffs_cache_drop_page + * + * Description: + * Drops the cache page for give page index + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * pgndx - Page index to be dropped + * + * Returned Value: + * None + * + ****************************************************************************/ + +void spiffs_cache_drop_page(FAR struct spiffs_s *fs, int16_t pgndx) +{ + FAR struct spiffs_cache_page_s *cp = spiffs_cache_page_get(fs, pgndx); + if (cp) + { + spiffs_cache_page_free(fs, cp->cpndx, false); + } +} + +/**************************************************************************** + * Name: spiffs_cache_read + * + * Description: + * Reads from SPI FLASH or the cache + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * op - Read options + * objid - File object ID to read + * addr - Address to read from + * len - The number of bytes to be read + * dest - The location in which the read data is to be returned. + * + * Returned Value: + * The number of bytes read (len) is returned on success; A negated + * errno value is returned on any failure. + * + ****************************************************************************/ + +ssize_t spiffs_cache_read(FAR struct spiffs_s *fs, uint8_t op, int16_t objid, + off_t addr, size_t len, FAR uint8_t *dest) +{ + FAR struct spiffs_cache_s *cache; + FAR struct spiffs_cache_page_s *cp; + int ret = OK; + + cache = spiffs_get_cache(fs); + cp = spiffs_cache_page_get(fs, SPIFFS_PADDR_TO_PAGE(fs, addr)); + + cache->last_access++; + if (cp != NULL) + { + FAR uint8_t *mem; + + /* We've already got a cache page */ + +#ifdef CONFIG_SPIFFS_CACHEDBG + fs->cache_hits++; +#endif + + cp->last_access = cache->last_access; + mem = spiffs_get_cache_page(fs, cache, cp->cpndx); + memcpy(dest, &mem[SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr)], len); + } + else + { + /* Check for second layer lookup */ + + if ((op & SPIFFS_OP_TYPE_MASK) == SPIFFS_OP_T_OBJ_LU2) + { + /* For second layer lookup functions, we do not cache in order to + * prevent shredding + */ + + ret = spiffs_mtd_read(fs, addr, len, dest); + } + else + { +#ifdef CONFIG_SPIFFS_CACHEDBG + fs->cache_misses++; +#endif + + /* This operation will always free one cache page (unless all + * already free), the result code stems from the write operation + * of the possibly freed cache page + */ + + ret = spiffs_cache_page_remove_oldest(fs, + SPIFFS_CACHE_FLAG_TYPE_WR, + 0); + + /* Allocate a new cache page */ + + cp = spiffs_cache_page_allocate(fs); + if (cp != NULL) + { + FAR uint8_t *mem; + + cp->flags = SPIFFS_CACHE_FLAG_WRTHRU; + cp->pgndx = SPIFFS_PADDR_TO_PAGE(fs, addr); + + spiffs_cacheinfo("Allocated cache page %d for pgndx %04x\n", + cp->cpndx, cp->pgndx); + + ret = spiffs_mtd_read(fs, addr - + SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr), + SPIFFS_CFG_LOG_PAGE_SZ(fs), + spiffs_get_cache_page(fs, cache, cp->cpndx)); + + mem = spiffs_get_cache_page(fs, cache, cp->cpndx); + memcpy(dest, &mem[SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr)], len); + } + else + { + /* This will never happen, last resort for sake of symmetry */ + + ret = spiffs_mtd_read(fs, addr, len, dest); + } + } + } + + if (ret < 0) + { + ferr("ERROR: spiffs_mtd_read: failed: %d\n", ret); + return (ssize_t)ret; + } + + return (ssize_t)len; +} + +/**************************************************************************** + * Name: spiffs_cache_write + * + * Description: + * Writes to SPI FLASH and/or the cache + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * op - Write options + * objid - File object ID to write + * addr - Address to write to + * len - The number of bytes to be write + * dest - The location that provide the data to be written. + * + * Returned Value: + * The number of bytes written (len) is returned on success; A negated + * errno value is returned on any failure. + * + ****************************************************************************/ + +size_t spiffs_cache_write(FAR struct spiffs_s *fs, uint8_t op, int16_t objid, + off_t addr, size_t len, FAR uint8_t *src) +{ + FAR struct spiffs_cache_s *cache; + FAR struct spiffs_cache_page_s *cp; + int16_t pgndx; + int ret = OK; + + pgndx = SPIFFS_PADDR_TO_PAGE(fs, addr); + cache = spiffs_get_cache(fs); + cp = spiffs_cache_page_get(fs, pgndx); + + /* Check for write-through */ + + if (cp != NULL && (op & SPIFFS_OP_COM_MASK) != SPIFFS_OP_C_WRTHRU) + { + /* Have a cache page. Copy in data to cache page */ + + if ((op & SPIFFS_OP_COM_MASK) == SPIFFS_OP_C_DELE && + (op & SPIFFS_OP_TYPE_MASK) != SPIFFS_OP_T_OBJ_LU) + { + /* Page is being deleted, wipe from cache - unless it is a lookup + * page + */ + + spiffs_cache_page_free(fs, cp->cpndx, false); + ret = spiffs_mtd_write(fs, addr, len, src); + } + else + { + uint8_t *mem = spiffs_get_cache_page(fs, cache, cp->cpndx); + memcpy(&mem[SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr)], src, len); + + cache->last_access++; + cp->last_access = cache->last_access; + + if (cp->flags & SPIFFS_CACHE_FLAG_WRTHRU) + { + /* Page is being updated, no write-cache, just pass through */ + + ret = spiffs_mtd_write(fs, addr, len, src); + } + else + { + ret = OK; + } + } + } + else + { + /* No cache page, no write cache - just write through */ + + ret = spiffs_mtd_write(fs, addr, len, src); + } + + if (ret < 0) + { + ferr("ERROR: spiffs_mtd_write: failed: %d\n", ret); + return (ssize_t)ret; + } + + return (ssize_t)len; +} + +/**************************************************************************** + * Name: spiffs_cache_write + * + * Description: + * Returns the cache page that this file object refers to (or NULL if + * there is no cache page + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * fobj - The file object instance + * + * Returned Value: + * Returns the cache page that this file object refers to (or NULL if + * there is no cache page + * + ****************************************************************************/ + +FAR struct spiffs_cache_page_s * + spiffs_cache_page_get_byobjid(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj) +{ + FAR struct spiffs_cache_s *cache = spiffs_get_cache(fs); + int i; + + if ((cache->cpage_use_map & cache->cpage_use_mask) == 0) + { + /* All cache pages free, no cache page can be assigned to the ID */ + + return NULL; + } + + /* Look at each cache index */ + + for (i = 0; i < cache->cpage_count; i++) + { + FAR struct spiffs_cache_page_s *cp; + + /* Is this page available? Is is writable? Do the object IDs match? */ + + cp = spiffs_get_cache_page_hdr(fs, cache, i); + if ((cache->cpage_use_map & (1 << i)) && + (cp->flags & SPIFFS_CACHE_FLAG_TYPE_WR) && + cp->objid == fobj->objid) + { + /* Yes... return the cache page reference */ + + return cp; + } + } + + /* Not found */ + + return NULL; +} + +/**************************************************************************** + * Name: spiffs_cache_page_allocate_byobjid + * + * Description: + * Allocates a new cache page and refers this to given object ID. It + * flushes an old cache page if all cache pages are busy + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * fobj - The file object instance + * + * Returned Value: + * Returns the allocated cache page(or NULL if the cache page could not + * be allocated). + * + ****************************************************************************/ + +FAR struct spiffs_cache_page_s * + spiffs_cache_page_allocate_byobjid(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj) +{ + FAR struct spiffs_cache_page_s *cp; + + /* before this function is called, it is ensured that there is no already + * existing cache page with same object ID + */ + + spiffs_cache_page_remove_oldest(fs, SPIFFS_CACHE_FLAG_TYPE_WR, 0); + + cp = spiffs_cache_page_allocate(fs); + if (cp == NULL) + { + /* could not get cache page */ + + return NULL; + } + + cp->flags = SPIFFS_CACHE_FLAG_TYPE_WR; + cp->objid = fobj->objid; + fobj->cache_page = cp; + + spiffs_cacheinfo("Allocated cache page %d for objid=%d\n", + cp->cpndx, fobj->objid); + return cp; +} + +/**************************************************************************** + * Name: spiffs_cache_page_release + * + * Description: + * "Unrefers" all file objects that this cache page refers to and releases + * the cache page + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * cp - The cache page to be released + * + * Returned Value: + * None + * + ****************************************************************************/ + +void spiffs_cache_page_release(FAR struct spiffs_s *fs, + FAR struct spiffs_cache_page_s *cp) +{ + FAR struct spiffs_file_s *fobj; + + /* Verify that a non-NULL cache page was provided */ + + if (cp != NULL) + { + /* Visit each file object */ + + for (fobj = (FAR struct spiffs_file_s *)dq_peek(&fs->objq); + fobj != NULL; + fobj = (FAR struct spiffs_file_s *)dq_next((FAR dq_entry_t *)fobj)) + { + /* "Unrefer" the cache page */ + + if (fobj->cache_page == cp) + { + fobj->cache_page = NULL; + } + } + + /* Free the cache page */ + + spiffs_cache_page_free(fs, cp->cpndx, false); + cp->objid = 0; + } +} diff --git a/fs/spiffs/src/spiffs_cache.h b/fs/spiffs/src/spiffs_cache.h new file mode 100644 index 00000000000..ac78667ab0a --- /dev/null +++ b/fs/spiffs/src/spiffs_cache.h @@ -0,0 +1,282 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_cache.h + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +#ifndef __FS_SPIFFS_SRC_SPIFFS_CACHE_H +#define __FS_SPIFFS_SRC_SPIFFS_CACHE_H + +#if defined(__cplusplus) +extern "C" +{ +#endif + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include + +/**************************************************************************** + * Pre-processor Definitions + ****************************************************************************/ + +#define SPIFFS_CACHE_FLAG_DIRTY (1 << 0) +#define SPIFFS_CACHE_FLAG_WRTHRU (1 << 1) +#define SPIFFS_CACHE_FLAG_OBJLU (1 << 2) +#define SPIFFS_CACHE_FLAG_OBJIX (1 << 3) +#define SPIFFS_CACHE_FLAG_DATA (1 << 4) +#define SPIFFS_CACHE_FLAG_TYPE_WR (1 << 7) + +#define SPIFFS_CACHE_PAGE_SIZE(fs) \ + (sizeof(struct spiffs_cache_page_s) + SPIFFS_CFG_LOG_PAGE_SZ(fs)) + +#define spiffs_get_cache(fs) \ + ((FAR struct spiffs_cache_s *)((fs)->cache)) + +#define spiffs_get_cache_page_hdr(fs, c, cpndx) \ + ((FAR struct spiffs_cache_page_s *)(&((c)->cpages[(cpndx) * \ + SPIFFS_CACHE_PAGE_SIZE(fs)]))) + +#define spiffs_get_cache_page(fs, c, cpndx) \ + ((FAR uint8_t *)(&((c)->cpages[(cpndx) * SPIFFS_CACHE_PAGE_SIZE(fs)])) + \ + sizeof(struct spiffs_cache_page_s)) + +/* Debug */ + +#ifdef CONFIG_SPIFFS_CACHEDBG +# ifdef CONFIG_CPP_HAVE_VARARGS +# define spiffs_cacheinfo(format, ...) _info(format, ##__VA_ARGS__) +# else +# define spiffs_cacheinfo _info +# endif +#else +# ifdef CONFIG_CPP_HAVE_VARARGS +# define spiffs_cacheinfo(format, ...) +# else +# define spiffs_cacheinfo (void) +# endif +#endif + +/**************************************************************************** + * Public Types + ****************************************************************************/ + +/* cache page struct */ + +struct spiffs_cache_page_s +{ + uint8_t flags; /* Cache flags */ + uint8_t cpndx; /* Cache page index */ + uint32_t last_access; /* Last access of this cache page */ + union + { + struct /* Type read cache */ + { + int16_t pgndx; /* Read cache page index */ + }; + + struct /* Type write cache */ + { + int16_t objid; /* Write cache */ + uint32_t offset; /* Offset in cache page */ + uint16_t size; /* Size of cache page */ + }; + }; +}; + +/* Cache structure */ + +struct spiffs_cache_s +{ + uint8_t cpage_count; + uint32_t last_access; + uint32_t cpage_use_map; + uint32_t cpage_use_mask; + FAR uint8_t *cpages; +}; + +/**************************************************************************** + * Public Function Prototypes + ****************************************************************************/ + +struct spiffs_s; /* Forward reference */ + +/**************************************************************************** + * Name: spiffs_cache_initialize + * + * Description: + * Initializes the cache + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * cp - The cache page to be released + * + * Returned Value: + * None + * + ****************************************************************************/ + +void spiffs_cache_initialize(FAR struct spiffs_s *fs); + +/**************************************************************************** + * Name: spiffs_cache_drop_page + * + * Description: + * Drops the cache page for give page index + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * pgndx - Page index to be dropped + * + * Returned Value: + * None + * + ****************************************************************************/ + +void spiffs_cache_drop_page(FAR struct spiffs_s *fs, int16_t pgndx); + +/**************************************************************************** + * Name: spiffs_cache_read + * + * Description: + * Reads from SPI FLASH or the cache + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * op - Read options + * objid - File object ID to read + * addr - Address to read from + * len - The number of bytes to be read + * dest - The location in which the read data is to be returned. + * + * Returned Value: + * The number of bytes read (len) is returned on success; A negated + * errno value is returned on any failure. + * + ****************************************************************************/ + +ssize_t spiffs_cache_read(FAR struct spiffs_s *fs, uint8_t op, int16_t objid, + off_t addr, size_t len, FAR uint8_t *dest); + +/**************************************************************************** + * Name: spiffs_cache_write + * + * Description: + * Writes to SPI FLASH and/or the cache + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * op - Write options + * objid - File object ID to write + * addr - Address to write to + * len - The number of bytes to be write + * dest - The location that provide the data to be written. + * + * Returned Value: + * The number of bytes written (len) is returned on success; A negated + * errno value is returned on any failure. + * + ****************************************************************************/ + +size_t spiffs_cache_write(FAR struct spiffs_s *fs, uint8_t op, int16_t objid, + off_t addr, size_t len, FAR uint8_t *src); + +/**************************************************************************** + * Name: spiffs_cache_write + * + * Description: + * Returns the cache page that this file object refers to (or NULL if + * there is no cache page + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * fobj - The file object instance + * + * Returned Value: + * Returns the cache page that this file object refers to (or NULL if + * there is no cache page + * + ****************************************************************************/ + +FAR struct spiffs_cache_page_s * + spiffs_cache_page_get_byobjid(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj); + +/**************************************************************************** + * Name: spiffs_cache_page_allocate_byobjid + * + * Description: + * Allocates a new cache page and refers this to given object ID. It + * flushes an old cache page if all cache pages are busy + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * fobj - The file object instance + * + * Returned Value: + * Returns the allocated cache page(or NULL if the cache page could not + * be allocated). + * + ****************************************************************************/ + +FAR struct spiffs_cache_page_s * + spiffs_cache_page_allocate_byobjid(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj); + +/**************************************************************************** + * Name: spiffs_cache_page_release + * + * Description: + * "Unrefers" all file objects that this cache page refers to and releases + * the cache page + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * cp - The cache page to be released + * + * Returned Value: + * None + * + ****************************************************************************/ + +void spiffs_cache_page_release(FAR struct spiffs_s *fs, + FAR struct spiffs_cache_page_s *cp); + +#if defined(__cplusplus) +} +#endif + +#endif /* __FS_SPIFFS_SRC_SPIFFS_CACHE_H */ diff --git a/fs/spiffs/src/spiffs_check.c b/fs/spiffs/src/spiffs_check.c new file mode 100644 index 00000000000..755706d08f7 --- /dev/null +++ b/fs/spiffs/src/spiffs_check.c @@ -0,0 +1,2027 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_check.c + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * This is a port of version 0.3.7 of SPIFFS by Peter Andersion. That + * version was originally released under the MIT license but is here re- + * released under the NuttX BSD license. + * + * Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976@gmail.com) + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +/* Contains functionality for checking file system consistency + * and mending problems. + * Three levels of consistency checks are implemented: + * + * Look up consistency + * Checks if indices in lookup pages are coherent with page headers + * Object index consistency + * Checks if there are any orphaned object indices (missing object index + * headers). + * If an object index is found but not its header, the object index is + * deleted. + * This is critical for the following page consistency check. + * Page consistency + * Checks for pages that ought to be indexed, ought not to be indexed, are + * multiple indexed + */ + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include + +#include "spiffs.h" +#include "spiffs_core.h" +#include "spiffs_cache.h" +#include "spiffs_check.h" + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_check_get_data_pgndx + * + * Description: + * Searches in the object indices and returns the referenced page index + * given the object ID and the data span index destroys fs->lu_work + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * objid - Object ID + * data_spndx - Data span index + * pgnx - Page index + * objndx_pgndx - Object index page index + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +static int spiffs_check_get_data_pgndx(FAR struct spiffs_s *fs, + int16_t objid, int16_t data_spndx, + FAR int16_t *pgndx, + FAR int16_t *objndx_pgndx) +{ + uint32_t addr; + int16_t objndx_spndx; + int ret; + + /* Calculate object index span index for given data page span index */ + + objndx_spndx = SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, data_spndx); + + /* Find the object index for the object ID and span index */ + + ret = spiffs_obj_lu_find_id_and_span(fs, objid | SPIFFS_OBJ_ID_IX_FLAG, + objndx_spndx, 0, objndx_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + + /* Load the object index entry */ + + addr = SPIFFS_PAGE_TO_PADDR(fs, *objndx_pgndx); + if (objndx_spndx == 0) + { + /* Get the referenced page from object index header */ + + addr += sizeof(struct spiffs_pgobj_ixheader_s) + + data_spndx * sizeof(int16_t); + } + else + { + /* Get the referenced page from object index */ + + addr += sizeof(spiffs_page_object_ix) + + SPIFFS_OBJ_IX_ENTRY(fs, data_spndx) * + sizeof(int16_t); + } + + /* Read the page from FLASH (or the cache) */ + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, 0, + addr, sizeof(int16_t), (FAR uint8_t *)pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_check_rewrite_page + * + * Description: + * Copies page contents to a new page + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * cur_pgndx - Current page index + * pghdr - Reference to page header + * new_pgndx - Location to return the new page index + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +static int spiffs_check_rewrite_page(FAR struct spiffs_s *fs, + int16_t cur_pgndx, + FAR struct spiffs_page_header_s *pghdr, + FAR int16_t *new_pgndx) +{ + int ret; + + ret = spiffs_page_allocate_data(fs, pghdr->objid, pghdr, 0, 0, 0, 0, + new_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_allocate_data() failed: %d\n", ret); + return ret; + } + + ret = spiffs_phys_cpy(fs, 0, + SPIFFS_PAGE_TO_PADDR(fs, *new_pgndx) + + sizeof(struct spiffs_page_header_s), + SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx) + + sizeof(struct spiffs_page_header_s), + SPIFFS_DATA_PAGE_SIZE(fs)); + if (ret < 0) + { + ferr("ERROR: spiffs_phys_cpy() failed: %d\n", ret); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_check_rewrite_index + * + * Description: + * Rewrites the object index for given object ID and replaces the + * data page index to a new page index + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * objid - Object ID + * data_spndx - Data span index + * pgnx - Page index + * objndx_pgndx - Object index page index + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +static int spiffs_check_rewrite_index(FAR struct spiffs_s *fs, + int16_t objid, int16_t data_spndx, + int16_t new_data_pgndx, + int16_t objndx_pgndx) +{ + FAR struct spiffs_page_header_s *objndx_phdr; + int16_t blkndx; + int16_t free_pgndx; + int entry; + int ret; + + objid |= SPIFFS_OBJ_ID_IX_FLAG; + + /* Find free entry */ + + ret = spiffs_obj_lu_find_free(fs, fs->free_blkndx, + fs->free_entry, &blkndx, &entry); + if (ret < 0) + { + fwarn("WARNING: spiffs_obj_lu_find_free() failed: %d\n", ret); + return ret; + } + + free_pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, entry); + + /* Calculate object index span index for given data page span index */ + + int16_t objndx_spndx = SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, data_spndx); + if (objndx_spndx == 0) + { + /* Calculate index in index header */ + + entry = data_spndx; + } + else + { + /* Calculate entry in index */ + + entry = SPIFFS_OBJ_IX_ENTRY(fs, data_spndx); + } + + /* Load index */ + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + objndx_phdr = (FAR struct spiffs_page_header_s *)fs->lu_work; + + /* Be ultra safe, double check header against provided data. Return -EFAULT + * to indicate this condition. + */ + + if (objndx_phdr->objid != objid) + { + spiffs_page_delete(fs, free_pgndx); + return -EFAULT; + } + + if (objndx_phdr->spndx != objndx_spndx) + { + spiffs_page_delete(fs, free_pgndx); + return -EFAULT; + } + + if ((objndx_phdr->flags & (SPIFFS_PH_FLAG_USED | SPIFFS_PH_FLAG_IXDELE | + SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_FINAL | + SPIFFS_PH_FLAG_DELET)) != + (SPIFFS_PH_FLAG_IXDELE | SPIFFS_PH_FLAG_DELET)) + { + spiffs_page_delete(fs, free_pgndx); + return -EFAULT; + } + + /* Rewrite in memory */ + + if (objndx_spndx == 0) + { + ((FAR int16_t *)((FAR uint8_t *)fs->lu_work + + sizeof(struct spiffs_pgobj_ixheader_s)))[data_spndx] = + new_data_pgndx; + } + else + { + ((FAR int16_t *)((FAR uint8_t *)fs->lu_work + + sizeof(spiffs_page_object_ix)))[SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)] = + new_data_pgndx; + } + + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, 0, + SPIFFS_PAGE_TO_PADDR(fs, free_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_write() failed: %d\n", ret); + return ret; + } + + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_UPDT, 0, + SPIFFS_BLOCK_TO_PADDR(fs, SPIFFS_BLOCK_FOR_PAGE(fs,free_pgndx)) + + SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(fs, free_pgndx) * + sizeof(int16_t), sizeof(int16_t), + (FAR uint8_t *)&objid); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_write() failed: %d\n", ret); + return ret; + } + + ret = spiffs_page_delete(fs, objndx_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_check_delobj_lazy + * + * Description: + * Deletes an object just by marking object index header as deleted + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * objid - Object ID to be deleted + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +static int spiffs_check_delobj_lazy(FAR struct spiffs_s *fs, int16_t objid) +{ + int16_t objhdr_pgndx; + uint8_t flags = 0xff; + int ret; + + ret = spiffs_obj_lu_find_id_and_span(fs, objid, 0, 0, &objhdr_pgndx); + if (ret == -ENOENT) + { + return OK; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + +#ifdef CONFIG_SPIFFS_NO_BLIND_WRITES + /* Perform a read-modify-write */ + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, 0, + SPIFFS_PAGE_TO_PADDR(fs, objhdr_pgndx) + + offsetof(struct spiffs_page_header_s, flags), sizeof(flags), + &flags); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } +#endif + + /* Clear the deleted flag in FLASH to mark the page deleted */ + + flags &= ~SPIFFS_PH_FLAG_IXDELE; + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_UPDT, 0, + SPIFFS_PAGE_TO_PADDR(fs, objhdr_pgndx) + + offsetof(struct spiffs_page_header_s, flags), + sizeof(flags), &flags); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_write() failed: %d\n", ret); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_check_rewrite_index + * + * Description: + * Validates the given look up entry + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * objid - Object ID + * pghdr - Page header + * cur_pgndx - Current page index + * cur_block - Current block + * cur_entry - Current entry + * reload_lu - Reload lookup entry (returned) + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +static int spiffs_check_luentry_validate(FAR struct spiffs_s *fs, + int16_t lu_objid, + FAR struct spiffs_page_header_s *pghdr, + int16_t cur_pgndx, + int16_t cur_block, + int cur_entry, + FAR bool *reload_lu) +{ + int16_t objndx_pgndx; + int16_t ref_pgndx; + bool delete_page = false; + int ret = OK; + + /* Check validity, take actions */ + + if (((lu_objid == SPIFFS_OBJ_ID_DELETED) && + (pghdr->flags & SPIFFS_PH_FLAG_DELET)) || + ((lu_objid == SPIFFS_OBJ_ID_FREE) && + (pghdr->flags & SPIFFS_PH_FLAG_USED) == 0)) + { + /* Look up entry deleted / free but used in page header */ + + spiffs_checkinfo("pgndx=%04x deleted/free in lu but not on page\n", + cur_pgndx); + + *reload_lu = true; + delete_page = true; + + if (pghdr->flags & SPIFFS_PH_FLAG_INDEX) + { + /* Header says data page data page can be removed if not + * referenced by some object index + */ + + ret = spiffs_check_get_data_pgndx(fs, pghdr->objid, + pghdr->spndx, + &ref_pgndx, &objndx_pgndx); + if (ret == -ENOENT) + { + /* No object with this objid, so remove page safely */ + + ret = OK; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_check_get_data_pgndx() failed: %d\n", ret); + return ret; + } + else if (ref_pgndx == cur_pgndx) + { + int16_t new_pgndx; + + /* Data page referenced by object index but deleted in lu copy + * page to new place and re-write the object index to new place + */ + + ret = spiffs_check_rewrite_page(fs, cur_pgndx, pghdr, &new_pgndx); + + spiffs_checkinfo("Data page not found elsewhere, rewriting %04x to new page %04x\n", + cur_pgndx, new_pgndx); + + if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_page() failed: %d\n", ret); + return ret; + } + + *reload_lu = true; + + spiffs_checkinfo("Page %04x rewritten to %04x, affected objndx_pgndx %04x\n", + cur_pgndx, new_pgndx, objndx_pgndx); + + ret = spiffs_check_rewrite_index(fs, pghdr->objid, pghdr->spndx, + new_pgndx, objndx_pgndx); + if (ret == -EFAULT) + { + int ret2; + + /* Index bad also, cannot mend this file */ + + spiffs_checkinfo("Index bad %d, cannot mend!\n", ret); + + ret2 = spiffs_page_delete(fs, new_pgndx); + if (ret2 < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret2); + return ret2; + } + + ret2 = spiffs_check_delobj_lazy(fs, pghdr->objid); + if (ret2 < 0) + { + ferr("ERROR: spiffs_check_delobj_lazy() failed: %d\n", ret2); + return ret2; + } + } + + if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_index() failed: %d\n", ret); + return ret; + } + } + } + else + { + /* Header says index page index page can be removed if other index + * with same objid and span index is found + */ + + ret = spiffs_obj_lu_find_id_and_span(fs, + pghdr->objid | SPIFFS_OBJ_ID_IX_FLAG, + pghdr->spndx, cur_pgndx, 0); + if (ret == -ENOENT) + { + /* No such index page found, check for a data page amongst page + * headers. lu cannot be trusted + */ + + ret = spiffs_obj_lu_find_id_and_span_byphdr(fs, + pghdr->objid | SPIFFS_OBJ_ID_IX_FLAG, + 0, 0, 0); + if (ret == OK) + { + int16_t new_pgndx; + + /* Ignore other errors. Got a data page also, assume lu + * corruption only, rewrite to new page + */ + + ret = spiffs_check_rewrite_page(fs, cur_pgndx, pghdr, &new_pgndx); + + spiffs_checkinfo("Index page with data not found elsewhere, " + "rewriting %04x to new page %04x\n", + cur_pgndx, new_pgndx); + + if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_page() failed: %d\n", ret); + return ret; + } + + *reload_lu = true; + } + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span_byphdr() failed: %d\n", ret); + return ret; + } + } + } + + if (lu_objid != SPIFFS_OBJ_ID_FREE && lu_objid != SPIFFS_OBJ_ID_DELETED) + { + /* look up entry used */ + + if ((pghdr->objid | SPIFFS_OBJ_ID_IX_FLAG) != + (lu_objid | SPIFFS_OBJ_ID_IX_FLAG)) + { + spiffs_checkinfo("LU: pgndx %04x differ in objid lu=" + "%04x ph:%04x\n", cur_pgndx, lu_objid, + pghdr->objid); + delete_page = true; + if ((pghdr->flags & SPIFFS_PH_FLAG_DELET) == 0 || + (pghdr->flags & SPIFFS_PH_FLAG_FINAL) || + (pghdr->flags & (SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_IXDELE)) == 0) + { + /* Page deleted or not finalized, just remove it */ + } + else if (pghdr->flags & SPIFFS_PH_FLAG_INDEX) + { + /* if data page, check for reference to this page */ + + ret = spiffs_check_get_data_pgndx(fs, + pghdr->objid, + pghdr->spndx, + &ref_pgndx, + &objndx_pgndx); + if (ret == -ENOENT) + { + /* no object with this objid, so remove page safely */ + + ret = OK; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_check_get_data_pgndx() failed: %d\n", ret); + return ret; + } + + /* if found, rewrite page with object ID, update index, and + * delete current + */ + + else if (ref_pgndx == cur_pgndx) + { + int16_t new_pgndx; + + ret = spiffs_check_rewrite_page(fs, cur_pgndx, pghdr, + &new_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_page() failed: %d\n", ret); + return ret; + } + + ret = spiffs_check_rewrite_index(fs, pghdr->objid, + pghdr->spndx, new_pgndx, + objndx_pgndx); + if (ret == -EFAULT) + { + int ret2; + + /* Index bad also, cannot mend this file */ + + spiffs_checkinfo("Index bad %d, cannot mend!\n", ret); + + ret2 = spiffs_page_delete(fs, new_pgndx); + if (ret2 < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret2); + return ret2; + } + + ret2 = spiffs_check_delobj_lazy(fs, pghdr->objid); + if (ret2 < 0) + { + ferr("ERROR: spiffs_check_delobj_lazy() failed: %d\n", ret2); + return ret2; + } + + *reload_lu = true; + } + + if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_index() failed: %d\n", ret); + return ret; + } + } + } + else + { + int16_t objndx_pgndx_lu; + int16_t objndx_pgndx_ph; + + /* Else if index, check for other pages with both ID's and + * span index + * + * See if other object index page exists for lookup objid + * and span index + */ + + ret = spiffs_obj_lu_find_id_and_span(fs, + lu_objid | SPIFFS_OBJ_ID_IX_FLAG, + pghdr->spndx, 0, + &objndx_pgndx_lu); + if (ret == -ENOENT) + { + ret = OK; + objndx_pgndx_lu = 0; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + + /* See if other object index exists for page header objid and + * span index + */ + + ret = spiffs_obj_lu_find_id_and_span(fs, + pghdr->objid | SPIFFS_OBJ_ID_IX_FLAG, + pghdr->spndx, 0, + &objndx_pgndx_ph); + if (ret == -ENOENT) + { + ret = OK; + objndx_pgndx_ph = 0; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + + /* If both ID's found, just delete current */ + + if (objndx_pgndx_ph == 0 || objndx_pgndx_lu == 0) + { + struct spiffs_page_header_s new_ph; + int16_t data_pgndx_lu; + int16_t data_pgndx_ph; + int16_t new_pgndx; + + /* Otherwise try finding first corresponding data pages. + * + * See if other data page exists for look up objid and + * span index + */ + + ret = spiffs_obj_lu_find_id_and_span(fs, + lu_objid & ~SPIFFS_OBJ_ID_IX_FLAG, + 0, 0, &data_pgndx_lu); + if (ret == -ENOENT) + { + ret = OK; + objndx_pgndx_lu = 0; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + + /* See if other data page exists for page header objid + * and span index + */ + + ret = spiffs_obj_lu_find_id_and_span(fs, + pghdr->objid & ~SPIFFS_OBJ_ID_IX_FLAG, + 0, 0, &data_pgndx_ph); + if (ret == -ENOENT) + { + ret = OK; + objndx_pgndx_ph = 0; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + + new_ph.flags = 0xff & ~(SPIFFS_PH_FLAG_USED | SPIFFS_PH_FLAG_INDEX | + SPIFFS_PH_FLAG_FINAL); + new_ph.spndx = pghdr->spndx; + + if ((objndx_pgndx_lu != 0 && data_pgndx_lu != 0 && + data_pgndx_ph != 0 && objndx_pgndx_ph == 0) || + (objndx_pgndx_lu == 0 && data_pgndx_ph && + objndx_pgndx_ph == 0)) + { + /* Got a data page for page header objid rewrite as objid_ph */ + + new_ph.objid = pghdr->objid | SPIFFS_OBJ_ID_IX_FLAG; + ret = spiffs_check_rewrite_page(fs, cur_pgndx, &new_ph, &new_pgndx); + + spiffs_checkinfo("Rewrite page %04x as %04x to pgndx %04x\n", + cur_pgndx, new_ph.objid, new_pgndx); + + if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_page() failed: %d\n", ret); + return ret; + } + + *reload_lu = true; + } + else if ((objndx_pgndx_ph != 0 && data_pgndx_ph != 0 && + data_pgndx_lu != 0 && objndx_pgndx_lu == 0) || + (objndx_pgndx_ph == 0 && data_pgndx_lu && + objndx_pgndx_lu == 0)) + { + /* Got a data page for look up objid rewrite as objid_lu */ + + new_ph.objid = lu_objid | SPIFFS_OBJ_ID_IX_FLAG; + + spiffs_checkinfo("Rewrite page %04x as %04x\n", + cur_pgndx, new_ph.objid); + + ret = spiffs_check_rewrite_page(fs, cur_pgndx, &new_ph, &new_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_page() failed: %d\n", ret); + return ret; + } + + *reload_lu = true; + } + else + { + /* Cannot safely do anything */ + + spiffs_checkinfo("Nothing to do, just delete\n"); + } + } + } + } + else if (((lu_objid & SPIFFS_OBJ_ID_IX_FLAG) != 0 && + (pghdr->flags & SPIFFS_PH_FLAG_INDEX) != 0) || + ((lu_objid & SPIFFS_OBJ_ID_IX_FLAG) == 0 && + (pghdr->flags & SPIFFS_PH_FLAG_INDEX) == 0)) + { + int16_t data_pgndx; + int16_t objndx_pgndx_d; + + spiffs_checkinfo("%04x lu/page index marking differ\n", cur_pgndx); + + /* see if other data page exists for given objid and span index */ + + ret = spiffs_obj_lu_find_id_and_span(fs, + lu_objid & ~SPIFFS_OBJ_ID_IX_FLAG, + pghdr->spndx, cur_pgndx, &data_pgndx); + if (ret == -ENOENT) + { + ret = OK; + data_pgndx = 0; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + + /* See if other object index exists for given objid and span index */ + + ret = spiffs_obj_lu_find_id_and_span(fs, + lu_objid | SPIFFS_OBJ_ID_IX_FLAG, + pghdr->spndx, cur_pgndx, + &objndx_pgndx_d); + if (ret == -ENOENT) + { + ret = OK; + objndx_pgndx_d = 0; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + + delete_page = true; + + /* If other data page exists and object index exists, just delete + * page + */ + + if (data_pgndx != 0 && objndx_pgndx_d != 0) + { + spiffs_checkinfo("Other index and data page exists, simply remove\n"); + } + + /* If only data page exists, make this page index */ + + else if (data_pgndx && objndx_pgndx_d == 0) + { + struct spiffs_page_header_s new_ph; + int16_t new_pgndx; + + spiffs_checkinfo("Other data page exists, make this index\n"); + + new_ph.flags = 0xff & ~(SPIFFS_PH_FLAG_USED | SPIFFS_PH_FLAG_FINAL | + SPIFFS_PH_FLAG_INDEX); + new_ph.objid = lu_objid | SPIFFS_OBJ_ID_IX_FLAG; + new_ph.spndx = pghdr->spndx; + + ret = spiffs_page_allocate_data(fs, new_ph.objid, &new_ph, + 0, 0, 0, 1, &new_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_allocate_data() failed: %d\n", ret); + return ret; + } + + ret = spiffs_phys_cpy(fs, 0, SPIFFS_PAGE_TO_PADDR(fs, new_pgndx) + + sizeof(struct spiffs_page_header_s), + SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx) + + sizeof(struct spiffs_page_header_s), + SPIFFS_CFG_LOG_PAGE_SZ(fs) - + sizeof(struct spiffs_page_header_s)); + if (ret < 0) + { + ferr("ERROR: spiffs_phys_cpy() failed: %d\n", ret); + return ret; + } + } + + /* If only index exists, make data page */ + + else if (data_pgndx == 0 && objndx_pgndx_d) + { + struct spiffs_page_header_s new_ph; + int16_t new_pgndx; + + spiffs_checkinfo("Other index page exists, make this data\n"); + + new_ph.flags = 0xff & ~(SPIFFS_PH_FLAG_USED | SPIFFS_PH_FLAG_FINAL); + new_ph.objid = lu_objid & ~SPIFFS_OBJ_ID_IX_FLAG; + new_ph.spndx = pghdr->spndx; + + ret = spiffs_page_allocate_data(fs, new_ph.objid, &new_ph, + 0, 0, 0, 1, &new_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_allocate_data() failed: %d\n", ret); + return ret; + } + + ret = spiffs_phys_cpy(fs, 0, SPIFFS_PAGE_TO_PADDR(fs, new_pgndx) + + sizeof(struct spiffs_page_header_s), + SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx) + + sizeof(struct spiffs_page_header_s), + SPIFFS_CFG_LOG_PAGE_SZ(fs) - + sizeof(struct spiffs_page_header_s)); + if (ret < 0) + { + ferr("ERROR: spiffs_phys_cpy() failed: %d\n", ret); + return ret; + } + } + else + { + /* If nothing exists, we cannot safely make a decision - delete */ + } + } + else if ((pghdr->flags & SPIFFS_PH_FLAG_DELET) == 0) + { + spiffs_checkinfo("pgndx=%04x busy in lu but deleted on page\n", + cur_pgndx); + delete_page = 1; + } + else if ((pghdr->flags & SPIFFS_PH_FLAG_FINAL)) + { + spiffs_checkinfo("LU: pgndx %04x busy but not final\n", + cur_pgndx); + + /* Page can be removed if not referenced by object index */ + + *reload_lu = true; + ret = spiffs_check_get_data_pgndx(fs, lu_objid, pghdr->spndx, + &ref_pgndx, &objndx_pgndx); + if (ret == -ENOENT) + { + /* No object with this ID, so remove page safely */ + + ret = OK; + delete_page = true; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_phys_cpy() failed: %d\n", ret); + return ret; + } + else if (ref_pgndx != cur_pgndx) + { + spiffs_checkinfo("Other finalized page is referred, just delete\n"); + delete_page = true; + } + else + { + uint8_t flags = 0xff; + + /* page referenced by object index but not final + * just finalize + */ + + spiffs_checkinfo("Unfinalized page is referred, finalizing\n"); + +#ifdef CONFIG_SPIFFS_NO_BLIND_WRITES + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx) + + offsetof(struct spiffs_page_header_s, flags), + sizeof(flags), &flags); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } +#endif + + flags &= ~SPIFFS_PH_FLAG_FINAL; + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + 0, SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx) + + offsetof(struct spiffs_page_header_s, flags), + sizeof(flags), &flags); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_write() failed: %d\n", ret); + return ret; + } + } + } + } + + if (delete_page) + { + spiffs_checkinfo("LU: FIXUP: deleting page %04x\n", cur_pgndx); + ret = spiffs_page_delete(fs, cur_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret); + return ret; + } + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_lucheck_callback + * + * Description: + * This is a callback from spiffs_foreach_objlu(). It is part of the + * logic of spiffs_check_luconsistency(). It checks the page page + * header for each entry for validity. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * objid - Object ID + * cur_block - Current block + * cur_entry - Current entry + * user_const - User provided constant data + * user_var - User provided variable data + * + * Returned Value: + * Returns SPIFFS_VIS_COUNTINUE_RELOAD, SPIFFS_VIS_COUNTINUE, or a + * negated errno value in the event of a failure. + * + ****************************************************************************/ + +static int spiffs_lucheck_callback(FAR struct spiffs_s *fs, int16_t objid, + int16_t cur_block, int cur_entry, + FAR const void *user_const, + FAR void *user_var) +{ + struct spiffs_page_header_s pghdr; + int16_t cur_pgndx; + int ret = OK; + bool reload_lu = false; + + cur_pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, cur_block, cur_entry); + + /* Load header */ + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx), + sizeof(struct spiffs_page_header_s), + (FAR uint8_t *)&pghdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + ret = spiffs_check_luentry_validate(fs, objid, &pghdr, cur_pgndx, cur_block, + cur_entry, &reload_lu); + if (ret < 0) + { + ferr("ERROR: spiffs_check_luentry_validate() failed: %d\n", ret); + return ret; + } + + return reload_lu ? SPIFFS_VIS_COUNTINUE_RELOAD : SPIFFS_VIS_COUNTINUE; +} + +/**************************************************************************** + * Name: spifss_check_objndx_search + * + * Description: + * Searches for given object ID in temporary object ID index. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * objid - The Object ID + * + * Returned Value: + * The index associated with the objid is returned on sucess. -ENOENT + * is resutled if the objid was not found. + * + ****************************************************************************/ + +static int spifss_check_objndx_search(FAR struct spiffs_s *fs, int16_t objid) +{ + FAR int16_t *obj_table = (FAR int16_t *)fs->work; + int i; + + objid &= ~SPIFFS_OBJ_ID_IX_FLAG; + for (i = 0; i < SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(int16_t); i++) + { + if ((obj_table[i] & ~SPIFFS_OBJ_ID_IX_FLAG) == objid) + { + return i; + } + } + + return -ENOENT; +} + +/**************************************************************************** + * Name: spiffs_check_objidconsistency_callback + * + * Description: + * Check object index consistency. This is callback from + * spiffs_foreach_objlu() and logically a part of + * spiffs_check_objidconsistency() + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * objid - Object ID + * cur_block - Current block + * cur_entry - Current entry + * user_const - User provided constant data + * user_var - User provided variable data + * + * Returned Value: + * Returns SPIFFS_VIS_COUNTINUE_RELOAD, SPIFFS_VIS_COUNTINUE, or a + * negated errno value in the event of a failure. + * + ****************************************************************************/ + +static int spiffs_check_objidconsistency_callback(FAR struct spiffs_s *fs, + int16_t objid, + int16_t cur_block, + int cur_entry, + FAR const void *user_const, + FAR void *user_var) +{ + FAR uint32_t *log_ndx = (FAR uint32_t *)user_var; + FAR int16_t *obj_table = (FAR int16_t *)fs->work; + int retc = SPIFFS_VIS_COUNTINUE; + int ret = OK; + + if (objid != SPIFFS_OBJ_ID_FREE && objid != SPIFFS_OBJ_ID_DELETED && + (objid & SPIFFS_OBJ_ID_IX_FLAG) != 0) + { + struct spiffs_page_header_s pghdr; + int16_t cur_pgndx; + + cur_pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, cur_block, cur_entry); + + /* Load header */ + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx), + sizeof(struct spiffs_page_header_s), (uint8_t *) & pghdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + if (pghdr.spndx == 0 && + (pghdr.flags & (SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_FINAL | + SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_IXDELE)) == + (SPIFFS_PH_FLAG_DELET)) + { + spiffs_checkinfo("pgndx=%04x, objid=%04x spndx=%04x " + "header not fully deleted - deleting\n", + cur_pgndx, objid, pghdr.spndx); + + ret = spiffs_page_delete(fs, cur_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret); + return ret; + } + + return retc; + } + + if ((pghdr.flags & (SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_FINAL | + SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_IXDELE)) == + (SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_IXDELE)) + { + return retc; + } + + if (pghdr.spndx == 0) + { + int ret2; + + /* objndx header page, register objid as reachable */ + + ret2 = spifss_check_objndx_search(fs, objid); + if (ret2 < 0) + { + /* Not registered, do it */ + + obj_table[*log_ndx] = objid & ~SPIFFS_OBJ_ID_IX_FLAG; + (*log_ndx)++; + if (*log_ndx >= SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(int16_t)) + { + *log_ndx = 0; + } + } + } + else + { + bool delete = false; + int ret2; + + /* Span index + * objndx page, see if header can be found + */ + + ret2 = spifss_check_objndx_search(fs, objid); + if (ret2 < 0) + { + int16_t objhdr_pgndx; + + /* Not in temporary index, try finding it */ + + ret = spiffs_obj_lu_find_id_and_span(fs, objid | SPIFFS_OBJ_ID_IX_FLAG, + 0, 0, &objhdr_pgndx); + retc = SPIFFS_VIS_COUNTINUE_RELOAD; + + if (ret == OK) + { + /* Found, register as reachable */ + + obj_table[*log_ndx] = objid & ~SPIFFS_OBJ_ID_IX_FLAG; + } + else if (ret == -ENOENT) + { + /* Not found, register as unreachable */ + + delete = true; + obj_table[*log_ndx] = objid | SPIFFS_OBJ_ID_IX_FLAG; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + + (*log_ndx)++; + if (*log_ndx >= SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(int16_t)) + { + *log_ndx = 0; + } + } + else + { + /* In temporary index, check reachable flag */ + + if ((obj_table[ret2] & SPIFFS_OBJ_ID_IX_FLAG)) + { + /* Registered as unreachable */ + + delete = true; + } + } + + if (delete) + { + spiffs_checkinfo("pgndx=%04x objid=%04x spndx:%04x" + " is orphan index - deleting\n", + cur_pgndx, objid, pghdr.spndx); + + ret = spiffs_page_delete(fs, cur_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret); + return ret; + } + } + } + } + + return retc; +} + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_check_luconsistency + * + * Description: + * Scans all object look up. For each entry, corresponding page header is + * checked for validity. If an object index header page is found, this is + * also checked + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_check_luconsistency(FAR struct spiffs_s *fs) +{ + int ret = OK; + + ret = spiffs_foreach_objlu(fs, 0, 0, 0, 0, spiffs_lucheck_callback, + 0, 0, 0, 0); + if (ret == SPIFFS_VIS_END) + { + ret = OK; + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_check_pgconsistency + * + * Description: + * Checks consistency amongst all pages and fixes irregularities + * Scans all pages (except lu pages), reserves 4 bits in working memory + * for each page + * + * bit 0: 0 == FREE|DELETED, 1 == USED + * bit 1: 0 == UNREFERENCED, 1 == REFERENCED + * bit 2: 0 == NOT_INDEX, 1 == INDEX + * bit 3: unused + * + * A consistent file system will have only pages being + * + * - x000 free, unreferenced, not index + * - x011 used, referenced only once, not index + * - x101 used, unreferenced, index + * + * The working memory might not fit all pages so several scans might be needed + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_check_pgconsistency(FAR struct spiffs_s *fs) +{ + const uint32_t bits = 4; + const int16_t pages_per_scan = SPIFFS_CFG_LOG_PAGE_SZ(fs) * 8 / bits; + int16_t pgndx_offset = 0; + int ret = OK; + + /* For each range of pages fitting into work memory */ + + while (pgndx_offset < SPIFFS_PAGES_PER_BLOCK(fs) * fs->geo.neraseblocks) + { + int16_t cur_block = 0; + bool restart = false; + + memset(fs->work, 0, SPIFFS_CFG_LOG_PAGE_SZ(fs)); + + /* Build consistency bitmap for ID range traversing all blocks */ + + while (!restart && cur_block < fs->geo.neraseblocks) + { + /* Traverse each page except for lookup pages */ + + int16_t cur_pgndx = SPIFFS_OBJ_LOOKUP_PAGES(fs) + + SPIFFS_PAGES_PER_BLOCK(fs) * cur_block; + + while (!restart && + cur_pgndx < SPIFFS_PAGES_PER_BLOCK(fs) * (cur_block + 1)) + { + struct spiffs_page_header_s pghdr; + uint32_t pgndx_bytendx; + uint8_t pgndx_bitndx; + bool within_range; + + /* read header */ + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx), + sizeof(struct spiffs_page_header_s), (uint8_t *) & pghdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + within_range = (cur_pgndx >= pgndx_offset && + cur_pgndx < pgndx_offset + pages_per_scan); + pgndx_bytendx = (cur_pgndx - pgndx_offset) / (8 / bits); + pgndx_bitndx = (cur_pgndx & ((8 / bits) - 1)) * bits; + + if (within_range && + (pghdr.flags & SPIFFS_PH_FLAG_DELET) && + (pghdr.flags & SPIFFS_PH_FLAG_USED) == 0) + { + /* Used */ + + fs->work[pgndx_bytendx] |= (1 << (pgndx_bitndx + 0)); + } + + if ((pghdr.flags & SPIFFS_PH_FLAG_DELET) && + (pghdr.flags & SPIFFS_PH_FLAG_IXDELE) && + (pghdr.flags & (SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_USED)) == 0) + { + FAR struct spiffs_page_header_s *objndx_phdr; + FAR int16_t *object_page_index; + int16_t data_spndx_offset; + int entries; + int i; + + /* Found non-deleted index */ + + if (within_range) + { + fs->work[pgndx_bytendx] |= (1 << (pgndx_bitndx + 2)); + } + + /* Load non-deleted index */ + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + /* traverse index for referenced pages */ + + objndx_phdr = (FAR struct spiffs_page_header_s *)fs->lu_work; + + if (pghdr.spndx == 0) + { + /* object header page index */ + + entries = SPIFFS_OBJ_HDR_IX_LEN(fs); + data_spndx_offset = 0; + object_page_index = + (FAR int16_t *)((FAR uint8_t *)fs->lu_work + + sizeof(struct spiffs_pgobj_ixheader_s)); + } + else + { + /* Object page index */ + + entries = SPIFFS_OBJ_IX_LEN(fs); + data_spndx_offset = SPIFFS_OBJ_HDR_IX_LEN(fs) + + SPIFFS_OBJ_IX_LEN(fs) * (pghdr.spndx - 1); + object_page_index = + (FAR int16_t *)((FAR uint8_t *) fs->lu_work + + sizeof(spiffs_page_object_ix)); + } + + /* For all entries in index */ + + for (i = 0; !restart && i < entries; i++) + { + int16_t rpgndx = object_page_index[i]; + bool rpgndx_within_range; + + rpgndx_within_range = (rpgndx >= pgndx_offset && + rpgndx < pgndx_offset + pages_per_scan); + + if ((rpgndx != (int16_t) - 1 && + rpgndx > SPIFFS_MAX_PAGES(fs)) || + (rpgndx_within_range && SPIFFS_IS_LOOKUP_PAGE(fs, rpgndx))) + { + int16_t data_pgndx; + + /* Bad reference */ + + spiffs_checkinfo("pgndx=%04x bad pgndx / LU referenced from page %04x\n", + rpgndx, cur_pgndx); + + /* Check for data page elsewhere */ + + ret = spiffs_obj_lu_find_id_and_span(fs, + objndx_phdr->objid & ~SPIFFS_OBJ_ID_IX_FLAG, + data_spndx_offset + i, + 0, &data_pgndx); + if (ret == -ENOENT) + { + ret = OK; + data_pgndx = 0; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", + ret); + return ret; + } + + if (data_pgndx == 0) + { + struct spiffs_page_header_s new_ph; + + /* If not, allocate free page */ + + new_ph.flags = 0xff & ~(SPIFFS_PH_FLAG_USED | + SPIFFS_PH_FLAG_FINAL); + new_ph.objid = objndx_phdr->objid & ~SPIFFS_OBJ_ID_IX_FLAG; + new_ph.spndx = data_spndx_offset + i; + + ret = spiffs_page_allocate_data(fs, new_ph.objid, + &new_ph, 0, 0, 0, 1, + &data_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_allocate_data() failed: %d\n", ret); + return ret; + } + + spiffs_checkinfo("Found no existing data page, created new @ %04x\n", + data_pgndx); + } + + /* Remap index */ + + spiffs_checkinfo("Rewriting index pgndx=%04x\n", cur_pgndx); + + ret = spiffs_check_rewrite_index(fs, + objndx_phdr->objid | SPIFFS_OBJ_ID_IX_FLAG, + data_spndx_offset + i, + data_pgndx, cur_pgndx); + if (ret == -EFAULT) + { + /* Index bad also, cannot mend this file */ + + spiffs_checkinfo("Index bad %d, cannot mend - delete object\n", + ret); + + /* Delete file */ + + ret = spiffs_page_delete(fs, cur_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret); + return ret; + } + } + else if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_index() failed: %d\n", ret); + return ret; + } + + restart = true; + } + else if (rpgndx_within_range) + { + /* Valid reference. read referenced page header */ + + struct spiffs_page_header_s rp_hdr; + ret = spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, rpgndx), + sizeof(struct spiffs_page_header_s), + (FAR uint8_t *)&rp_hdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + /* Cross reference page header check */ + + if (rp_hdr.objid != (pghdr.objid & ~SPIFFS_OBJ_ID_IX_FLAG) || + rp_hdr.spndx != data_spndx_offset + i || + (rp_hdr.flags & (SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_INDEX | + SPIFFS_PH_FLAG_USED)) != + (SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_INDEX)) + { + int16_t data_pgndx; + + spiffs_checkinfo("pgndx=%04x has inconsistent page header index objid/span:" + "%04x/%04x, ref objid/span:%04x/%04x flags=%02x\n", + rpgndx, + pghdr.objid & ~SPIFFS_OBJ_ID_IX_FLAG, + data_spndx_offset + i, + rp_hdr.objid, rp_hdr.spndx, + rp_hdr.flags); + + /* Try finding correct page */ + + ret = spiffs_obj_lu_find_id_and_span(fs, + pghdr.objid & ~SPIFFS_OBJ_ID_IX_FLAG, + data_spndx_offset + i, rpgndx, + &data_pgndx); + if (ret == -ENOENT) + { + ret = OK; + data_pgndx = 0; + } + else if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_find_id_and_span() failed: %d\n", ret); + return ret; + } + + if (data_pgndx == 0) + { + /* Not found, this index is badly borked */ + + spiffs_checkinfo("Index bad, delete object objid %04x\n", + pghdr.objid); + + ret = spiffs_check_delobj_lazy(fs, pghdr.objid); + if (ret < 0) + { + ferr("ERROR: spiffs_check_delobj_lazy() failed: %d\n", + ret); + return ret; + } + + break; + } + else + { + /* Found it, so rewrite index */ + + spiffs_checkinfo("Found correct data pgndx=%04x, " + "rewrite index pgndx=%04x objid=%04x\n", + data_pgndx, cur_pgndx, pghdr.objid); + + ret = spiffs_check_rewrite_index(fs, pghdr.objid, + data_spndx_offset + i, + data_pgndx, cur_pgndx); + if (ret == -EFAULT) + { + /* Index bad also, cannot mend this file */ + + spiffs_checkinfo("Index bad %d, cannot mend!\n", + ret); + + ret = spiffs_check_delobj_lazy(fs, pghdr.objid); + } + else if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_index() failed: %d\n", ret); + return ret; + } + + restart = true; + } + } + else + { + /* Mark rpgndx as referenced */ + + const uint32_t rpgndx_byte_ix = + (rpgndx - pgndx_offset) / (8 / bits); + const uint8_t rpgndx_bit_ix = + (rpgndx & ((8 / bits) - 1)) * bits; + + if ((fs->work[rpgndx_byte_ix] & (1 << (rpgndx_bit_ix + 1))) != 0) + { + spiffs_checkinfo("pgndx=%04x multiple referenced from page %04x\n", + rpgndx, cur_pgndx); + + /* Here, we should have fixed all broken + * references - getting this means there + * must be multiple files with same object + * ID. Only solution is to delete + * the object which is referring to this + * page + */ + + spiffs_checkinfo("Removing objid=%04x and page=%04x\n", + pghdr.objid, cur_pgndx); + + ret = spiffs_check_delobj_lazy(fs, pghdr.objid); + if (ret < 0) + { + ferr("ERROR: spiffs_check_delobj_lazy() failed: %d\n", ret); + return ret; + } + + /* Dxtra precaution, delete this page also */ + + ret = spiffs_page_delete(fs, cur_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret); + return ret; + } + + restart = true; + } + + fs->work[rpgndx_byte_ix] |= (1 << (rpgndx_bit_ix + 1)); + } + } + } + } + + /* Next page */ + + cur_pgndx++; + } + + /* Next block */ + + cur_block++; + } + + /* Check consistency bitmap */ + + if (!restart) + { + uint32_t byte_ndx; + int16_t objndx_pgndx; + int16_t rpgndx; + uint8_t bit_ndx; + + for (byte_ndx = 0; + !restart && byte_ndx < SPIFFS_CFG_LOG_PAGE_SZ(fs); + byte_ndx++) + { + for (bit_ndx = 0; !restart && bit_ndx < 8 / bits; bit_ndx++) + { + uint8_t bitmask; + int16_t cur_pgndx; + + bitmask = (fs->work[byte_ndx] >> (bit_ndx * bits)) & 0x7; + cur_pgndx = pgndx_offset + byte_ndx * (8 / bits) + bit_ndx; + + /* 000 ok - free, unreferenced, not index */ + + if (bitmask == 0x1) + { + struct spiffs_page_header_s pghdr; + bool rewrite_ndx_to_this = false; + bool delete_page = false; + + /* 001 */ + + spiffs_checkinfo("pgndx=%04x USED, UNREFERENCED, not index\n", + cur_pgndx); + + /* Check corresponding object index entry */ + + ret = spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, + SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx), + sizeof(struct spiffs_page_header_s), + (FAR uint8_t *)&pghdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + ret = spiffs_check_get_data_pgndx(fs, pghdr.objid, + pghdr.spndx, &rpgndx, + &objndx_pgndx); + if (ret == OK) + { + if (((rpgndx == (int16_t) - 1 || + rpgndx > SPIFFS_MAX_PAGES(fs)) || + (SPIFFS_IS_LOOKUP_PAGE(fs, rpgndx)))) + { + /* Pointing to a bad page altogether, rewrite + * index to this + */ + + rewrite_ndx_to_this = true; + + spiffs_checkinfo("Corresponding ref is bad: " + "%04x, rewrite to this %04x\n", + rpgndx, cur_pgndx); + } + else + { + struct spiffs_page_header_s rp_hdr; + + /* Pointing to something else, check what */ + + ret = spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, + SPIFFS_PAGE_TO_PADDR(fs, rpgndx), + sizeof(struct spiffs_page_header_s), + (FAR uint8_t *)&rp_hdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + if (((pghdr.objid & ~SPIFFS_OBJ_ID_IX_FLAG) == + rp_hdr.objid) && + ((rp_hdr.flags & (SPIFFS_PH_FLAG_INDEX | + SPIFFS_PH_FLAG_DELET | + SPIFFS_PH_FLAG_USED | + SPIFFS_PH_FLAG_FINAL)) == + (SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_DELET))) + { + /* Pointing to something else valid, just + * delete this page then + */ + + spiffs_checkinfo("Corresponding ref is good but different: " + "%04x, delete this %04x\n", + rpgndx, cur_pgndx); + + delete_page = true; + } + + /* Pointing to something weird, update index + * to point to this page instead + */ + + else if (rpgndx != cur_pgndx) + { + spiffs_checkinfo + ("PA: corresponding ref is weird: " + "%04x %s%s%s%s, rewrite this " + "%04x\n", rpgndx, + (rp_hdr.flags & SPIFFS_PH_FLAG_INDEX) ? "" : + "INDEX ", + (rp_hdr.flags & SPIFFS_PH_FLAG_DELET) ? "" : + "DELETED ", + (rp_hdr.flags & SPIFFS_PH_FLAG_USED) ? + "NOTUSED " : "", + (rp_hdr.flags & SPIFFS_PH_FLAG_FINAL) ? + "NOTFINAL " : "", cur_pgndx); + + rewrite_ndx_to_this = true; + } + else + { + /* Should not happen, destined for fubar */ + } + } + } + else if (ret == -ENOENT) + { + spiffs_checkinfo("Corresponding ref not found, delete %04x\n", + cur_pgndx); + + delete_page = true; + ret = OK; + } + + if (rewrite_ndx_to_this) + { + /* If pointing to invalid page, redirect index to + * this page + */ + + spiffs_checkinfo("Rewrite index objid=%04x data spndx=%04x" + " to point to this pgndx: %04x\n", + pghdr.objid, pghdr.spndx, cur_pgndx); + + ret = spiffs_check_rewrite_index(fs, pghdr.objid, + pghdr.spndx, cur_pgndx, + objndx_pgndx); + if (ret == -EFAULT) + { + int ret2; + + /* Index bad also, cannot mend this file */ + + spiffs_checkinfo("PA: FIXUP: index bad %d" + ", cannot mend!\n", ret); + + ret2 = spiffs_page_delete(fs, cur_pgndx); + if (ret2 < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", + ret2); + return ret2; + } + + ret2 = spiffs_check_delobj_lazy(fs, pghdr.objid); + if (ret2 < 0) + { + ferr("ERROR: spiffs_check_delobj_lazy() failed: %d\n", ret2); + return ret2; + } + } + else if (ret < 0) + { + ferr("ERROR: spiffs_check_rewrite_index() failed: %d\n", + ret); + return ret; + } + + restart = true; + continue; + } + else if (delete_page) + { + spiffs_checkinfo("Deleting page %04x\n", + cur_pgndx); + + ret = spiffs_page_delete(fs, cur_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret); + return ret; + } + } + } + + if (bitmask == 0x2) + { + /* 010 */ + + spiffs_checkinfo("pgndx=%04x FREE, REFERENCED, not index\n", + cur_pgndx); + + /* No op, this should be taken care of when checking + * valid references + */ + } + + /* 011 OK - busy, referenced, not index */ + + if (bitmask == 0x4) + { + /* 100 */ + + spiffs_checkinfo("pgndx=%04x FREE, unreferenced, INDEX\n", + cur_pgndx); + + /* This should never happen, major fubar */ + } + + /* 101 OK - busy, unreferenced, index */ + + if (bitmask == 0x6) + { + /* 110 */ + + spiffs_checkinfo("pgndx=%04x FREE, REFERENCED, INDEX\n", + cur_pgndx); + + /* No op, this should be taken care of when checking + * valid references + */ + } + + if (bitmask == 0x7) + { + /* 111 */ + + spiffs_checkinfo("pgndx=%04x USED, REFERENCED, INDEX\n", + cur_pgndx); + + /* No op, this should be taken care of when checking + * valid references + */ + } + } + } + } + + spiffs_checkinfo("Processed %04x, restart %d\n", + pgndx_offset, restart); + + /* next page range */ + + if (!restart) + { + pgndx_offset += pages_per_scan; + } + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_check_pgconsistency + * + * Description: + * Removes orphaned and partially deleted index pages. + * Scans for index pages. When an index page is found, corresponding index + * header is searched for. If no such page exists, the index page cannot + * be reached as no index header exists and must be deleted. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_check_objidconsistency(FAR struct spiffs_s *fs) +{ + uint32_t objid_logndx = 0; + int ret = OK; + + /* Implementation not: + * fs->work is used for a temporary object index memory, listing found + * object ids and indicating whether they can be reached or not. Acting + * as a FIFO if object ids cannot fit. In the temporary object index + * memory, SPIFFS_OBJ_ID_IX_FLAG bit is used to indicate a reachable/ + * unreachable object ID. + */ + + memset(fs->work, 0, SPIFFS_CFG_LOG_PAGE_SZ(fs)); + + ret = spiffs_foreach_objlu(fs, 0, 0, 0, 0, + spiffs_check_objidconsistency_callback, 0, + &objid_logndx, 0, 0); + if (ret == SPIFFS_VIS_END) + { + ret = OK; + } + + return ret; +} diff --git a/fs/spiffs/src/spiffs_check.h b/fs/spiffs/src/spiffs_check.h new file mode 100644 index 00000000000..403348b491d --- /dev/null +++ b/fs/spiffs/src/spiffs_check.h @@ -0,0 +1,154 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_check.h + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +#ifndef __FS_SPIFFS_SRC_SPIFFS_CHECK_H +#define __FS_SPIFFS_SRC_SPIFFS_CHECK_H + +#if defined(__cplusplus) +extern "C" +{ +#endif + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include + +/**************************************************************************** + * Pre-processor Definitions + ****************************************************************************/ + +/* Consistency check output */ + +#ifdef CONFIG_SPIFFS_CHECK_OUTPUT +# ifdef CONFIG_CPP_HAVE_VARARGS +# define spiffs_checkinfo(format, ...) \ + syslog(LOG_NOTICE, "SPIFFS: " format, ##__VA_ARGS__) +# else +# define spiffs_checkinfo _info +# endif +#else +# ifdef CONFIG_CPP_HAVE_VARARGS +# define spiffs_checkinfo(format, ...) +# else +# define spiffs_checkinfo (void) +# endif +#endif + +/**************************************************************************** + * Public Function Prototypes + ****************************************************************************/ + +struct spiffs_s; /* Forward reference */ + +/**************************************************************************** + * Name: spiffs_check_luconsistency + * + * Description: + * Scans all object look up. For each entry, corresponding page header is + * checked for validity. If an object index header page is found, this is + * also checked + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_check_luconsistency(FAR struct spiffs_s *fs); + +/**************************************************************************** + * Name: spiffs_check_pgconsistency + * + * Description: + * Checks consistency amongst all pages and fixes irregularities + * Scans all pages (except lu pages), reserves 4 bits in working memory + * for each page + * + * bit 0: 0 == FREE|DELETED, 1 == USED + * bit 1: 0 == UNREFERENCED, 1 == REFERENCED + * bit 2: 0 == NOT_INDEX, 1 == INDEX + * bit 3: unused + * + * A consistent file system will have only pages being + * + * - x000 free, unreferenced, not index + * - x011 used, referenced only once, not index + * - x101 used, unreferenced, index + * + * The working memory might not fit all pages so several scans might be needed + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_check_pgconsistency(FAR struct spiffs_s *fs); + +/**************************************************************************** + * Name: spiffs_check_pgconsistency + * + * Description: + * Removes orphaned and partially deleted index pages. + * Scans for index pages. When an index page is found, corresponding index + * header is searched for. If no such page exists, the index page cannot + * be reached as no index header exists and must be deleted. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_check_objidconsistency(FAR struct spiffs_s *fs); + +#if defined(__cplusplus) +} +#endif + +#endif /* __FS_SPIFFS_SRC_SPIFFS_CHECK_H */ diff --git a/fs/spiffs/src/spiffs_core.c b/fs/spiffs/src/spiffs_core.c new file mode 100644 index 00000000000..0c68e59336c --- /dev/null +++ b/fs/spiffs/src/spiffs_core.c @@ -0,0 +1,2989 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_core.c + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * This is a port of version 0.3.7 of SPIFFS by Peter Andersion. That + * version was originally released under the MIT license but is here re- + * released under the NuttX BSD license. + * + * Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976@gmail.com) + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include + +#include + +#include "spiffs.h" +#include "spiffs_mtd.h" +#include "spiffs_gc.h" +#include "spiffs_cache.h" +#include "spiffs_core.h" + +/**************************************************************************** + * Private Types + ****************************************************************************/ + +struct spiffs_free_objid_s +{ + int16_t min_objid; + int16_t max_objid; + uint32_t compaction; + FAR const uint8_t *conflicting_name; +}; + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_page_data_check + * + * Description: + * + ****************************************************************************/ + +static int spiffs_page_data_check(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, + int16_t pgndx, int16_t spndx) +{ + int ret = OK; + + if (pgndx == (int16_t) - 1) + { + /* referring to page 0xffff...., bad object index */ + + return SPIFFS_ERR_INDEX_REF_FREE; + } + + if (pgndx % SPIFFS_PAGES_PER_BLOCK(fs) < SPIFFS_OBJ_LOOKUP_PAGES(fs)) + { + /* referring to an object lookup page, bad object index */ + + return SPIFFS_ERR_INDEX_REF_LU; + } + + if (pgndx > SPIFFS_MAX_PAGES(fs)) + { + /* referring to a bad page */ + + return SPIFFS_ERR_INDEX_REF_INVALID; + } + +#ifdef CONFIG_SPIFFS_PAGE_CHECK + struct spiffs_page_header_s ph; + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, pgndx), + sizeof(struct spiffs_page_header_s), (uint8_t *) & ph); + SPIFFS_CHECK_RES(ret); + SPIFFS_VALIDATE_DATA(ph, fobj->objid & ~SPIFFS_OBJ_ID_IX_FLAG, spndx); +#endif + return ret; +} + +/**************************************************************************** + * Name: spiffs_page_index_check + * + * Description: + * + ****************************************************************************/ + +static int spiffs_page_index_check(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, + int16_t pgndx, int16_t spndx) +{ + int ret = OK; + + if (pgndx == (int16_t) - 1) + { + /* referring to page 0xffff...., bad object index */ + + return SPIFFS_ERR_INDEX_FREE; + } + + if (pgndx % SPIFFS_PAGES_PER_BLOCK(fs) < SPIFFS_OBJ_LOOKUP_PAGES(fs)) + { + /* referring to an object lookup page, bad object index */ + + return SPIFFS_ERR_INDEX_LU; + } + + if (pgndx > SPIFFS_MAX_PAGES(fs)) + { + /* referring to a bad page */ + + return SPIFFS_ERR_INDEX_INVALID; + } + +#ifdef CONFIG_SPIFFS_PAGE_CHECK + struct spiffs_page_header_s ph; + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, pgndx), + sizeof(struct spiffs_page_header_s), (uint8_t *) & ph); + SPIFFS_CHECK_RES(ret); + SPIFFS_VALIDATE_OBJIX(ph, fobj->objid, spndx); +#endif + + return ret; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_scan_v + * + * Description: + * + ****************************************************************************/ + +static int spiffs_obj_lu_scan_v(FAR struct spiffs_s *fs, + int16_t objid, + int16_t blkndx, + int ix_entry, + FAR const void *user_const, + FAR void *user_var) +{ + if (objid == SPIFFS_OBJ_ID_FREE) + { + if (ix_entry == 0) + { + /* todo optimize further, return SPIFFS_NEXT_BLOCK */ + + fs->free_blocks++; + } + } + else if (objid == SPIFFS_OBJ_ID_DELETED) + { + fs->stats_p_deleted++; + } + else + { + fs->stats_p_allocated++; + } + + return SPIFFS_VIS_COUNTINUE; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_find_id_and_span_v + * + * Description: + * + ****************************************************************************/ + +static int spiffs_obj_lu_find_id_and_span_v(FAR struct spiffs_s *fs, + int16_t objid, + int16_t blkndx, + int ix_entry, + FAR const void *user_const, + FAR void *user_var) +{ + struct spiffs_page_header_s ph; + int16_t pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, ix_entry); + int ret; + + ret = spiffs_cache_read(fs, 0, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + SPIFFS_PAGE_TO_PADDR(fs, pgndx), sizeof(struct spiffs_page_header_s), + (uint8_t *) & ph); + + SPIFFS_CHECK_RES(ret); + + if (ph.objid == objid && + ph.spndx == *((int16_t *) user_var) && + (ph. + flags & (SPIFFS_PH_FLAG_FINAL | SPIFFS_PH_FLAG_DELET | + SPIFFS_PH_FLAG_USED)) == SPIFFS_PH_FLAG_DELET && + !((objid & SPIFFS_OBJ_ID_IX_FLAG) && + (ph.flags & SPIFFS_PH_FLAG_IXDELE) == 0 && ph.spndx == 0) && + (user_const == 0 || *((const int16_t *)user_const) != pgndx)) + { + return OK; + } + else + { + return SPIFFS_VIS_COUNTINUE; + } +} + +/**************************************************************************** + * Name: spiffs_find_objhdr_pgndx_callback + * + * Description: + * + ****************************************************************************/ + +static int spiffs_find_objhdr_pgndx_callback(FAR struct spiffs_s *fs, int16_t objid, + int16_t blkndx, int ix_entry, + FAR const void *user_const, + FAR void *user_var) +{ + struct spiffs_pgobj_ixheader_s objhdr; + int16_t pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, ix_entry); + int ret; + + if (objid == SPIFFS_OBJ_ID_FREE || objid == SPIFFS_OBJ_ID_DELETED || + (objid & SPIFFS_OBJ_ID_IX_FLAG) == 0) + { + return SPIFFS_VIS_COUNTINUE; + } + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, pgndx), + sizeof(struct spiffs_pgobj_ixheader_s), (uint8_t *) & objhdr); + SPIFFS_CHECK_RES(ret); + if (objhdr.p_hdr.spndx == 0 && + (objhdr.p_hdr. + flags & (SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_FINAL | + SPIFFS_PH_FLAG_IXDELE)) == + (SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_IXDELE)) + { + if (strcmp((const char *)user_const, (char *)objhdr.name) == 0) + { + return OK; + } + } + + return SPIFFS_VIS_COUNTINUE; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_find_free_objid_bitmap_v + * + * Description: + * + ****************************************************************************/ + +static int spiffs_obj_lu_find_free_objid_bitmap_v(FAR struct spiffs_s *fs, + int16_t objid, + int16_t blkndx, + int ix_entry, + FAR const void *user_const, + FAR void *user_var) +{ + if (objid != SPIFFS_OBJ_ID_FREE && objid != SPIFFS_OBJ_ID_DELETED) + { + int16_t min_objid = *((int16_t *) user_var); + const uint8_t *conflicting_name = (const uint8_t *)user_const; + + /* if conflicting name parameter is given, also check if this name is + * found in object index hdrs + */ + + if (conflicting_name && (objid & SPIFFS_OBJ_ID_IX_FLAG)) + { + int16_t pgndx = + SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, ix_entry); + int ret; + struct spiffs_pgobj_ixheader_s objhdr; + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, pgndx), + sizeof(struct spiffs_pgobj_ixheader_s), + (uint8_t *) & objhdr); + SPIFFS_CHECK_RES(ret); + if (objhdr.p_hdr.spndx == 0 && + (objhdr.p_hdr. + flags & (SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_FINAL | + SPIFFS_PH_FLAG_IXDELE)) == + (SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_IXDELE)) + { + if (strcmp((const char *)user_const, (char *)objhdr.name) == + 0) + { + return -EEXIST; + } + } + } + + objid &= ~SPIFFS_OBJ_ID_IX_FLAG; + uint32_t bit_ix = (objid - min_objid) & 7; + int byte_ix = (objid - min_objid) >> 3; + if (byte_ix >= 0 && (uint32_t) byte_ix < SPIFFS_CFG_LOG_PAGE_SZ(fs)) + { + fs->work[byte_ix] |= (1 << bit_ix); + } + } + + return SPIFFS_VIS_COUNTINUE; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_find_free_objid_compact_v + * + * Description: + * + ****************************************************************************/ + +static int spiffs_obj_lu_find_free_objid_compact_v(FAR struct spiffs_s *fs, + int16_t objid, + int16_t blkndx, + int ix_entry, + FAR const void *user_const, + FAR void *user_var) +{ + if (objid != SPIFFS_OBJ_ID_FREE && objid != SPIFFS_OBJ_ID_DELETED && + (objid & SPIFFS_OBJ_ID_IX_FLAG)) + { + int ret; + FAR const struct spiffs_free_objid_s *state = + (FAR const struct spiffs_free_objid_s *)user_const; + struct spiffs_pgobj_ixheader_s objhdr; + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_OBJ_LOOKUP_ENTRY_TO_PADDR(fs, blkndx, ix_entry), + sizeof(struct spiffs_pgobj_ixheader_s), + (uint8_t *) & objhdr); + if (ret == OK && objhdr.p_hdr.spndx == 0 && + ((objhdr.p_hdr. + flags & (SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_FINAL | + SPIFFS_PH_FLAG_DELET)) == (SPIFFS_PH_FLAG_DELET))) + { + /* ok object look up entry */ + + if (state->conflicting_name && + strcmp((const char *)state->conflicting_name, (char *)objhdr.name) == 0) + { + return -EEXIST; + } + + objid &= ~SPIFFS_OBJ_ID_IX_FLAG; + if (objid >= state->min_objid && objid <= state->max_objid) + { + uint8_t *map = (uint8_t *) fs->work; + int ndx = (objid - state->min_objid) / state->compaction; + map[ndx]++; + } + } + } + + return SPIFFS_VIS_COUNTINUE; +} + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_phys_cpy + * + * Description: + * + ****************************************************************************/ + +int spiffs_phys_cpy(FAR struct spiffs_s *fs, + int16_t objid, uint32_t dst, uint32_t src, uint32_t len) +{ + uint8_t b[CONFIG_SPIFFS_COPYBUF_STACK]; + int ret; + + while (len > 0) + { + uint32_t chunk_size = MIN(CONFIG_SPIFFS_COPYBUF_STACK, len); + ret = + spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_MOVS, objid, src, + chunk_size, b); + SPIFFS_CHECK_RES(ret); + ret = + spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_MOVD, objid, dst, + chunk_size, b); + SPIFFS_CHECK_RES(ret); + len -= chunk_size; + src += chunk_size; + dst += chunk_size; + } + + return OK; +} + +/**************************************************************************** + * Name: spiffs_foreach_objlu + * + * Description: + * Find object lookup entry containing given objid with visitor. + * Iterate over object lookup pages in each block until a given object + * objid entry is found. + * + * When found, the visitor function is called with block index, entry + * index and user data. + * + * If visitor returns SPIFFS_VIS_CONTINUE, the search goes on. Otherwise, + * the search will be ended and visitor's return code is returned to + * caller. + * + * If no visitor is given (0) the search returns on first entry with + * matching object objid. + * + * If no match is found in all look up, SPIFFS_VIS_END is returned. + * + * Input Parameters: + * fs - The file system + * starting_block - The starting block to start search in + * starting_lu_entry - The look up index entry to start search in + * flags - ORed combination of SPIFFS_VIS_CHECK_ID, + * SPIFFS_VIS_CHECK_PH, SPIFFS_VIS_NO_WRAP + * objid - Argument object ID + * v - Visitor callback function + * user_const - Any const pointer, passed to the callback visitor function + * user_var - Any pointer, passed to the callback visitor function + * block_ix - Reported block index where match was found + * lu_entry - Reported look up index where match was found + * + ****************************************************************************/ + +int spiffs_foreach_objlu(FAR struct spiffs_s *fs, int16_t starting_block, + int starting_lu_entry, uint8_t flags, int16_t objid, + spiffs_visitor_f v, FAR const void *user_const, + FAR void *user_var, FAR int16_t *block_ix, + FAR int *lu_entry) +{ + int32_t entry_count = fs->geo.neraseblocks * SPIFFS_OBJ_LOOKUP_MAX_ENTRIES(fs); + int16_t cur_block = starting_block; + uint32_t cur_block_addr = starting_block * SPIFFS_CFG_LOG_BLOCK_SZ(fs); + int16_t *obj_lu_buf = (int16_t *) fs->lu_work; + int cur_entry = starting_lu_entry; + int entries_per_page = (SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(int16_t)); + int ret = OK; + + /* wrap initial */ + + if (cur_entry > (int)SPIFFS_OBJ_LOOKUP_MAX_ENTRIES(fs) - 1) + { + cur_entry = 0; + cur_block++; + cur_block_addr = cur_block * SPIFFS_CFG_LOG_BLOCK_SZ(fs); + if (cur_block >= fs->geo.neraseblocks) + { + if (flags & SPIFFS_VIS_NO_WRAP) + { + return SPIFFS_VIS_END; + } + else + { + /* block wrap */ + + cur_block = 0; + cur_block_addr = 0; + } + } + } + + /* check each block */ + + while (ret == OK && entry_count > 0) + { + int obj_lookup_page = cur_entry / entries_per_page; + + /* check each object lookup page */ + + while (ret == OK && + obj_lookup_page < (int)SPIFFS_OBJ_LOOKUP_PAGES(fs)) + { + int entry_offset = obj_lookup_page * entries_per_page; + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, + 0, cur_block_addr + SPIFFS_PAGE_TO_PADDR(fs, + obj_lookup_page), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work); + + /* check each entry */ + + while (ret == OK && cur_entry - entry_offset < entries_per_page && /* for + * non-last + * obj + * lookup + * pages */ + cur_entry < (int)SPIFFS_OBJ_LOOKUP_MAX_ENTRIES(fs)) /* for + * last + * obj + * lookup + * page */ + { + if ((flags & SPIFFS_VIS_CHECK_ID) == 0 || + obj_lu_buf[cur_entry - entry_offset] == objid) + { + if (block_ix) + { + *block_ix = cur_block; + } + + if (lu_entry) + { + *lu_entry = cur_entry; + } + + if (v) + { + ret = v(fs, + (flags & SPIFFS_VIS_CHECK_PH) ? objid : + obj_lu_buf[cur_entry - entry_offset], cur_block, + cur_entry, user_const, user_var); + if (ret == SPIFFS_VIS_COUNTINUE || + ret == SPIFFS_VIS_COUNTINUE_RELOAD) + { + if (ret == SPIFFS_VIS_COUNTINUE_RELOAD) + { + ret = + spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU | + SPIFFS_OP_C_READ, 0, + cur_block_addr + + SPIFFS_PAGE_TO_PADDR(fs, + obj_lookup_page), + SPIFFS_CFG_LOG_PAGE_SZ(fs), + fs->lu_work); + SPIFFS_CHECK_RES(ret); + } + + ret = OK; + cur_entry++; + entry_count--; + continue; + } + else + { + return ret; + } + } + else + { + return OK; + } + } + + entry_count--; + cur_entry++; + } + + obj_lookup_page++; + } + + cur_entry = 0; + cur_block++; + cur_block_addr += SPIFFS_CFG_LOG_BLOCK_SZ(fs); + + if (cur_block >= fs->geo.neraseblocks) + { + if (flags & SPIFFS_VIS_NO_WRAP) + { + return SPIFFS_VIS_END; + } + else + { + /* block wrap */ + + cur_block = 0; + cur_block_addr = 0; + } + } + } + + SPIFFS_CHECK_RES(ret); + return SPIFFS_VIS_END; +} + +/**************************************************************************** + * Name: spiffs_erase_block + * + * Description: + * + ****************************************************************************/ + +int spiffs_erase_block(FAR struct spiffs_s *fs, int16_t blkndx) +{ + uint32_t addr = SPIFFS_BLOCK_TO_PADDR(fs, blkndx); + int32_t size = SPIFFS_CFG_LOG_BLOCK_SZ(fs); + int ret; + + /* here we ignore ret, just try erasing the block */ + + while (size > 0) + { + finfo("erase %08lx:%d\n", + (unsigned long)addr, SPIFFS_CFG_PHYS_ERASE_SZ(fs)); + + spiffs_mtd_erase(fs, addr, SPIFFS_CFG_PHYS_ERASE_SZ(fs)); + + addr += SPIFFS_CFG_PHYS_ERASE_SZ(fs); + size -= SPIFFS_CFG_PHYS_ERASE_SZ(fs); + } + + fs->free_blocks++; + + /* register erase count for this block */ + + ret = spiffs_cache_write(fs, SPIFFS_OP_C_WRTHRU | SPIFFS_OP_T_OBJ_LU2, 0, + SPIFFS_ERASE_COUNT_PADDR(fs, blkndx), + sizeof(int16_t), (uint8_t *) & fs->max_erase_count); + SPIFFS_CHECK_RES(ret); + + fs->max_erase_count++; + if (fs->max_erase_count == SPIFFS_OBJ_ID_IX_FLAG) + { + fs->max_erase_count = 0; + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_scan + * + * Description: + * Scans thru all obj lu and counts free, deleted and used pages. + * Find the maximum block erase count Checks magic if enabled + * + ****************************************************************************/ + +int spiffs_obj_lu_scan(FAR struct spiffs_s *fs) +{ + int16_t blkndx; + int entry; + int ret; + + /* find out erase count. if enabled, check magic */ + + blkndx = 0; + int16_t erase_count_final; + int16_t erase_count_min = SPIFFS_OBJ_ID_FREE; + int16_t erase_count_max = 0; + + while (blkndx < fs->geo.neraseblocks) + { + int16_t erase_count; + ret = spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_ERASE_COUNT_PADDR(fs, blkndx), + sizeof(int16_t), (uint8_t *) & erase_count); + SPIFFS_CHECK_RES(ret); + if (erase_count != SPIFFS_OBJ_ID_FREE) + { + erase_count_min = MIN(erase_count_min, erase_count); + erase_count_max = MAX(erase_count_max, erase_count); + } + + blkndx++; + } + + if (erase_count_min == 0 && erase_count_max == SPIFFS_OBJ_ID_FREE) + { + /* clean system, set counter to zero */ + + erase_count_final = 0; + } + else if (erase_count_max - erase_count_min > (SPIFFS_OBJ_ID_FREE) / 2) + { + /* wrap, take min */ + + erase_count_final = erase_count_min + 1; + } + else + { + erase_count_final = erase_count_max + 1; + } + + fs->max_erase_count = erase_count_final; + + /* count blocks */ + + fs->free_blocks = 0; + fs->stats_p_allocated = 0; + fs->stats_p_deleted = 0; + + ret = spiffs_foreach_objlu(fs, + 0, + 0, + 0, + 0, + spiffs_obj_lu_scan_v, + 0, 0, &blkndx, &entry); + + if (ret == SPIFFS_VIS_END) + { + ret = OK; + } + + SPIFFS_CHECK_RES(ret); + return ret; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_find_free + * + * Description: + * Find free object lookup entry. Iterate over object lookup pages in each + * block until a free object ID entry is found + * + ****************************************************************************/ + +int spiffs_obj_lu_find_free(FAR struct spiffs_s *fs, int16_t starting_block, + int starting_lu_entry, FAR int16_t *block_ix, + FAR int *lu_entry) +{ + int ret; + + if (fs->free_blocks < 2) + { + ret = spiffs_gc_quick(fs, 0); + + /* spiffs_gc_quick() will return -ENODATA if no blocks were freed */ + + if (ret == -ENODATA) + { + ret = OK; + } + else if (ret < 0) + { + return ret; + } + + if (fs->free_blocks < 2) + { + return -ENOSPC; + } + } + + ret = spiffs_obj_lu_find_id(fs, starting_block, starting_lu_entry, + SPIFFS_OBJ_ID_FREE, block_ix, lu_entry); + if (ret == OK) + { + fs->free_blkndx = *block_ix; + fs->free_entry = (*lu_entry) + 1; + if (*lu_entry == 0) + { + fs->free_blocks--; + } + } + + if (ret == -ENOSPC) + { + finfo("fs full\n"); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_find_id + * + * Description: + * Find object lookup entry containing given objid. Iterate over object + * lookup pages in each block until a given object objid entry is found + * + ****************************************************************************/ + +int spiffs_obj_lu_find_id(FAR struct spiffs_s *fs, int16_t starting_block, + int starting_lu_entry, int16_t objid, + FAR int16_t *block_ix, FAR int *lu_entry) +{ + int ret = + spiffs_foreach_objlu(fs, starting_block, starting_lu_entry, + SPIFFS_VIS_CHECK_ID, objid, 0, 0, 0, + block_ix, lu_entry); + if (ret == SPIFFS_VIS_END) + { + ret = -ENOENT; + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_find_id_and_span + * + * Description: + * Find object lookup entry containing given objid and span index. + * Iterate over object lookup pages in each block until a given object + * objid entry is found + * + ****************************************************************************/ + +int spiffs_obj_lu_find_id_and_span(FAR struct spiffs_s *fs, int16_t objid, + int16_t spndx, int16_t exclusion_pgndx, + FAR int16_t *pgndx) +{ + int16_t blkndx; + int entry; + int ret; + + ret = spiffs_foreach_objlu(fs, + fs->lu_blkndx, + fs->lu_entry, + SPIFFS_VIS_CHECK_ID, + objid, + spiffs_obj_lu_find_id_and_span_v, + exclusion_pgndx ? &exclusion_pgndx : 0, + &spndx, &blkndx, &entry); + + if (ret == SPIFFS_VIS_END) + { + ret = -ENOENT; + } + + SPIFFS_CHECK_RES(ret); + + if (pgndx) + { + *pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, entry); + } + + fs->lu_blkndx = blkndx; + fs->lu_entry = entry; + + return ret; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_find_id_and_span_byphdr + * + * Description: + * Find object lookup entry containing given objid and span index in page + * headers only. Iterate over object lookup pages in each block until a + * given object objid entry is found + * + ****************************************************************************/ + +int spiffs_obj_lu_find_id_and_span_byphdr(FAR struct spiffs_s *fs, + int16_t objid, int16_t spndx, + int16_t exclusion_pgndx, + FAR int16_t *pgndx) +{ + int16_t blkndx; + int entry; + int ret; + + ret = spiffs_foreach_objlu(fs, + fs->lu_blkndx, + fs->lu_entry, + SPIFFS_VIS_CHECK_PH, + objid, + spiffs_obj_lu_find_id_and_span_v, + exclusion_pgndx ? &exclusion_pgndx : 0, + &spndx, &blkndx, &entry); + + if (ret == SPIFFS_VIS_END) + { + ret = -ENOENT; + } + + if (ret < 0) + { + return ret; + } + + if (pgndx) + { + *pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, entry); + } + + fs->lu_blkndx = blkndx; + fs->lu_entry = entry; + return ret; +} + +/**************************************************************************** + * Name: spiffs_page_allocate_data + * + * Description: + * Allocates a free defined page with given objid. Occupies object + * lookup entry and page. Data may be NULL; where only page header is + * stored, len and page_offs is ignored + * + ****************************************************************************/ + +int spiffs_page_allocate_data(FAR struct spiffs_s *fs, int16_t objid, + FAR struct spiffs_page_header_s *ph, + FAR uint8_t *data, uint32_t len, + uint32_t page_offs, + bool finalize, FAR int16_t *pgndx) +{ + int16_t blkndx; + int entry; + int ret = OK; + + /* find free entry */ + + ret = + spiffs_obj_lu_find_free(fs, fs->free_blkndx, + fs->free_entry, &blkndx, &entry); + SPIFFS_CHECK_RES(ret); + + /* occupy page in object lookup */ + + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_UPDT, + 0, SPIFFS_BLOCK_TO_PADDR(fs, blkndx) + + entry * sizeof(int16_t), sizeof(int16_t), + (uint8_t *) & objid); + SPIFFS_CHECK_RES(ret); + + fs->stats_p_allocated++; + + /* write page header */ + + ph->flags &= ~SPIFFS_PH_FLAG_USED; + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + 0, SPIFFS_OBJ_LOOKUP_ENTRY_TO_PADDR(fs, blkndx, entry), + sizeof(struct spiffs_page_header_s), (uint8_t *) ph); + SPIFFS_CHECK_RES(ret); + + /* write page data */ + + if (data) + { + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + 0, SPIFFS_OBJ_LOOKUP_ENTRY_TO_PADDR(fs, blkndx, + entry) + + sizeof(struct spiffs_page_header_s) + page_offs, len, data); + SPIFFS_CHECK_RES(ret); + } + + /* finalize header if necessary */ + + if (finalize && (ph->flags & SPIFFS_PH_FLAG_FINAL)) + { + ph->flags &= ~SPIFFS_PH_FLAG_FINAL; + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + 0, SPIFFS_OBJ_LOOKUP_ENTRY_TO_PADDR(fs, blkndx, + entry) + + offsetof(struct spiffs_page_header_s, flags), sizeof(uint8_t), + (uint8_t *) & ph->flags); + SPIFFS_CHECK_RES(ret); + } + + /* return written page */ + + if (pgndx) + { + *pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, entry); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_page_move + * + * Description: + * Moves a page from src to a free page and finalizes it. Updates page + * index. Page data is given in param page. If page data is NULL, + * provided header is used for meta-info and page data is physically + * copied. + * + ****************************************************************************/ + +int spiffs_page_move(FAR struct spiffs_s *fs, + int16_t objid, FAR uint8_t *page_data, int16_t ndx, + FAR struct spiffs_page_header_s *page_hdr, + int16_t src_pgndx, FAR int16_t *dst_pgndx) +{ + uint8_t was_final = 0; + struct spiffs_page_header_s *p_hdr; + int16_t blkndx; + int entry; + int16_t free_pgndx; + int ret; + + /* find free entry */ + + ret = spiffs_obj_lu_find_free(fs, fs->free_blkndx, + fs->free_entry, &blkndx, &entry); + SPIFFS_CHECK_RES(ret); + free_pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, entry); + + if (dst_pgndx) + { + *dst_pgndx = free_pgndx; + } + + p_hdr = page_data ? (struct spiffs_page_header_s *) page_data : page_hdr; + if (page_data) + { + /* got page data */ + + was_final = (p_hdr->flags & SPIFFS_PH_FLAG_FINAL) == 0; + + /* write unfinalized page */ + + p_hdr->flags |= SPIFFS_PH_FLAG_FINAL; + p_hdr->flags &= ~SPIFFS_PH_FLAG_USED; + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + 0, SPIFFS_PAGE_TO_PADDR(fs, free_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), page_data); + } + else + { + /* copy page data */ + + ret = spiffs_phys_cpy(fs, objid, SPIFFS_PAGE_TO_PADDR(fs, free_pgndx), + SPIFFS_PAGE_TO_PADDR(fs, src_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs)); + } + + SPIFFS_CHECK_RES(ret); + + /* mark entry in destination object lookup */ + + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_UPDT, 0, + SPIFFS_BLOCK_TO_PADDR(fs, SPIFFS_BLOCK_FOR_PAGE(fs, free_pgndx)) + + SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(fs, free_pgndx) * + sizeof(int16_t), sizeof(int16_t), + (FAR uint8_t *)&ndx); + SPIFFS_CHECK_RES(ret); + + fs->stats_p_allocated++; + + if (was_final) + { + /* mark finalized in destination page */ + + p_hdr->flags &= ~(SPIFFS_PH_FLAG_FINAL | SPIFFS_PH_FLAG_USED); + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, objid, + SPIFFS_PAGE_TO_PADDR(fs, free_pgndx) + + offsetof(struct spiffs_page_header_s, flags), sizeof(uint8_t), + (uint8_t *) & p_hdr->flags); + SPIFFS_CHECK_RES(ret); + } + + /* mark source deleted */ + + ret = spiffs_page_delete(fs, src_pgndx); + return ret; +} + +/**************************************************************************** + * Name: spiffs_page_delete + * + * Description: + * Deletes a page and removes it from object lookup. + * + ****************************************************************************/ + +int spiffs_page_delete(FAR struct spiffs_s *fs, int16_t pgndx) +{ + int ret; + + /* mark deleted entry in source object lookup */ + + int16_t d_objid = SPIFFS_OBJ_ID_DELETED; + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_DELE, 0, + SPIFFS_BLOCK_TO_PADDR(fs, SPIFFS_BLOCK_FOR_PAGE(fs, pgndx)) + + SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(fs, pgndx) * + sizeof(int16_t), sizeof(int16_t), + (uint8_t *) & d_objid); + SPIFFS_CHECK_RES(ret); + + fs->stats_p_deleted++; + fs->stats_p_allocated--; + + /* mark deleted in source page */ + + uint8_t flags = 0xff; +#ifdef CONFIG_SPIFFS_NO_BLIND_WRITES + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, + pgndx) + offsetof(struct spiffs_page_header_s, + flags), + sizeof(flags), &flags); + SPIFFS_CHECK_RES(ret); +#endif + flags &= ~(SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_USED); + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_DELE, 0, + SPIFFS_PAGE_TO_PADDR(fs, pgndx) + offsetof(struct spiffs_page_header_s, flags), + sizeof(flags), &flags); + + return ret; +} + +/**************************************************************************** + * Name: spiffs_object_create + * + * Description: + * Create an object index header page with empty index and undefined length + * + ****************************************************************************/ + +int spiffs_object_create(FAR struct spiffs_s *fs, + int16_t objid, const uint8_t name[], + uint8_t type, FAR int16_t *objhdr_pgndx) +{ + struct spiffs_pgobj_ixheader_s objndx_hdr; + int16_t blkndx; + int entry; + int ret = OK; + + ret = spiffs_gc_check(fs, SPIFFS_DATA_PAGE_SIZE(fs)); + SPIFFS_CHECK_RES(ret); + + objid |= SPIFFS_OBJ_ID_IX_FLAG; + + /* find free entry */ + + ret = + spiffs_obj_lu_find_free(fs, fs->free_blkndx, + fs->free_entry, &blkndx, &entry); + SPIFFS_CHECK_RES(ret); + finfo("create: found free page @ %04x blkndx=%04x entry:" + "%04x\n", (int16_t) SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, + blkndx, + entry), + blkndx, entry); + + /* occupy page in object lookup */ + + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_UPDT, + 0, SPIFFS_BLOCK_TO_PADDR(fs, + blkndx) + + entry * sizeof(int16_t), sizeof(int16_t), + (uint8_t *) & objid); + SPIFFS_CHECK_RES(ret); + + fs->stats_p_allocated++; + + /* write empty object index page */ + + objndx_hdr.p_hdr.objid = objid; + objndx_hdr.p_hdr.spndx = 0; + objndx_hdr.p_hdr.flags = + 0xff & ~(SPIFFS_PH_FLAG_FINAL | SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_USED); + objndx_hdr.type = type; + objndx_hdr.size = SPIFFS_UNDEFINED_LEN; /* keep ones so we can update later + * without wasting this page */ + strncpy((char *)objndx_hdr.name, (const char *)name, CONFIG_SPIFFS_NAME_MAX); + + /* update page */ + + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + 0, SPIFFS_OBJ_LOOKUP_ENTRY_TO_PADDR(fs, blkndx, entry), + sizeof(struct spiffs_pgobj_ixheader_s), (uint8_t *) & objndx_hdr); + + SPIFFS_CHECK_RES(ret); + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) & objndx_hdr, + SPIFFS_EV_IX_NEW, objid, 0, + SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, entry), + SPIFFS_UNDEFINED_LEN); + + if (objhdr_pgndx) + { + *objhdr_pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, entry); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_object_update_index_hdr + * + * Description: + * Update object index header with any combination of name/size/index. + * new_objhdr_data may be NULL, if so the object index header page is + * loaded. Name may be NULL, if so name is not changed. size may be NULL, + * if so size is not changed + * + ****************************************************************************/ + +int spiffs_object_update_index_hdr(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, + int16_t objid, int16_t objhdr_pgndx, + FAR uint8_t *new_objhdr_data, + const uint8_t name[], + uint32_t size, FAR int16_t *new_pgndx) +{ + FAR struct spiffs_pgobj_ixheader_s *objhdr; + int16_t new_objhdr_pgndx; + int ret = OK; + + objid |= SPIFFS_OBJ_ID_IX_FLAG; + + if (new_objhdr_data) + { + /* object index header page already given to us, no need to load it */ + + objhdr = (struct spiffs_pgobj_ixheader_s *) new_objhdr_data; + } + else + { + /* read object index header page */ + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, objhdr_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(ret); + objhdr = (struct spiffs_pgobj_ixheader_s *) fs->work; + } + + SPIFFS_VALIDATE_OBJIX(objhdr->p_hdr, objid, 0); + + /* change name */ + + if (name) + { + strncpy((char *)objhdr->name, (const char *)name, CONFIG_SPIFFS_NAME_MAX); + } + + if (size) + { + objhdr->size = size; + } + + /* move and update page */ + + ret = spiffs_page_move(fs, fobj == NULL ? 0 : fobj->objid, + (FAR uint8_t *)objhdr, objid, + 0, objhdr_pgndx, &new_objhdr_pgndx); + + if (ret == OK) + { + if (new_pgndx) + { + *new_pgndx = new_objhdr_pgndx; + } + + /* callback on object index update */ + + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) objhdr, + new_objhdr_data ? SPIFFS_EV_IX_UPD : + SPIFFS_EV_IX_UPD_HDR, objid, + objhdr->p_hdr.spndx, new_objhdr_pgndx, + objhdr->size); + if (fobj) + { + fobj->objhdr_pgndx = new_objhdr_pgndx; /* if this is not in the + * registered cluster */ + } + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_cb_object_event + * + * Description: + * + ****************************************************************************/ + +void spiffs_cb_object_event(FAR struct spiffs_s *fs, + FAR spiffs_page_object_ix *objndx, + int ev, + int16_t objid_raw, + int16_t spndx, + int16_t new_pgndx, uint32_t new_size) +{ + FAR struct spiffs_file_s *fobj; + FAR struct spiffs_file_s *next; + int16_t objid = objid_raw & ~SPIFFS_OBJ_ID_IX_FLAG; + + finfo(" CALLBACK %s objid=%04x spndx=%04x npgndx:" + "%04x nsz=%d\n", (const char *[]) + { + "UPD", "NEW", "DEL", "MOV", "HUP", "???"}[MIN(ev, 5)], + objid_raw, spndx, new_pgndx, new_size); + + /* Update index caches in all file descriptors */ + + for (fobj = (FAR struct spiffs_file_s *)dq_peek(&fs->objq); + fobj != NULL; + fobj = next) + { + /* Set up for the next time through the loop (in case fobj is deleted) */ + + next = (FAR struct spiffs_file_s *)dq_next((FAR dq_entry_t *)fobj); + + /* Is this the object we are looking for? */ + + if ((fobj->objid & ~SPIFFS_OBJ_ID_IX_FLAG) != objid) + { + continue; /* Object not related to updated file */ + } + + if (spndx == 0) + { + /* object index header update */ + + if (ev != SPIFFS_EV_IX_DEL) + { + finfo("Setting objid=%d (offset=%d) objhdr_pgndx to %04x size=%d\n", + fobj->objid, fobj->offset, new_pgndx, new_size); + + fobj->objhdr_pgndx = new_pgndx; + if (new_size != 0) + { + /* update size and offsets for fobj to this file */ + + fobj->size = new_size; + uint32_t act_new_size = + new_size == SPIFFS_UNDEFINED_LEN ? 0 : new_size; + if (act_new_size > 0 && fobj->cache_page) + { + act_new_size = + MAX(act_new_size, + fobj->cache_page->offset + + fobj->cache_page->size); + } + + if (fobj->offset > act_new_size) + { + fobj->offset = act_new_size; + } + + if (fobj->cache_page && + fobj->cache_page->offset > act_new_size + 1) + { + spiffs_cacheinfo + ("CACHE_DROP: file truncated, dropping cache page " + "%d, no writeback\n", fobj->cache_page->cpndx); + spiffs_cache_page_release(fs, fobj->cache_page); + } + } + } + else + { + /* Removing file */ + + if (fobj->cache_page) + { + spiffs_cacheinfo + ("CACHE_DROP: file deleted, dropping cache page=%d" + ", no writeback\n", fobj->cache_page->cpndx); + spiffs_cache_page_release(fs, fobj->cache_page); + } + + finfo("callback: release objid=%d span=%04x objndx_pgndx to %04x\n", + fobj->objid, spndx, new_pgndx); + + /* Remove the file object */ + + spiffs_fobj_free(fs, fobj); + } + } + + if (fobj->objndx_spndx == spndx) + { + if (ev != SPIFFS_EV_IX_DEL) + { + finfo("callback: setting objid=%d span=%04x objndx_pgndx to %04x\n", + fobj->objid, spndx, new_pgndx); + + fobj->objndx_pgndx = new_pgndx; + } + else + { + fobj->objndx_pgndx = 0; + } + } + } +} + +/**************************************************************************** + * Name: spiffs_object_open_bypage + * + * Description: + * Open object by page index + * + ****************************************************************************/ + +int spiffs_object_open_bypage(FAR struct spiffs_s *fs, int16_t pgndx, + FAR struct spiffs_file_s *fobj, uint16_t flags, + uint16_t mode) +{ + struct spiffs_pgobj_ixheader_s objndx_hdr; + int16_t objid; + int16_t blkndx; + int entry; + int ret = OK; + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, pgndx), + sizeof(struct spiffs_pgobj_ixheader_s), + (FAR uint8_t *)&objndx_hdr); + SPIFFS_CHECK_RES(ret); + + blkndx = SPIFFS_BLOCK_FOR_PAGE(fs, pgndx); + entry = SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(fs, pgndx); + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, + 0, SPIFFS_BLOCK_TO_PADDR(fs, blkndx) + + entry * sizeof(int16_t), sizeof(int16_t), + (FAR uint8_t *)&objid); + + fobj->objhdr_pgndx = pgndx; + fobj->size = objndx_hdr.size; + fobj->offset = 0; + fobj->objndx_pgndx = pgndx; + fobj->objndx_spndx = 0; + fobj->objid = objid; + fobj->flags = flags; + + SPIFFS_VALIDATE_OBJIX(objndx_hdr.p_hdr, fobj->objid, 0); + + finfo("open: objid=%d\n", fobj->objid); + return ret; +} + +/**************************************************************************** + * Name: spiffs_object_append + * + * Description: + * Append to object. Deep current object index (header) page in fs->work + * buffer + * + ****************************************************************************/ + +int spiffs_object_append(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, off_t offset, + FAR uint8_t *data, size_t len) +{ + size_t written = 0; + int ret = OK; + + finfo("append: %d bytes @ offs=%d of size=%d" + "\n", len, offset, fobj->size); + + if (offset > fobj->size) + { + finfo("append: offset reversed to size\n"); + offset = fobj->size; + } + + ret = spiffs_gc_check(fs, len + SPIFFS_DATA_PAGE_SIZE(fs)); /* add an extra + * page of data + * worth for meta */ + if (ret != OK) + { + ferr("ERROR: spiffs_gc_check() failed: %d\n", ret); + } + + SPIFFS_CHECK_RES(ret); + + FAR struct spiffs_pgobj_ixheader_s *objhdr = + (FAR struct spiffs_pgobj_ixheader_s *) fs->work; + spiffs_page_object_ix *objndx = (spiffs_page_object_ix *) fs->work; + struct spiffs_page_header_s p_hdr; + + int16_t cur_objndx_spndx = 0; + int16_t prev_objndx_spndx = (int16_t) - 1; + int16_t cur_objndx_pgndx = fobj->objhdr_pgndx; + int16_t new_objhdr_page; + + int16_t data_spndx = offset / SPIFFS_DATA_PAGE_SIZE(fs); + int16_t data_page; + uint32_t page_offs = offset % SPIFFS_DATA_PAGE_SIZE(fs); + + /* write all data */ + + while (ret == OK && written < len) + { + /* calculate object index page span index */ + + cur_objndx_spndx = SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, data_spndx); + + /* handle storing and loading of object indices */ + + if (cur_objndx_spndx != prev_objndx_spndx) + { + /* new object index page. Within this clause we return directly + * if something fails, object index mess-up + */ + + if (written > 0) + { + /* store previous object index page, unless first pass */ + + finfo("append: %04x store objndx %04x:" + "%04x, written=%d\n", fobj->objid, + cur_objndx_pgndx, prev_objndx_spndx, written); + + if (prev_objndx_spndx == 0) + { + /* this is an update to object index header page */ + + objhdr->size = offset + written; + if (offset == 0) + { + /* was an empty object, update same page (size was + * 0xffffffff) + */ + + ret = spiffs_page_index_check(fs, fobj, cur_objndx_pgndx, 0); + SPIFFS_CHECK_RES(ret); + ret = + spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_UPDT, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, cur_objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(ret); + } + else + { + /* was a nonempty object, update to new page */ + + ret = spiffs_object_update_index_hdr(fs, fobj, fobj->objid, + fobj->objhdr_pgndx, + fs->work, 0, + offset + written, + &new_objhdr_page); + SPIFFS_CHECK_RES(ret); + finfo("append: %04x store new objhdr, " + "%04x:%04x, written=%d" + "\n", fobj->objid, new_objhdr_page, 0, + written); + } + } + else + { + /* this is an update to an object index page */ + + ret = + spiffs_page_index_check(fs, fobj, cur_objndx_pgndx, + prev_objndx_spndx); + SPIFFS_CHECK_RES(ret); + + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_UPDT, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, + cur_objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(ret); + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) fs->work, + SPIFFS_EV_IX_UPD, fobj->objid, + objndx->p_hdr.spndx, cur_objndx_pgndx, + 0); + + /* update length in object index header page */ + + ret = spiffs_object_update_index_hdr(fs, fobj, fobj->objid, + fobj->objhdr_pgndx, 0, 0, + offset + written, + &new_objhdr_page); + SPIFFS_CHECK_RES(ret); + finfo("append: %04x store new size I %d" + " in objhdr, %04x:%04x" + ", written=%d\n", fobj->objid, + offset + written, new_objhdr_page, 0, written); + } + + fobj->size = offset + written; + fobj->offset = offset + written; + } + + /* create or load new object index page */ + + if (cur_objndx_spndx == 0) + { + /* load object index header page, must always exist */ + + finfo("append: %04x load objndxhdr page %04x:%04x\n", + fobj->objid, cur_objndx_pgndx, cur_objndx_spndx); + ret = + spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, + cur_objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(ret); + SPIFFS_VALIDATE_OBJIX(objhdr->p_hdr, fobj->objid, + cur_objndx_spndx); + } + else + { + int16_t len_objndx_spndx = + SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, + (fobj->size - 1) / + SPIFFS_DATA_PAGE_SIZE(fs)); + + /* on subsequent passes, create a new object index page */ + + if (written > 0 || cur_objndx_spndx > len_objndx_spndx) + { + p_hdr.objid = fobj->objid | SPIFFS_OBJ_ID_IX_FLAG; + p_hdr.spndx = cur_objndx_spndx; + p_hdr.flags = + 0xff & ~(SPIFFS_PH_FLAG_FINAL | SPIFFS_PH_FLAG_INDEX); + ret = + spiffs_page_allocate_data(fs, + fobj->objid | SPIFFS_OBJ_ID_IX_FLAG, + &p_hdr, 0, 0, 0, 1, + &cur_objndx_pgndx); + SPIFFS_CHECK_RES(ret); + + /* quick "load" of new object index page */ + + memset(fs->work, 0xff, SPIFFS_CFG_LOG_PAGE_SZ(fs)); + memcpy(fs->work, &p_hdr, sizeof(struct spiffs_page_header_s)); + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) fs->work, + SPIFFS_EV_IX_NEW, fobj->objid, + cur_objndx_spndx, cur_objndx_pgndx, 0); + finfo("append: %04x create objndx page, " + "%04x:%04x, written=%d\n", + fobj->objid, cur_objndx_pgndx, cur_objndx_spndx, + written); + } + else + { + /* on first pass, we load existing object index page */ + + int16_t pgndx; + finfo("append: %04x find objndx spndx:" + "%04x\n", fobj->objid, cur_objndx_spndx); + if (fobj->objndx_spndx == cur_objndx_spndx) + { + pgndx = fobj->objndx_pgndx; + } + else + { + ret = + spiffs_obj_lu_find_id_and_span(fs, + fobj->objid | SPIFFS_OBJ_ID_IX_FLAG, + cur_objndx_spndx, 0, &pgndx); + SPIFFS_CHECK_RES(ret); + } + finfo("append: %04x found object index at page " + "%04x [fobj size=%d]\n", fobj->objid, + pgndx, fobj->size); + ret = + spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(ret); + SPIFFS_VALIDATE_OBJIX(objhdr->p_hdr, fobj->objid, + cur_objndx_spndx); + cur_objndx_pgndx = pgndx; + } + + fobj->objndx_pgndx = cur_objndx_pgndx; + fobj->objndx_spndx = cur_objndx_spndx; + fobj->offset = offset + written; + fobj->size = offset + written; + } + + prev_objndx_spndx = cur_objndx_spndx; + } + + /* write data */ + + uint32_t to_write = + MIN(len - written, SPIFFS_DATA_PAGE_SIZE(fs) - page_offs); + if (page_offs == 0) + { + /* at beginning of a page, allocate and write a new page of data */ + + p_hdr.objid = fobj->objid & ~SPIFFS_OBJ_ID_IX_FLAG; + p_hdr.spndx = data_spndx; + p_hdr.flags = 0xff & ~(SPIFFS_PH_FLAG_FINAL); /* finalize immediately */ + ret = + spiffs_page_allocate_data(fs, fobj->objid & ~SPIFFS_OBJ_ID_IX_FLAG, + &p_hdr, &data[written], to_write, + page_offs, 1, &data_page); + finfo("append: %04x store new data page, %04x:" + "%04x offset=%d, len=%d" + ", written=%d\n", fobj->objid, data_page, + data_spndx, page_offs, to_write, written); + } + else + { + /* append to existing page, fill out free data in existing page */ + + if (cur_objndx_spndx == 0) + { + /* get data page from object index header page */ + + data_page = + ((int16_t *) ((uint8_t *) objhdr + + sizeof(struct spiffs_pgobj_ixheader_s))) + [data_spndx]; + } + else + { + /* get data page from object index page */ + + data_page = + ((int16_t *) ((uint8_t *) objndx + + sizeof(spiffs_page_object_ix))) + [SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)]; + } + + ret = spiffs_page_data_check(fs, fobj, data_page, data_spndx); + SPIFFS_CHECK_RES(ret); + + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, + data_page) + + sizeof(struct spiffs_page_header_s) + page_offs, to_write, + &data[written]); + finfo("append: %04x store to existing data page, " + "%04x:%04x offset=%d, len " + "%d, written=%d\n", fobj->objid, data_page, + data_spndx, page_offs, to_write, written); + } + + if (ret != OK) + { + break; + } + + /* update memory representation of object index page with new data page */ + + if (cur_objndx_spndx == 0) + { + /* update object index header page */ + + ((int16_t *) ((uint8_t *) objhdr + + sizeof(struct spiffs_pgobj_ixheader_s)))[data_spndx] + = data_page; + + finfo("append: %04x wrote page %04x" + " to objhdr entry %04x in mem\n", fobj->objid, + data_page, data_spndx); + + objhdr->size = offset + written; + } + else + { + /* update object index page */ + + ((int16_t *) ((uint8_t *) objndx + + sizeof(spiffs_page_object_ix))) + [SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)] = data_page; + + finfo("append: %04x wrote page %04x" + " to objndx entry %04x in mem\n", fobj->objid, + data_page, (int16_t) SPIFFS_OBJ_IX_ENTRY(fs, + data_spndx)); + } + + /* update internals */ + + page_offs = 0; + data_spndx++; + written += to_write; + } + + fobj->size = offset + written; + fobj->offset = offset + written; + fobj->objndx_pgndx = cur_objndx_pgndx; + fobj->objndx_spndx = cur_objndx_spndx; + + /* finalize updated object indices */ + + int res2 = OK; + if (cur_objndx_spndx != 0) + { + /* wrote beyond object index header page. Write last modified object + * index page, unless object header index page + */ + + finfo("append: %04x store objndx page, %04x:" + "%04x, written=%d\n", fobj->objid, + cur_objndx_pgndx, cur_objndx_spndx, written); + + res2 = spiffs_page_index_check(fs, fobj, cur_objndx_pgndx, cur_objndx_spndx); + SPIFFS_CHECK_RES(res2); + + res2 = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_UPDT, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, cur_objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(res2); + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) fs->work, + SPIFFS_EV_IX_UPD, fobj->objid, objndx->p_hdr.spndx, + cur_objndx_pgndx, 0); + + /* update size in object header index page */ + + res2 = spiffs_object_update_index_hdr(fs, fobj, fobj->objid, + fobj->objhdr_pgndx, 0, 0, + offset + written, + &new_objhdr_page); + finfo("append: %04x store new size II %d" + " in objhdr, %04x:%04x, written=%d" + ", ret=%d\n", fobj->objid, offset + written, + new_objhdr_page, 0, written, res2); + SPIFFS_CHECK_RES(res2); + } + else + { + /* wrote within object index header page */ + + if (offset == 0) + { + /* wrote to empty object - simply update size and write whole page */ + + objhdr->size = offset + written; + finfo("append: %04x store fresh objhdr page, " + "%04x:%04x, written=%d\n", + fobj->objid, cur_objndx_pgndx, cur_objndx_spndx, written); + + res2 = spiffs_page_index_check(fs, fobj, cur_objndx_pgndx, cur_objndx_spndx); + SPIFFS_CHECK_RES(res2); + + res2 = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_UPDT, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, cur_objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(res2); + + /* callback on object index update */ + + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) fs->work, + SPIFFS_EV_IX_UPD_HDR, fobj->objid, + objhdr->p_hdr.spndx, cur_objndx_pgndx, + objhdr->size); + } + else + { + /* modifying object index header page, update size and make new copy */ + + res2 = spiffs_object_update_index_hdr(fs, fobj, fobj->objid, + fobj->objhdr_pgndx, fs->work, 0, + offset + written, + &new_objhdr_page); + finfo("append: %04x store modified objhdr page, " + "%04x:%04x, written=%d\n", + fobj->objid, new_objhdr_page, 0, written); + SPIFFS_CHECK_RES(res2); + } + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_object_modify + * + * Description: + * Modify object. Keep current object index (header) page in fs->work + * buffer + * + ****************************************************************************/ + +int spiffs_object_modify(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, off_t offset, + FAR uint8_t *data, size_t len) +{ + size_t written = 0; + int ret = OK; + + ret = spiffs_gc_check(fs, len + SPIFFS_DATA_PAGE_SIZE(fs)); + SPIFFS_CHECK_RES(ret); + + FAR struct spiffs_pgobj_ixheader_s *objhdr = + (FAR struct spiffs_pgobj_ixheader_s *) fs->work; + spiffs_page_object_ix *objndx = (spiffs_page_object_ix *) fs->work; + struct spiffs_page_header_s p_hdr; + + int16_t cur_objndx_spndx = 0; + int16_t prev_objndx_spndx = (int16_t) - 1; + int16_t cur_objndx_pgndx = fobj->objhdr_pgndx; + int16_t new_objhdr_pgndx; + + int16_t data_spndx = offset / SPIFFS_DATA_PAGE_SIZE(fs); + int16_t data_pgndx; + uint32_t page_offs = offset % SPIFFS_DATA_PAGE_SIZE(fs); + + /* write all data */ + + while (ret == OK && written < len) + { + /* calculate object index page span index */ + + cur_objndx_spndx = SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, data_spndx); + + /* handle storing and loading of object indices */ + + if (cur_objndx_spndx != prev_objndx_spndx) + { + /* new object index page. Within this clause we return directly + * if something fails, object index mess-up + */ + + if (written > 0) + { + /* store previous object index (header) page, unless first pass */ + + if (prev_objndx_spndx == 0) + { + /* store previous object index header page */ + + ret = spiffs_object_update_index_hdr(fs, fobj, fobj->objid, + fobj->objhdr_pgndx, + fs->work, 0, 0, + &new_objhdr_pgndx); + finfo("modify: store modified objhdr page, %04x" + ":%04x, written=%d\n", + new_objhdr_pgndx, 0, written); + SPIFFS_CHECK_RES(ret); + } + else + { + /* store new version of previous object index page */ + + int16_t new_objndx_pgndx; + + ret = + spiffs_page_index_check(fs, fobj, cur_objndx_pgndx, + prev_objndx_spndx); + SPIFFS_CHECK_RES(ret); + + ret = spiffs_page_move(fs, fobj->objid, (FAR uint8_t *)objndx, + fobj->objid, 0, cur_objndx_pgndx, + &new_objndx_pgndx); + finfo("modify: store previous modified objndx page, " + "%04x:%04x, written=%d\n", + new_objndx_pgndx, objndx->p_hdr.spndx, written); + SPIFFS_CHECK_RES(ret); + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) objndx, + SPIFFS_EV_IX_UPD, fobj->objid, + objndx->p_hdr.spndx, new_objndx_pgndx, + 0); + } + } + + /* load next object index page */ + + if (cur_objndx_spndx == 0) + { + /* load object index header page, must exist */ + + finfo("modify: load objndxhdr page %04x:%04x" + "\n", cur_objndx_pgndx, cur_objndx_spndx); + ret = + spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, + cur_objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(ret); + SPIFFS_VALIDATE_OBJIX(objhdr->p_hdr, fobj->objid, + cur_objndx_spndx); + } + else + { + /* load existing object index page on first pass */ + + int16_t pgndx; + finfo("modify: find objndx spndx=%04x\n", + cur_objndx_spndx); + if (fobj->objndx_spndx == cur_objndx_spndx) + { + pgndx = fobj->objndx_pgndx; + } + else + { + ret = + spiffs_obj_lu_find_id_and_span(fs, + fobj->objid | SPIFFS_OBJ_ID_IX_FLAG, + cur_objndx_spndx, 0, &pgndx); + SPIFFS_CHECK_RES(ret); + } + finfo("modify: found object index at page %04x\n", + pgndx); + ret = + spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(ret); + SPIFFS_VALIDATE_OBJIX(objhdr->p_hdr, fobj->objid, + cur_objndx_spndx); + cur_objndx_pgndx = pgndx; + } + + fobj->objndx_pgndx = cur_objndx_pgndx; + fobj->objndx_spndx = cur_objndx_spndx; + fobj->offset = offset + written; + prev_objndx_spndx = cur_objndx_spndx; + } + + /* write partial data */ + + uint32_t to_write = + MIN(len - written, SPIFFS_DATA_PAGE_SIZE(fs) - page_offs); + int16_t orig_data_pgndx; + + if (cur_objndx_spndx == 0) + { + /* get data page from object index header page */ + + orig_data_pgndx = + ((int16_t *) ((uint8_t *) objhdr + + sizeof(struct spiffs_pgobj_ixheader_s))) + [data_spndx]; + } + else + { + /* get data page from object index page */ + + orig_data_pgndx = + ((int16_t *) ((uint8_t *) objndx + + sizeof(spiffs_page_object_ix))) + [SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)]; + } + + p_hdr.objid = fobj->objid & ~SPIFFS_OBJ_ID_IX_FLAG; + p_hdr.spndx = data_spndx; + p_hdr.flags = 0xff; + + if (page_offs == 0 && to_write == SPIFFS_DATA_PAGE_SIZE(fs)) + { + /* a full page, allocate and write a new page of data */ + + ret = + spiffs_page_allocate_data(fs, fobj->objid & ~SPIFFS_OBJ_ID_IX_FLAG, + &p_hdr, &data[written], to_write, + page_offs, 1, &data_pgndx); + finfo("modify: store new data page, %04x:%04x" + " offset=%d, len=%d, written=%d" + "\n", data_pgndx, data_spndx, page_offs, to_write, written); + } + else + { + /* write to existing page, allocate new and copy unmodified data */ + + ret = spiffs_page_data_check(fs, fobj, orig_data_pgndx, data_spndx); + SPIFFS_CHECK_RES(ret); + + ret = + spiffs_page_allocate_data(fs, fobj->objid & ~SPIFFS_OBJ_ID_IX_FLAG, + &p_hdr, 0, 0, 0, 0, &data_pgndx); + if (ret != OK) + break; + + /* copy unmodified data */ + + if (page_offs > 0) + { + /* before modification */ + + ret = spiffs_phys_cpy(fs, fobj->objid, + SPIFFS_PAGE_TO_PADDR(fs, data_pgndx) + + sizeof(struct spiffs_page_header_s), + SPIFFS_PAGE_TO_PADDR(fs, orig_data_pgndx) + + sizeof(struct spiffs_page_header_s), page_offs); + if (ret != OK) + break; + } + if (page_offs + to_write < SPIFFS_DATA_PAGE_SIZE(fs)) + { + /* after modification */ + + ret = spiffs_phys_cpy(fs, fobj->objid, + SPIFFS_PAGE_TO_PADDR(fs, + data_pgndx) + + sizeof(struct spiffs_page_header_s) + page_offs + + to_write, SPIFFS_PAGE_TO_PADDR(fs, + orig_data_pgndx) + + sizeof(struct spiffs_page_header_s) + page_offs + + to_write, + SPIFFS_DATA_PAGE_SIZE(fs) - (page_offs + + to_write)); + if (ret != OK) + { + break; + } + } + + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, data_pgndx) + + sizeof(struct spiffs_page_header_s) + page_offs, to_write, + &data[written]); + if (ret != OK) + { + break; + } + + p_hdr.flags &= ~SPIFFS_PH_FLAG_FINAL; + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, data_pgndx) + + offsetof(struct spiffs_page_header_s, flags), sizeof(uint8_t), + (uint8_t *) & p_hdr.flags); + if (ret != OK) + { + break; + } + + finfo("modify: store to existing data page, src=%04x" + ", dst=%04x:%04x offset:%d" + ", len=%d, written=%d\n", + orig_data_pgndx, data_pgndx, data_spndx, page_offs, to_write, + written); + } + + /* delete original data page */ + + ret = spiffs_page_delete(fs, orig_data_pgndx); + if (ret != OK) + { + break; + } + + /* update memory representation of object index page with new data page */ + + if (cur_objndx_spndx == 0) + { + /* update object index header page */ + + ((int16_t *) ((uint8_t *) objhdr + + sizeof(struct spiffs_pgobj_ixheader_s)))[data_spndx] + = data_pgndx; + finfo("modify: wrote page %04x to objhdr entry " + "%04x in mem\n", data_pgndx, data_spndx); + } + else + { + /* update object index page */ + + ((int16_t *) ((uint8_t *) objndx + + sizeof(spiffs_page_object_ix))) + [SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)] = data_pgndx; + finfo("modify: wrote page %04x to objndx entry " + "%04x in mem\n", data_pgndx, + (int16_t) SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)); + } + + /* update internals */ + + page_offs = 0; + data_spndx++; + written += to_write; + } + + fobj->offset = offset + written; + fobj->objndx_pgndx = cur_objndx_pgndx; + fobj->objndx_spndx = cur_objndx_spndx; + + /* finalize updated object indices */ + + int res2 = OK; + if (cur_objndx_spndx != 0) + { + /* wrote beyond object index header page. Write last modified object + * index page. Move and update page + */ + + int16_t new_objndx_pgndx; + + res2 = spiffs_page_index_check(fs, fobj, cur_objndx_pgndx, cur_objndx_spndx); + SPIFFS_CHECK_RES(res2); + + res2 = spiffs_page_move(fs, fobj->objid, (uint8_t *) objndx, fobj->objid, 0, + cur_objndx_pgndx, &new_objndx_pgndx); + finfo("modify: store modified objndx page, %04x:%04x" + ", written=%d\n", new_objndx_pgndx, cur_objndx_spndx, + written); + fobj->objndx_pgndx = new_objndx_pgndx; + fobj->objndx_spndx = cur_objndx_spndx; + SPIFFS_CHECK_RES(res2); + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) objndx, + SPIFFS_EV_IX_UPD, fobj->objid, objndx->p_hdr.spndx, + new_objndx_pgndx, 0); + + } + else + { + /* wrote within object index header page */ + + res2 = spiffs_object_update_index_hdr(fs, fobj, fobj->objid, + fobj->objhdr_pgndx, fs->work, 0, + 0, &new_objhdr_pgndx); + finfo("modify: store modified objhdr page, %04x:" + "%04x, written=%d\n", new_objhdr_pgndx, 0, + written); + SPIFFS_CHECK_RES(res2); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_find_objhdr_pgndx + * + * Description: + * Finds object index header page by name + * + ****************************************************************************/ + +int spiffs_find_objhdr_pgndx(FAR struct spiffs_s *fs, + const uint8_t name[CONFIG_SPIFFS_NAME_MAX], + FAR int16_t *pgndx) +{ + int16_t blkndx; + int entry; + int ret; + + ret = spiffs_foreach_objlu(fs, fs->lu_blkndx, fs->lu_entry, + 0, 0, spiffs_find_objhdr_pgndx_callback, + name, 0, &blkndx, &entry); + + if (ret == SPIFFS_VIS_END) + { + ret = -ENOENT; + } + + SPIFFS_CHECK_RES(ret); + + if (pgndx) + { + *pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, entry); + } + + fs->lu_blkndx = blkndx; + fs->lu_entry = entry; + + return ret; +} + +/**************************************************************************** + * Name: spiffs_object_truncate + * + * Description: + * Truncates object to new size. If new size is NULL, object may be removed + * totally + * + ****************************************************************************/ + +int spiffs_object_truncate(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, off_t new_size, + bool remove_full) +{ + int ret = OK; + + /* If the file has zero (or undefined) length and we were not asked to + * remove the file, then there is nothing to do. + */ + + if ((fobj->size == SPIFFS_UNDEFINED_LEN || fobj->size == 0) && !remove_full) + { + /* Do nothing */ + + return ret; + } + + /* need 2 pages if not removing: object index page + possibly chopped data + * page + */ + + if (!remove_full) + { + ret = spiffs_gc_check(fs, SPIFFS_DATA_PAGE_SIZE(fs) * 2); + SPIFFS_CHECK_RES(ret); + } + + int16_t objndx_pgndx = fobj->objhdr_pgndx; + int16_t data_spndx = + (fobj->size > 0 ? fobj->size - 1 : 0) / SPIFFS_DATA_PAGE_SIZE(fs); + uint32_t cur_size = fobj->size == (uint32_t) SPIFFS_UNDEFINED_LEN ? 0 : fobj->size; + int16_t cur_objndx_spndx = 0; + int16_t prev_objndx_spndx = (int16_t) - 1; + FAR struct spiffs_pgobj_ixheader_s *objhdr = + (FAR struct spiffs_pgobj_ixheader_s *) fs->work; + spiffs_page_object_ix *objndx = (spiffs_page_object_ix *) fs->work; + int16_t data_pgndx; + int16_t new_objhdr_pgndx; + + /* before truncating, check if object is to be fully removed and mark this */ + + if (remove_full && new_size == 0) + { + uint8_t flags = + ~(SPIFFS_PH_FLAG_USED | SPIFFS_PH_FLAG_INDEX | SPIFFS_PH_FLAG_FINAL | + SPIFFS_PH_FLAG_IXDELE); + ret = + spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_UPDT, fobj->objid, + SPIFFS_PAGE_TO_PADDR(fs, + fobj->objhdr_pgndx) + + offsetof(struct spiffs_page_header_s, flags), sizeof(uint8_t), + (uint8_t *) & flags); + SPIFFS_CHECK_RES(ret); + } + + /* Delete from end of object until desired len is reached */ + + while (cur_size > new_size) + { + cur_objndx_spndx = SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, data_spndx); + + /* Put object index for current data span index in work buffer */ + + if (prev_objndx_spndx != cur_objndx_spndx) + { + if (prev_objndx_spndx != (int16_t)- 1) + { + /* remove previous object index page */ + + finfo("truncate: delete objndx page %04x:%04x\n", + objndx_pgndx, prev_objndx_spndx); + + ret = spiffs_page_index_check(fs, fobj, objndx_pgndx, prev_objndx_spndx); + SPIFFS_CHECK_RES(ret); + + ret = spiffs_page_delete(fs, objndx_pgndx); + SPIFFS_CHECK_RES(ret); + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) 0, + SPIFFS_EV_IX_DEL, fobj->objid, + objndx->p_hdr.spndx, objndx_pgndx, 0); + if (prev_objndx_spndx > 0) + { + /* Update object index header page, unless we totally want to + * remove the file. + * If fully removing, we're not keeping consistency as good + * as when storing the header between chunks, + * would we be aborted. But when removing full files, a + * crammed system may otherwise + * report ERR_FULL a la windows. We cannot have that. + * Hence, take the risk - if aborted, a file check would free + * the lost pages and mend things + * as the file is marked as fully deleted in the beginning. + */ + + if (!remove_full) + { + finfo("truncate: update objndx hdr page %04x" + ":%04x to size=%d\n", + fobj->objhdr_pgndx, prev_objndx_spndx, cur_size); + ret = + spiffs_object_update_index_hdr(fs, fobj, fobj->objid, + fobj->objhdr_pgndx, 0, 0, + cur_size, &new_objhdr_pgndx); + SPIFFS_CHECK_RES(ret); + } + + fobj->size = cur_size; + } + } + + /* load current object index (header) page */ + + if (cur_objndx_spndx == 0) + { + objndx_pgndx = fobj->objhdr_pgndx; + } + else + { + ret = + spiffs_obj_lu_find_id_and_span(fs, + fobj->objid | SPIFFS_OBJ_ID_IX_FLAG, + cur_objndx_spndx, 0, &objndx_pgndx); + SPIFFS_CHECK_RES(ret); + } + + finfo("truncate: load objndx page %04x:%04x" + " for data spndx=%04x\n", objndx_pgndx, + cur_objndx_spndx, data_spndx); + ret = + spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, fobj->objid, + SPIFFS_PAGE_TO_PADDR(fs, objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + SPIFFS_CHECK_RES(ret); + SPIFFS_VALIDATE_OBJIX(objhdr->p_hdr, fobj->objid, cur_objndx_spndx); + fobj->objndx_pgndx = objndx_pgndx; + fobj->objndx_spndx = cur_objndx_spndx; + fobj->offset = cur_size; + + prev_objndx_spndx = cur_objndx_spndx; + } + + if (cur_objndx_spndx == 0) + { + /* get data page from object index header page */ + + data_pgndx = + ((int16_t *) ((uint8_t *) objhdr + + sizeof(struct spiffs_pgobj_ixheader_s))) + [data_spndx]; + ((int16_t *) ((uint8_t *) objhdr + + sizeof(struct spiffs_pgobj_ixheader_s)))[data_spndx] + = SPIFFS_OBJ_ID_FREE; + } + else + { + /* get data page from object index page */ + + data_pgndx = + ((int16_t *) ((uint8_t *) objndx + + sizeof(spiffs_page_object_ix))) + [SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)]; + ((int16_t *) ((uint8_t *) objndx + + sizeof(spiffs_page_object_ix))) + [SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)] = SPIFFS_OBJ_ID_FREE; + } + + finfo("truncate: got data pgndx %04x\n", data_pgndx); + + if (new_size == 0 || remove_full || + cur_size - new_size >= SPIFFS_DATA_PAGE_SIZE(fs)) + { + /* delete full data page */ + + ret = spiffs_page_data_check(fs, fobj, data_pgndx, data_spndx); + if (ret != SPIFFS_ERR_DELETED && ret != OK && + ret != SPIFFS_ERR_INDEX_REF_FREE) + { + finfo("truncate: err validating data pgndx=%d\n", + ret); + break; + } + + if (ret == OK) + { + ret = spiffs_page_delete(fs, data_pgndx); + if (ret != OK) + { + finfo("truncate: err deleting data pgndx=%d\n", + ret); + break; + } + } + else if (ret == SPIFFS_ERR_DELETED || + ret == SPIFFS_ERR_INDEX_REF_FREE) + { + ret = OK; + } + + /* update current size */ + + if (cur_size % SPIFFS_DATA_PAGE_SIZE(fs) == 0) + { + cur_size -= SPIFFS_DATA_PAGE_SIZE(fs); + } + else + { + cur_size -= cur_size % SPIFFS_DATA_PAGE_SIZE(fs); + } + + fobj->size = cur_size; + fobj->offset = cur_size; + + finfo("truncate: delete data page %04x for data spndx:" + "%04x, cur_size=%d\n", data_pgndx, data_spndx, + cur_size); + } + else + { + /* delete last page, partially */ + + struct spiffs_page_header_s p_hdr; + int16_t new_data_pgndx; + uint32_t bytes_to_remove = + SPIFFS_DATA_PAGE_SIZE(fs) - (new_size % SPIFFS_DATA_PAGE_SIZE(fs)); + finfo("truncate: delete %d bytes from data page " + "%04x for data spndx=%04x, cur_size:" + "%d\n", bytes_to_remove, data_pgndx, data_spndx, + cur_size); + + ret = spiffs_page_data_check(fs, fobj, data_pgndx, data_spndx); + if (ret != OK) + { + break; + } + + p_hdr.objid = fobj->objid & ~SPIFFS_OBJ_ID_IX_FLAG; + p_hdr.spndx = data_spndx; + p_hdr.flags = 0xff; + + /* allocate new page and copy unmodified data */ + + ret = + spiffs_page_allocate_data(fs, fobj->objid & ~SPIFFS_OBJ_ID_IX_FLAG, + &p_hdr, 0, 0, 0, 0, &new_data_pgndx); + if (ret != OK) + { + break; + } + + ret = spiffs_phys_cpy(fs, 0, + SPIFFS_PAGE_TO_PADDR(fs, + new_data_pgndx) + + sizeof(struct spiffs_page_header_s), + SPIFFS_PAGE_TO_PADDR(fs, + data_pgndx) + + sizeof(struct spiffs_page_header_s), + SPIFFS_DATA_PAGE_SIZE(fs) - bytes_to_remove); + if (ret != OK) + { + break; + } + + /* delete original data page */ + + ret = spiffs_page_delete(fs, data_pgndx); + if (ret != OK) + { + break; + } + + p_hdr.flags &= ~SPIFFS_PH_FLAG_FINAL; + ret = spiffs_cache_write(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_UPDT, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, new_data_pgndx) + + offsetof(struct spiffs_page_header_s, flags), sizeof(uint8_t), + (uint8_t *) & p_hdr.flags); + if (ret != OK) + { + break; + } + + /* update memory representation of object index page with new data + * page + */ + + if (cur_objndx_spndx == 0) + { + /* update object index header page */ + + ((int16_t *) ((uint8_t *) objhdr + + sizeof(struct spiffs_pgobj_ixheader_s))) + [data_spndx] = new_data_pgndx; + finfo("truncate: wrote page %04x" + " to objhdr entry %04x in mem\n", + new_data_pgndx, (int16_t) SPIFFS_OBJ_IX_ENTRY(fs, + data_spndx)); + } + else + { + /* update object index page */ + + ((int16_t *) ((uint8_t *) objndx + + sizeof(spiffs_page_object_ix))) + [SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)] = new_data_pgndx; + finfo("truncate: wrote page %04x to objndx entry " + "%04x in mem\n", new_data_pgndx, + (int16_t) SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)); + } + + cur_size = new_size; + fobj->size = new_size; + fobj->offset = cur_size; + break; + } + + data_spndx--; + } + + /* update object indices */ + + if (cur_objndx_spndx == 0) + { + /* update object index header page */ + + if (cur_size == 0) + { + if (remove_full) + { + /* remove object altogether */ + + finfo("truncate: remove object index header page %04x" + "\n", objndx_pgndx); + + ret = spiffs_page_index_check(fs, fobj, objndx_pgndx, 0); + SPIFFS_CHECK_RES(ret); + + ret = spiffs_page_delete(fs, objndx_pgndx); + SPIFFS_CHECK_RES(ret); + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) 0, + SPIFFS_EV_IX_DEL, fobj->objid, 0, objndx_pgndx, + 0); + } + else + { + /* make uninitialized object */ + + finfo("truncate: reset objhdr page %04x\n", + objndx_pgndx); + memset(fs->work + sizeof(struct spiffs_pgobj_ixheader_s), 0xff, + SPIFFS_CFG_LOG_PAGE_SZ(fs) - + sizeof(struct spiffs_pgobj_ixheader_s)); + ret = + spiffs_object_update_index_hdr(fs, fobj, fobj->objid, objndx_pgndx, + fs->work, 0, SPIFFS_UNDEFINED_LEN, + &new_objhdr_pgndx); + SPIFFS_CHECK_RES(ret); + } + } + else + { + /* update object index header page */ + + finfo("truncate: update object index header page with indices and size\n"); + ret = + spiffs_object_update_index_hdr(fs, fobj, fobj->objid, objndx_pgndx, + fs->work, 0, cur_size, + &new_objhdr_pgndx); + SPIFFS_CHECK_RES(ret); + } + } + else + { + /* update both current object index page and object index header page */ + + int16_t new_objndx_pgndx; + + ret = spiffs_page_index_check(fs, fobj, objndx_pgndx, cur_objndx_spndx); + SPIFFS_CHECK_RES(ret); + + /* move and update object index page */ + + ret = spiffs_page_move(fs, fobj->objid, (uint8_t *) objhdr, fobj->objid, 0, + objndx_pgndx, &new_objndx_pgndx); + SPIFFS_CHECK_RES(ret); + spiffs_cb_object_event(fs, (spiffs_page_object_ix *) objhdr, + SPIFFS_EV_IX_UPD, fobj->objid, objndx->p_hdr.spndx, + new_objndx_pgndx, 0); + finfo("truncate: store modified objndx page, %04x:%04x" + "\n", new_objndx_pgndx, cur_objndx_spndx); + fobj->objndx_pgndx = new_objndx_pgndx; + fobj->objndx_spndx = cur_objndx_spndx; + fobj->offset = cur_size; + + /* update object index header page with new size */ + + ret = spiffs_object_update_index_hdr(fs, fobj, fobj->objid, + fobj->objhdr_pgndx, 0, 0, cur_size, + &new_objhdr_pgndx); + SPIFFS_CHECK_RES(ret); + } + + fobj->size = cur_size; + return ret; +} + +/**************************************************************************** + * Name: spiffs_object_read + * + * Description: + * + ****************************************************************************/ + +ssize_t spiffs_object_read(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, off_t offset, + size_t len, FAR uint8_t *dst) +{ + int16_t objndx_pgndx; + int16_t data_pgndx; + int16_t data_spndx = offset / SPIFFS_DATA_PAGE_SIZE(fs); + uint32_t cur_offset = offset; + int16_t cur_objndx_spndx; + int16_t prev_objndx_spndx = (int16_t) - 1; + FAR struct spiffs_pgobj_ixheader_s *objhdr = + (FAR struct spiffs_pgobj_ixheader_s *) fs->work; + spiffs_page_object_ix *objndx = (spiffs_page_object_ix *) fs->work; + int ret = OK; + + while (cur_offset < offset + len) + { + cur_objndx_spndx = SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, data_spndx); + if (prev_objndx_spndx != cur_objndx_spndx) + { + /* load current object index (header) page */ + + if (cur_objndx_spndx == 0) + { + objndx_pgndx = fobj->objhdr_pgndx; + } + else + { + finfo("read: find objndx %04x:%04x\n", + fobj->objid, cur_objndx_spndx); + if (fobj->objndx_spndx == cur_objndx_spndx) + { + objndx_pgndx = fobj->objndx_pgndx; + } + else + { + ret = spiffs_obj_lu_find_id_and_span(fs, + fobj->objid | SPIFFS_OBJ_ID_IX_FLAG, + cur_objndx_spndx, 0, + &objndx_pgndx); + if (ret < 0) + { + return ret; + } + } + } + + finfo("read: load objndx page %d:%d for data spndx=%d\n", + objndx_pgndx, cur_objndx_spndx, data_spndx); + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + if (ret < 0) + { + return ret; + } + + SPIFFS_VALIDATE_OBJIX(objndx->p_hdr, fobj->objid, cur_objndx_spndx); + + fobj->offset = cur_offset; + fobj->objndx_pgndx = objndx_pgndx; + fobj->objndx_spndx = cur_objndx_spndx; + prev_objndx_spndx = cur_objndx_spndx; + } + + if (cur_objndx_spndx == 0) + { + /* Get data page from object index header page */ + + data_pgndx = ((int16_t *) + ((uint8_t *) objhdr + + sizeof(struct spiffs_pgobj_ixheader_s)))[data_spndx]; + } + else + { + /* Get data page from object index page */ + + data_pgndx = ((int16_t *) + ((uint8_t *) objndx + + sizeof(spiffs_page_object_ix)))[SPIFFS_OBJ_IX_ENTRY(fs, data_spndx)]; + } + + /* All remaining data */ + + uint32_t len_to_read = offset + len - cur_offset; + + /* Remaining data in page */ + + len_to_read = MIN(len_to_read, + SPIFFS_DATA_PAGE_SIZE(fs) - + (cur_offset % SPIFFS_DATA_PAGE_SIZE(fs))); + + /* Remaining data in file */ + + len_to_read = MIN(len_to_read, fobj->size); + + finfo("read: offset=%d rd=%d data spndx=%d is data_pgndx=%d addr=%p\n", + cur_offset, len_to_read, data_spndx, data_pgndx, + (SPIFFS_PAGE_TO_PADDR(fs, data_pgndx) + + sizeof(struct spiffs_page_header_s) + + (cur_offset % SPIFFS_DATA_PAGE_SIZE(fs)))); + + if (len_to_read <= 0) + { + ret = 0; + break; + } + + ret = spiffs_page_data_check(fs, fobj, data_pgndx, data_spndx); + if (ret < 0) + { + return ret; + } + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_DA | SPIFFS_OP_C_READ, + fobj->objid, SPIFFS_PAGE_TO_PADDR(fs, data_pgndx) + + sizeof(struct spiffs_page_header_s) + + (cur_offset % SPIFFS_DATA_PAGE_SIZE(fs)), len_to_read, + dst); + if (ret < 0) + { + return ret; + } + + dst += len_to_read; + cur_offset += len_to_read; + fobj->offset = cur_offset; + data_spndx++; + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_obj_lu_find_free_objid + * + * Description: + * Scans thru all object lookup for object index header pages. If total + * possible number of object ids cannot fit into a work buffer, these are + * grouped. When a group containing free object ids is found, the object + * lu is again scanned for object ids within group and bitmasked. Finally, + * the bitmask is searched for a free objid + * + ****************************************************************************/ + +int spiffs_obj_lu_find_free_objid(FAR struct spiffs_s *fs, int16_t *objid, + FAR const uint8_t *conflicting_name) +{ + uint32_t max_objects = (fs->geo.neraseblocks * SPIFFS_OBJ_LOOKUP_MAX_ENTRIES(fs)) / 2; + struct spiffs_free_objid_s state; + int16_t free_objid = SPIFFS_OBJ_ID_FREE; + int ret = OK; + + state.min_objid = 1; + state.max_objid = max_objects + 1; + if (state.max_objid & SPIFFS_OBJ_ID_IX_FLAG) + { + state.max_objid = ((int16_t) - 1) & ~SPIFFS_OBJ_ID_IX_FLAG; + } + + state.compaction = 0; + state.conflicting_name = conflicting_name; + while (ret == OK && free_objid == SPIFFS_OBJ_ID_FREE) + { + if (state.max_objid - state.min_objid <= + (int16_t) SPIFFS_CFG_LOG_PAGE_SZ(fs) * 8) + { + /* possible to represent in bitmap */ + + uint32_t i, j; + finfo("free_objid: BITM min=%04x max=%04x\n", + state.min_objid, state.max_objid); + + memset(fs->work, 0, SPIFFS_CFG_LOG_PAGE_SZ(fs)); + ret = + spiffs_foreach_objlu(fs, 0, 0, 0, 0, + spiffs_obj_lu_find_free_objid_bitmap_v, + conflicting_name, + &state.min_objid, 0, 0); + if (ret == SPIFFS_VIS_END) + { + ret = OK; + } + + SPIFFS_CHECK_RES(ret); + + /* traverse bitmask until found free objid */ + + for (i = 0; i < SPIFFS_CFG_LOG_PAGE_SZ(fs); i++) + { + uint8_t mask = fs->work[i]; + if (mask == 0xff) + { + continue; + } + + for (j = 0; j < 8; j++) + { + if ((mask & (1 << j)) == 0) + { + *objid = (i << 3) + j + state.min_objid; + return OK; + } + } + } + + return -ENOSPC; + } + else + { + /* not possible to represent all ids in range in a bitmap, compact + * and count + */ + + if (state.compaction != 0) + { + /* select element in compacted table, decrease range and + * recompact + */ + + uint32_t i; + uint32_t min_i = 0; + uint8_t *map = (uint8_t *) fs->work; + uint8_t min_count = 0xff; + + for (i = 0; i < SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(uint8_t); i++) + { + if (map[i] < min_count) + { + min_count = map[i]; + min_i = i; + if (min_count == 0) + { + break; + } + } + } + + if (min_count == state.compaction) + { + /* there are no free objids! */ + + finfo("free_objid: compacted table is full\n"); + return -ENOSPC; + } + + finfo("free_objid: COMP select index:%d" + " min_count=%d min=%04x max:" + "%04x compact:%d\n", min_i, min_count, + state.min_objid, state.max_objid, state.compaction); + + if (min_count == 0) + { + /* no objid in this range, skip compacting and use directly */ + + *objid = min_i * state.compaction + state.min_objid; + return OK; + } + else + { + finfo("free_objid: COMP SEL chunk=%d min:" + "%04x -> %04x\n", state.compaction, + state.min_objid, + state.min_objid + min_i * state.compaction); + state.min_objid += min_i * state.compaction; + state.max_objid = state.min_objid + state.compaction; + + /* decrease compaction */ + } + + if ((state.max_objid - state.min_objid <= + (int16_t) SPIFFS_CFG_LOG_PAGE_SZ(fs) * 8)) + { + /* no need for compacting, use bitmap */ + + continue; + } + } + + /* in a work memory of bytes, we may fit in + * ids. NOTE: Currently page size is equivalent to block size. + * + * todo what if compaction is > 255 - then we cannot fit it in a byte + */ + + state.compaction = + (state.max_objid - + state.min_objid) / ((SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(uint8_t))); + + finfo("free_objid: COMP min=%04x max=%04x" + " compact=%d\n", state.min_objid, + state.max_objid, state.compaction); + + memset(fs->work, 0, SPIFFS_CFG_LOG_PAGE_SZ(fs)); + ret = + spiffs_foreach_objlu(fs, 0, 0, 0, 0, + spiffs_obj_lu_find_free_objid_compact_v, + &state, 0, 0, 0); + if (ret == SPIFFS_VIS_END) + { + ret = OK; + } + + SPIFFS_CHECK_RES(ret); + state.conflicting_name = 0; /* searched for conflicting name once, + * no need to do it again */ + } + } + + return ret; +} diff --git a/fs/spiffs/src/spiffs_core.h b/fs/spiffs/src/spiffs_core.h new file mode 100644 index 00000000000..9159a309850 --- /dev/null +++ b/fs/spiffs/src/spiffs_core.h @@ -0,0 +1,495 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_core.h + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * This is a port of version 0.3.7 of SPIFFS by Peter Andersion. That + * version was originally released under the MIT license but is here re- + * released under the NuttX BSD license. + * + * Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976@gmail.com) + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +#ifndef __FS_SPIFFS_SRC_SPIFFS_NUCLEIUS_H +#define __FS_SPIFFS_SRC_SPIFFS_NUCLEIUS_H + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include + +#include "spiffs.h" +#include "spiffs_mtd.h" + +/**************************************************************************** + * Pre-processor Definitions + ****************************************************************************/ + +/* SPIFFS layout + * + * spiffs is designed for following spi flash characteristics: + * - only big areas of data (blocks) can be erased + * - erasing resets all bits in a block to ones + * - writing pulls ones to zeroes + * - zeroes cannot be pulled to ones, without erase + * - wear leveling + * + * spiffs is also meant to be run on embedded, memory constraint devices. + * + * Entire area is divided in blocks. Entire area is also divided in pages. + * Each block contains same number of pages. A page cannot be erased, but a + * block can be erased. + * + * Entire area must be block_size * x + * page_size must be block_size / (2^y) where y > 2 + * + * ex: area = 1024*1024 bytes, block size = 65536 bytes, page size = 256 bytes + * + * BLOCK 0 PAGE 0 object lookup 1 + * PAGE 1 object lookup 2 + * ... + * PAGE n-1 object lookup n + * PAGE n object data 1 + * PAGE n+1 object data 2 + * ... + * PAGE n+m-1 object data m + * + * BLOCK 1 PAGE n+m object lookup 1 + * PAGE n+m+1 object lookup 2 + * ... + * PAGE 2n+m-1 object lookup n + * PAGE 2n+m object data 1 + * PAGE 2n+m object data 2 + * ... + * PAGE 2n+2m-1 object data m + * ... + * + * n is number of object lookup pages, which is number of pages needed to index all pages + * in a block by object objid + * : block_size / page_size * sizeof(objid) / page_size + * m is number data pages, which is number of pages in block minus number of lookup pages + * : block_size / page_size - block_size / page_size * sizeof(objid) / page_size + * thus, n+m is total number of pages in a block + * : block_size / page_size + * + * ex: n = 65536/256*2/256 = 2, m = 65536/256 - 2 = 254 => n+m = 65536/256 = 256 + * + * Object lookup pages contain object objid entries. Each entry represent the corresponding + * data page. + * Assuming a 16 bit object objid, an object objid being 0xffff represents a free page. + * An object objid being 0x0000 represents a deleted page. + * + * ex: page 0 : lookup : 0008 0001 0aaa ffff ffff ffff ffff ffff .. + * page 1 : lookup : ffff ffff ffff ffff ffff ffff ffff ffff .. + * page 2 : data : data for object objid 0008 + * page 3 : data : data for object objid 0001 + * page 4 : data : data for object objid 0aaa + * ... + * + * + * Object data pages can be either object index pages or object content. + * All object data pages contains a data page header, containing object objid and span index. + * The span index denotes the object page ordering amongst data pages with same object objid. + * This applies to both object index pages (when index spans more than one page of entries), + * and object data pages. + * An object index page contains page entries pointing to object content page. The entry index + * in a object index page correlates to the span index in the actual object data page. + * The first object index page (span index 0) is called object index header page, and also + * contains object flags (directory/file), size, object name etc. + * + * ex: + * BLOCK 1 + * PAGE 256: object lookup page 1 + * [*123] [ 123] [ 123] [ 123] + * [ 123] [*123] [ 123] [ 123] + * [free] [free] [free] [free] ... + * PAGE 257: object lookup page 2 + * [free] [free] [free] [free] ... + * PAGE 258: object index page (header) + * obj.objid:0123 span.ndx:0000 flags:INDEX + * size:1600 name:ex.txt type:file + * [259] [260] [261] [262] + * PAGE 259: object data page + * obj.objid:0123 span.ndx:0000 flags:DATA + * PAGE 260: object data page + * obj.objid:0123 span.ndx:0001 flags:DATA + * PAGE 261: object data page + * obj.objid:0123 span.ndx:0002 flags:DATA + * PAGE 262: object data page + * obj.objid:0123 span.ndx:0003 flags:DATA + * PAGE 263: object index page + * obj.objid:0123 span.ndx:0001 flags:INDEX + * [264] [265] [fre] [fre] + * [fre] [fre] [fre] [fre] + * PAGE 264: object data page + * obj.objid:0123 span.ndx:0004 flags:DATA + * PAGE 265: object data page + * obj.objid:0123 span.ndx:0005 flags:DATA + */ + +/* visitor result, continue searching */ + +#define SPIFFS_VIS_COUNTINUE (SPIFFS_ERR_INTERNAL - 20) + +/* visitor result, continue searching after reloading lu buffer */ + +#define SPIFFS_VIS_COUNTINUE_RELOAD (SPIFFS_ERR_INTERNAL - 21) + +/* visitor result, stop searching */ + +#define SPIFFS_VIS_END (SPIFFS_ERR_INTERNAL - 22) + +/* updating an object index contents */ + +#define SPIFFS_EV_IX_UPD (0) + +/* creating a new object index */ + +#define SPIFFS_EV_IX_NEW (1) + +/* deleting an object index */ + +#define SPIFFS_EV_IX_DEL (2) + +/* moving an object index without updating contents */ + +#define SPIFFS_EV_IX_MOV (3) + +/* updating an object index header data only, not the table itself */ + +#define SPIFFS_EV_IX_UPD_HDR (4) + +#define SPIFFS_OBJ_ID_IX_FLAG ((int16_t)(1<<(8*sizeof(int16_t)-1))) + +#define SPIFFS_UNDEFINED_LEN (uint32_t)(-1) + +#define SPIFFS_OBJ_ID_DELETED ((int16_t)0) +#define SPIFFS_OBJ_ID_FREE ((int16_t)-1) + +/* total number of pages */ + +#define SPIFFS_MAX_PAGES(fs) \ + (SPIFFS_CFG_PHYS_SZ(fs)/SPIFFS_CFG_LOG_PAGE_SZ(fs)) + +/* total number of pages per block, including object lookup pages */ + +#define SPIFFS_PAGES_PER_BLOCK(fs) \ + (SPIFFS_CFG_LOG_BLOCK_SZ(fs)/SPIFFS_CFG_LOG_PAGE_SZ(fs)) + +/* number of object lookup pages per block */ + +#define SPIFFS_OBJ_LOOKUP_PAGES(fs) \ + (MAX(1, (SPIFFS_PAGES_PER_BLOCK(fs) * sizeof(int16_t)) / SPIFFS_CFG_LOG_PAGE_SZ(fs))) + +/* checks if page index belongs to object lookup */ + +#define SPIFFS_IS_LOOKUP_PAGE(fs,pgndx) \ + (((pgndx) % SPIFFS_PAGES_PER_BLOCK(fs)) < SPIFFS_OBJ_LOOKUP_PAGES(fs)) + +/* number of object lookup entries in all object lookup pages */ + +#define SPIFFS_OBJ_LOOKUP_MAX_ENTRIES(fs) \ + (SPIFFS_PAGES_PER_BLOCK(fs)-SPIFFS_OBJ_LOOKUP_PAGES(fs)) + +/* converts a block to physical address */ + +#define SPIFFS_BLOCK_TO_PADDR(fs, block) ((block) * SPIFFS_CFG_LOG_BLOCK_SZ(fs)) + +/* converts a object lookup entry to page index */ + +#define SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, block, entry) \ + ((block)*SPIFFS_PAGES_PER_BLOCK(fs) + (SPIFFS_OBJ_LOOKUP_PAGES(fs) + entry)) + +/* converts a object lookup entry to physical address of corresponding page */ + +#define SPIFFS_OBJ_LOOKUP_ENTRY_TO_PADDR(fs, block, entry) \ + (SPIFFS_BLOCK_TO_PADDR(fs, block) + (SPIFFS_OBJ_LOOKUP_PAGES(fs) + entry) * SPIFFS_CFG_LOG_PAGE_SZ(fs)) + +/* converts a page to physical address */ + +#define SPIFFS_PAGE_TO_PADDR(fs, page) ((page) * SPIFFS_CFG_LOG_PAGE_SZ(fs)) + +/* converts a physical address to page */ + +#define SPIFFS_PADDR_TO_PAGE(fs, addr) ((addr) / SPIFFS_CFG_LOG_PAGE_SZ(fs)) + +/* gives index in page for a physical address */ + +#define SPIFFS_PADDR_TO_PAGE_OFFSET(fs, addr) ((addr) % SPIFFS_CFG_LOG_PAGE_SZ(fs)) + +/* returns containing block for given page */ + +#define SPIFFS_BLOCK_FOR_PAGE(fs, page) ((page) / SPIFFS_PAGES_PER_BLOCK(fs)) + +/* returns starting page for block */ + +#define SPIFFS_PAGE_FOR_BLOCK(fs, block) ((block) * SPIFFS_PAGES_PER_BLOCK(fs)) + +/* converts page to entry in object lookup page */ + +#define SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(fs, page) \ + ((page) % SPIFFS_PAGES_PER_BLOCK(fs) - SPIFFS_OBJ_LOOKUP_PAGES(fs)) + +/* returns data size in a data page */ + +#define SPIFFS_DATA_PAGE_SIZE(fs) \ + (SPIFFS_CFG_LOG_PAGE_SZ(fs) - sizeof(struct spiffs_page_header_s)) + +/* returns physical address for block's erase count, + * always in the physical last entry of the last object lookup page + */ + +#define SPIFFS_ERASE_COUNT_PADDR(fs, blkndx) \ + (SPIFFS_BLOCK_TO_PADDR(fs, blkndx) + SPIFFS_OBJ_LOOKUP_PAGES(fs) * SPIFFS_CFG_LOG_PAGE_SZ(fs) - sizeof(int16_t)) + +/* checks if there is any room for magic in the object luts */ + +#define SPIFFS_CHECK_MAGIC_POSSIBLE(fs) \ + ((SPIFFS_OBJ_LOOKUP_MAX_ENTRIES(fs) % (SPIFFS_CFG_LOG_PAGE_SZ(fs)/sizeof(int16_t))) * sizeof(int16_t) \ + <= (SPIFFS_CFG_LOG_PAGE_SZ(fs)-sizeof(int16_t)*2)) + +/* define helpers object */ + +/* entries in an object header page index */ + +#define SPIFFS_OBJ_HDR_IX_LEN(fs) \ + ((SPIFFS_CFG_LOG_PAGE_SZ(fs) - sizeof(struct spiffs_pgobj_ixheader_s))/sizeof(int16_t)) + +/* entries in an object page index */ + +#define SPIFFS_OBJ_IX_LEN(fs) \ + ((SPIFFS_CFG_LOG_PAGE_SZ(fs) - sizeof(spiffs_page_object_ix))/sizeof(int16_t)) + +/* object index entry for given data span index */ + +#define SPIFFS_OBJ_IX_ENTRY(fs, spndx) \ + ((spndx) < SPIFFS_OBJ_HDR_IX_LEN(fs) ? (spndx) : (((spndx)-SPIFFS_OBJ_HDR_IX_LEN(fs))%SPIFFS_OBJ_IX_LEN(fs))) + +/* object index span index number for given data span index or entry */ + +#define SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, spndx) \ + ((spndx) < SPIFFS_OBJ_HDR_IX_LEN(fs) ? 0 : (1+((spndx)-SPIFFS_OBJ_HDR_IX_LEN(fs))/SPIFFS_OBJ_IX_LEN(fs))) + +/* get data span index for object index span index */ + +#define SPIFFS_DATA_SPAN_IX_FOR_OBJ_IX_SPAN_IX(fs, spndx) \ + ((spndx) == 0 ? 0 : (SPIFFS_OBJ_HDR_IX_LEN(fs) + (((spndx)-1) * SPIFFS_OBJ_IX_LEN(fs)))) + +#define SPIFFS_OP_T_OBJ_LU (0 << 0) +#define SPIFFS_OP_T_OBJ_LU2 (1 << 0) +#define SPIFFS_OP_T_OBJ_IX (2 << 0) +#define SPIFFS_OP_T_OBJ_DA (3 << 0) +#define SPIFFS_OP_C_DELE (0 << 2) +#define SPIFFS_OP_C_UPDT (1 << 2) +#define SPIFFS_OP_C_MOVS (2 << 2) +#define SPIFFS_OP_C_MOVD (3 << 2) +#define SPIFFS_OP_C_FLSH (4 << 2) +#define SPIFFS_OP_C_READ (5 << 2) +#define SPIFFS_OP_C_WRTHRU (6 << 2) + +#define SPIFFS_OP_TYPE_MASK (3 << 0) +#define SPIFFS_OP_COM_MASK (7 << 2) + +/* if 0, this page is written to, else clean */ + +#define SPIFFS_PH_FLAG_USED (1<<0) + +/* if 0, writing is finalized, else under modification */ + +#define SPIFFS_PH_FLAG_FINAL (1<<1) + +/* if 0, this is an index page, else a data page */ + +#define SPIFFS_PH_FLAG_INDEX (1<<2) + +/* if 0, page is deleted, else valid */ + +#define SPIFFS_PH_FLAG_DELET (1<<7) + +/* if 0, this index header is being deleted */ + +#define SPIFFS_PH_FLAG_IXDELE (1<<6) + +#define SPIFFS_CHECK_RES(res) \ + do { \ + if ((res) < OK) return (res); \ + } while (0); + +#define SPIFFS_VALIDATE_OBJIX(_ph, _objid, _spndx) \ + if (((_ph).flags & SPIFFS_PH_FLAG_USED) != 0) return SPIFFS_ERR_IS_FREE; \ + if (((_ph).flags & SPIFFS_PH_FLAG_DELET) == 0) return SPIFFS_ERR_DELETED; \ + if (((_ph).flags & SPIFFS_PH_FLAG_FINAL) != 0) return SPIFFS_ERR_NOT_FINALIZED; \ + if (((_ph).flags & SPIFFS_PH_FLAG_INDEX) != 0) return SPIFFS_ERR_NOT_INDEX; \ + if (((_objid) & SPIFFS_OBJ_ID_IX_FLAG) == 0) return SPIFFS_ERR_NOT_INDEX; \ + if ((_ph).spndx != (_spndx)) return SPIFFS_ERR_INDEX_SPAN_MISMATCH; + /* if ((_spndx) == 0 && ((_ph).flags & SPIFFS_PH_FLAG_IXDELE) == 0) return + * SPIFFS_ERR_DELETED; + */ + +#define SPIFFS_VALIDATE_DATA(_ph, _objid, _spndx) \ + if (((_ph).flags & SPIFFS_PH_FLAG_USED) != 0) return SPIFFS_ERR_IS_FREE; \ + if (((_ph).flags & SPIFFS_PH_FLAG_DELET) == 0) return SPIFFS_ERR_DELETED; \ + if (((_ph).flags & SPIFFS_PH_FLAG_FINAL) != 0) return SPIFFS_ERR_NOT_FINALIZED; \ + if (((_ph).flags & SPIFFS_PH_FLAG_INDEX) == 0) return SPIFFS_ERR_IS_INDEX; \ + if ((_objid) & SPIFFS_OBJ_ID_IX_FLAG) return SPIFFS_ERR_IS_INDEX; \ + if ((_ph).spndx != (_spndx)) return SPIFFS_ERR_DATA_SPAN_MISMATCH; + +/* check objid, only visit matching objec ids */ + +#define SPIFFS_VIS_CHECK_ID (1<<0) + +/* report argument object objid to visitor - else object lookup objid is reported */ + +#define SPIFFS_VIS_CHECK_PH (1<<1) + +/* Stop searching at end of all look up pages */ + +#define SPIFFS_VIS_NO_WRAP (1<<2) + +#ifndef MIN +# define MIN(a,b) ((a) < (b) ? (a) : (b)) +#endif +#ifndef MAX +# define MAX(a,b) ((a) > (b) ? (a) : (b)) +#endif + +/**************************************************************************** + * Public Types + ****************************************************************************/ + +/* object structs */ + +/* page header, part of each page except object lookup pages + * NB: this is always aligned when the data page is an object index, + * as in this case struct spiffs_page_object_ix is used + */ + +begin_packed_struct struct spiffs_page_header_s +{ + int16_t objid; /* object objid */ + int16_t spndx; /* object span index */ + uint8_t flags; /* flags */ +} end_packed_struct; + +/* object index header page header */ + +begin_packed_struct struct spiffs_pgobj_ixheader_s +{ + struct spiffs_page_header_s p_hdr; /* common page header */ + uint32_t size; /* size of object */ + uint8_t type; /* type of object */ + uint8_t name[CONFIG_SPIFFS_NAME_MAX]; /* name of object */ +} end_packed_struct; + +/* object index page header */ + +typedef begin_packed_struct struct +{ + struct spiffs_page_header_s p_hdr; + uint8_t _align[4 - ((sizeof(struct spiffs_page_header_s) & 3) == + 0 ? 4 : (sizeof(struct spiffs_page_header_s) & 3))]; +} begin_packed_struct spiffs_page_object_ix; + +/* callback func for object lookup visitor */ + +typedef int (*spiffs_visitor_f)(FAR struct spiffs_s *fs, int16_t objid, + int16_t blkndx, int ix_entry, + FAR const void *user_const, + FAR void *user_var); + +/**************************************************************************** + * Public Function Prototypes + ****************************************************************************/ + +int spiffs_phys_cpy(FAR struct spiffs_s *fs, + int16_t objid, uint32_t dst, uint32_t src, uint32_t len); +int spiffs_foreach_objlu(FAR struct spiffs_s *fs, int16_t starting_block, + int starting_lu_entry, uint8_t flags, int16_t objid, + spiffs_visitor_f v, FAR const void *user_const, + FAR void *user_var, FAR int16_t *block_ix, int *lu_entry); +int spiffs_erase_block(FAR struct spiffs_s *fs, int16_t blkndx); +int spiffs_obj_lu_scan(FAR struct spiffs_s *fs); +int spiffs_obj_lu_find_free_obj_id(FAR struct spiffs_s *fs, + int16_t *objid, FAR const uint8_t *conflicting_name); +int spiffs_obj_lu_find_free(FAR struct spiffs_s *fs, + int16_t starting_block, int starting_lu_entry, + FAR int16_t *block_ix, FAR int *lu_entry); +int spiffs_obj_lu_find_id(FAR struct spiffs_s *fs, + int16_t starting_block, int starting_lu_entry, int16_t objid, + FAR int16_t *block_ix, FAR int *lu_entry); +int spiffs_obj_lu_find_id_and_span(FAR struct spiffs_s *fs, + int16_t objid, int16_t spndx, int16_t exclusion_pgndx, + FAR int16_t *pgndx); +int spiffs_obj_lu_find_id_and_span_byphdr(FAR struct spiffs_s *fs, + int16_t objid, int16_t spndx, int16_t exclusion_pgndx, + FAR int16_t *pgndx); +int spiffs_page_allocate_data(FAR struct spiffs_s *fs, + int16_t objid, FAR struct spiffs_page_header_s *ph, + FAR uint8_t *data, uint32_t len, uint32_t page_offs, + bool finalize, FAR int16_t *pgndx); +int spiffs_page_move(FAR struct spiffs_s *fs, + int16_t objid, FAR uint8_t *page_data, int16_t ndx, + FAR struct spiffs_page_header_s *page_hdr, int16_t src_pgndx, + FAR int16_t *dst_pgndx); +int spiffs_page_delete(FAR struct spiffs_s *fs, int16_t pgndx); +int spiffs_object_create(FAR struct spiffs_s *fs, + int16_t objid, const uint8_t name[], uint8_t type, + FAR int16_t *objhdr_pgndx); +int spiffs_object_update_index_hdr(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, int16_t objid, int16_t objhdr_pgndx, + FAR uint8_t *new_objhdr_data, const uint8_t name[], + uint32_t size, FAR int16_t *new_pgndx); +void spiffs_cb_object_event(FAR struct spiffs_s *fs, + FAR spiffs_page_object_ix * objndx, int ev, int16_t objid, + int16_t spndx, int16_t new_pgndx, uint32_t new_size); +int spiffs_object_open_bypage(FAR struct spiffs_s *fs, + int16_t pgndx, FAR struct spiffs_file_s *f, uint16_t flags, + uint16_t mode); +int spiffs_object_append(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, off_t offset, FAR uint8_t *data, + size_t len); +ssize_t spiffs_object_read(FAR struct spiffs_s *fs, FAR + FAR struct spiffs_file_s *fobj, off_t offset, size_t len, + FAR uint8_t *dst); +int spiffs_object_truncate(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, off_t new_size, bool remove_full); +int spiffs_object_modify(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, off_t offset, FAR uint8_t *data, + size_t len); +int spiffs_find_objhdr_pgndx(FAR struct spiffs_s *fs, + const uint8_t name[CONFIG_SPIFFS_NAME_MAX], FAR int16_t *pgndx); + +#endif /* __FS_SPIFFS_SRC_SPIFFS_NUCLEIUS_H */ diff --git a/fs/spiffs/src/spiffs_gc.c b/fs/spiffs/src/spiffs_gc.c new file mode 100644 index 00000000000..6f00f84c6fd --- /dev/null +++ b/fs/spiffs/src/spiffs_gc.c @@ -0,0 +1,1223 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_gc.c + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * This is a port of version 0.3.7 of SPIFFS by Peter Andersion. That + * version was originally released under the MIT license but is here re- + * released under the NuttX BSD license. + * + * Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976@gmail.com) + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include +#include +#include + +#include "spiffs.h" +#include "spiffs_core.h" +#include "spiffs_cache.h" +#include "spiffs_gc.h" + +/**************************************************************************** + * Private Types + ****************************************************************************/ + +enum spiffs_gc_clean_state_e +{ + FIND_OBJ_DATA, + MOVE_OBJ_DATA, + MOVE_OBJ_IX, + FINISHED +}; + +struct spiffs_gc_s +{ + enum spiffs_gc_clean_state_e state; + int16_t cur_objid; + int16_t cur_objndx_spndx; + int16_t cur_objndx_pgndx; + int16_t cur_data_pgndx; + int stored_scan_entry_index; + bool objid_found; +}; + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_gc_erase_block + * + * Description: + * Erases a logical block and updates the erase counter. + * If cache is enabled, all pages that might be cached in this block + * is dropped. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * blkndx - The block index to erase + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + + +static int spiffs_gc_erase_block(FAR struct spiffs_s *fs, int16_t blkndx) +{ + int ret; + int i; + + spiffs_gcinfo("Erase block=%04x\n", blkndx); + + /* Erase the block */ + ret = spiffs_erase_block(fs, blkndx); + if (ret < 0) + { + ferr("ERROR: spiffs_erase_block() blkndx=%d failed: %d\n", blkndx, ret); + } + + /* Then remove the pages from the cache. */ + + for (i = 0; i < SPIFFS_PAGES_PER_BLOCK(fs); i++) + { + spiffs_cache_drop_page(fs, SPIFFS_PAGE_FOR_BLOCK(fs, blkndx) + i); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_gc_epage_stats + * + * Description: + * Checks if garbage collecting is necessary. If so a candidate block is + * found, cleansed and erased + * + * This function will try to make room for given amount of bytes in the + * filesystem by moving pages and erasing blocks. + * + * If it is physically impossible, err_no will be set to -ENOSPC. If + * there already is this amount (or more) of free space, SPIFFS_gc will + * silently return. It is recommended to call statfs() before invoking + * this method in order to determine what amount of bytes to give. + * + * NB: the garbage collector is automatically called when spiffs needs free + * pages. The reason for this function is to give possibility to do background + * tidying when user knows the system is idle. + * + * Input Parameters: + * fs the file system struct + * len amount of data that should be freed + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. -ENODATA is returned if no blocks were deleted. + * + ****************************************************************************/ + +/* Updates page statistics for a block that is about to be erased */ + +static int spiffs_gc_epage_stats(FAR struct spiffs_s *fs, int16_t blkndx) +{ + FAR int16_t *obj_lu_buf = (int16_t *) fs->lu_work; + uint32_t dele = 0; + uint32_t allo = 0; + int entries_per_page = (SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(int16_t)); + int cur_entry = 0; + int obj_lookup_page = 0; + int ret = OK; + + /* Check each object lookup page */ + + while (ret == OK && obj_lookup_page < (int)SPIFFS_OBJ_LOOKUP_PAGES(fs)) + { + int entry_offset = obj_lookup_page * entries_per_page; + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, 0, + blkndx * SPIFFS_CFG_LOG_BLOCK_SZ(fs) + + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work); + + /* Check each entry */ + + while (ret == OK && + cur_entry - entry_offset < entries_per_page && + cur_entry < + (int)(SPIFFS_PAGES_PER_BLOCK(fs) - SPIFFS_OBJ_LOOKUP_PAGES(fs))) + { + int16_t id = obj_lu_buf[cur_entry - entry_offset]; + + if (id == SPIFFS_OBJ_ID_FREE) + { + } + else if (id == SPIFFS_OBJ_ID_DELETED) + { + dele++; + } + else + { + allo++; + } + + cur_entry++; + } + + obj_lookup_page++; + } + + spiffs_gcinfo("Wipe pallo=%d pdele=%d\n", allo, dele); + + fs->stats_p_allocated -= allo; + fs->stats_p_deleted -= dele; + return ret; +} + +/**************************************************************************** + * Name: spiffs_gc_find_candidate + * + * Description: + * Finds block candidates to erase + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * block_candidates - Memory to return an array of candidates + * candidate_count - Memory to return the number of candidates + * fs_crammed - True: Filesystem is full + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. -ENODATA is returned if no blocks were deleted. + * + ****************************************************************************/ + +static int spiffs_gc_find_candidate(FAR struct spiffs_s *fs, + FAR int16_t **block_candidates, + FAR int *candidate_count, bool fs_crammed) +{ + FAR int32_t *cand_scores; + FAR int16_t *obj_lu_buf = (FAR int16_t *)fs->lu_work; + FAR int16_t *cand_blocks; + uint32_t blocks = fs->geo.neraseblocks; + int16_t cur_block = 0; + uint32_t cur_block_addr = 0; + int entries_per_page; + int max_candidates; + int cur_entry = 0; + int ret = OK; + + /* Using fs->work area as sorted candidate memory, + * (int16_t)cand_blkndx/(int32_t)score + */ + + max_candidates = + MIN(fs->geo.neraseblocks, + (SPIFFS_CFG_LOG_PAGE_SZ(fs) - 8) / + (sizeof(int16_t) + sizeof(int32_t))); + *candidate_count = 0; + memset(fs->work, 0xff, SPIFFS_CFG_LOG_PAGE_SZ(fs)); + + /* Divide up work area into block indices and scores */ + + cand_blocks = (FAR int16_t *)fs->work; + cand_scores = + (FAR int32_t *)(fs->work + max_candidates * sizeof(int16_t)); + + /* Align cand_scores on int32_t boundary */ + + cand_scores = + (FAR int32_t *)(((intptr_t)cand_scores + sizeof(intptr_t) - 1) & + ~(sizeof(intptr_t) - 1)); + + *block_candidates = cand_blocks; + + entries_per_page = (SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(int16_t)); + + /* Check each block */ + + while (ret == OK && blocks--) + { + uint16_t deleted_pages_in_block = 0; + uint16_t used_pages_in_block = 0; + int obj_lookup_page = 0; + + /* check each object lookup page */ + + while (ret == OK && + obj_lookup_page < (int)SPIFFS_OBJ_LOOKUP_PAGES(fs)) + { + int entry_offset = obj_lookup_page * entries_per_page; + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, 0, + cur_block_addr + + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work); + + /* Check each entry */ + + while (ret == OK && + cur_entry - entry_offset < entries_per_page && + cur_entry < + (int)(SPIFFS_PAGES_PER_BLOCK(fs) - + SPIFFS_OBJ_LOOKUP_PAGES(fs))) + { + int16_t id = obj_lu_buf[cur_entry - entry_offset]; + + if (id == SPIFFS_OBJ_ID_FREE) + { + /* When a free entry is encountered, scan logic ensures that + * all following entries are free also + */ + + ret = 1; /* Kill object lu loop */ + break; + } + else if (id == SPIFFS_OBJ_ID_DELETED) + { + deleted_pages_in_block++; + } + else + { + used_pages_in_block++; + } + + cur_entry++; + } + + obj_lookup_page++; + } + + if (ret == 1) + { + ret = OK; + } + + /* Calculate score and insert into candidate table stoneage sort, but + * probably not so many blocks + */ + + if (ret == OK /* && deleted_pages_in_block > 0 */ ) + { + int32_t score; + int16_t erase_count; + int16_t erase_age; + int candndx; + + /* Read erase count */ + + ret = spiffs_cache_read(fs, SPIFFS_OP_C_READ | SPIFFS_OP_T_OBJ_LU2, 0, + SPIFFS_ERASE_COUNT_PADDR(fs, cur_block), + sizeof(int16_t), (uint8_t *) & erase_count); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + /* Calculate the erase age */ + + if (fs->max_erase_count > erase_count) + { + erase_age = fs->max_erase_count - erase_count; + } + else + { + erase_age = + SPIFFS_OBJ_ID_FREE - (erase_count - fs->max_erase_count); + } + + score = + deleted_pages_in_block * CONFIG_SPIFFS_GC_DELWGT + + used_pages_in_block * CONFIG_SPIFFS_GC_USEDWGT + + erase_age * (fs_crammed ? 0 : CONFIG_SPIFFS_GC_ERASEAGEWGT); + + candndx = 0; + + spiffs_gcinfo("blkndx=%04x del=%d use=%d score=%d\n", + cur_block, deleted_pages_in_block, + used_pages_in_block, score); + + while (candndx < max_candidates) + { + if (cand_blocks[candndx] == (int16_t) - 1) + { + cand_blocks[candndx] = cur_block; + cand_scores[candndx] = score; + break; + } + else if (cand_scores[candndx] < score) + { + int reorder_candndx = max_candidates - 2; + while (reorder_candndx >= candndx) + { + cand_blocks[reorder_candndx + 1] = + cand_blocks[reorder_candndx]; + cand_scores[reorder_candndx + 1] = + cand_scores[reorder_candndx]; + reorder_candndx--; + } + + cand_blocks[candndx] = cur_block; + cand_scores[candndx] = score; + break; + } + + candndx++; + } + + (*candidate_count)++; + } + + cur_entry = 0; + cur_block++; + cur_block_addr += SPIFFS_CFG_LOG_BLOCK_SZ(fs); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_gc_clean + * + * Description: + * Empties given block by moving all data into free pages of another block + * Strategy: + * loop: + * - Scan object lookup for object data pages. + * - For first found id, check spndx and load corresponding object index + * page to memory. + * - Push object scan lookup entry index. + * Rescan object lookup, find data pages with same id and referenced + * by same object index. + * Move data page, update object index in memory. + * When reached end of lookup, store updated object index. + * - Pop object scan lookup entry index. + * - Repeat loop until end of object lookup. + * - Scan object lookup again for remaining object index pages, move to + * new page in other block. + * + * Input Parameters: + * fs the file system struct + * len amount of data that should be freed + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. -ENODATA is returned if no blocks were deleted. + * + ****************************************************************************/ + +static int spiffs_gc_clean(FAR struct spiffs_s *fs, int16_t blkndx) +{ + const int entries_per_page = (SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(int16_t)); + int ret = OK; + + /* This is the global localizer being pushed and popped */ + + int cur_entry = 0; + FAR int16_t *obj_lu_buf = (FAR int16_t *)fs->lu_work; + struct spiffs_gc_s gc; /* Our stack frame/state */ + int16_t cur_pgndx = 0; + FAR struct spiffs_pgobj_ixheader_s *objhdr = + (FAR struct spiffs_pgobj_ixheader_s *) fs->work; + spiffs_page_object_ix *objndx = (spiffs_page_object_ix *)fs->work; + + spiffs_gcinfo("Cleaning block %04x\n", blkndx); + + memset(&gc, 0, sizeof(struct spiffs_gc_s)); + gc.state = FIND_OBJ_DATA; + + if (fs->free_blkndx == blkndx) + { + /* Move free cursor to next block, cannot use free pages from the block + * we want to clean + */ + + fs->free_blkndx = (blkndx + 1) % fs->geo.neraseblocks; + fs->free_entry = 0; + spiffs_gcinfo("Move free cursor to block=%0rx\n", fs->free_blkndx); + } + + while (ret == OK && gc.state != FINISHED) + { + int obj_lookup_page; + uint8_t scan; + + spiffs_gcinfo("gc_clean: state=%d entry=%d\n", gc.state, cur_entry); + + gc.objid_found = false; /* Reset (to no found data page) */ + + /* Scan through lookup pages */ + + obj_lookup_page = cur_entry / entries_per_page; + scan = 1; + + /* Check each object lookup page */ + + while (scan && ret == OK && + obj_lookup_page < (int)SPIFFS_OBJ_LOOKUP_PAGES(fs)) + { + int entry_offset = obj_lookup_page * entries_per_page; + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, 0, + blkndx * SPIFFS_CFG_LOG_BLOCK_SZ(fs) + + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work); + + /* Check each object lookup entry */ + + while (scan && ret == OK && + cur_entry - entry_offset < entries_per_page && + cur_entry < + (int)(SPIFFS_PAGES_PER_BLOCK(fs) - + SPIFFS_OBJ_LOOKUP_PAGES(fs))) + { + int16_t id = obj_lu_buf[cur_entry - entry_offset]; + + cur_pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, cur_entry); + + /* Act upon object id depending on gc state */ + + switch (gc.state) + { + case FIND_OBJ_DATA: + /* Find a data page. */ + + if (id != SPIFFS_OBJ_ID_DELETED && + id != SPIFFS_OBJ_ID_FREE && + ((id & SPIFFS_OBJ_ID_IX_FLAG) == 0)) + { + /* Found a data page, stop scanning and handle in switch + * case below + */ + + spiffs_gcinfo("Found data page, state=%d, objid=%04x\n", + gc.state, id); + + gc.objid_found = true; + gc.cur_objid = id; + gc.cur_data_pgndx = cur_pgndx; + scan = 0; + } + break; + + case MOVE_OBJ_DATA: + /* Evacuate found data pages for corresponding object index + * we have in memory, update memory representation + */ + + if (id == gc.cur_objid) + { + struct spiffs_page_header_s p_hdr; + + ret = spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx), + sizeof(struct spiffs_page_header_s), + (FAR uint8_t *)&p_hdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + spiffs_gcinfo("Found data page %04x:%04x @%04x\n", + gc.cur_objid, p_hdr.spndx, cur_pgndx); + + if (SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, p_hdr.spndx) != + gc.cur_objndx_spndx) + { + spiffs_gcinfo("No objndx spndx match, take in another run\n"); + } + else + { + int16_t new_data_pgndx; + if (p_hdr.flags & SPIFFS_PH_FLAG_DELET) + { + /* Move page */ + + ret = spiffs_page_move(fs, 0, 0, id, &p_hdr, + cur_pgndx, &new_data_pgndx); + + spiffs_gcinfo("Move objndx=%04x:%04x page=%04x to %04x\n", + gc.cur_objid, p_hdr.spndx, + cur_pgndx, new_data_pgndx); + + if (ret < 0) + { + ferr("ERROR: spiffs_page_move() failed: %d\n", ret); + return ret; + } + + /* Move wipes obj_lu, reload it */ + + ret = spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, + 0, + blkndx * SPIFFS_CFG_LOG_BLOCK_SZ(fs) + + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page), + SPIFFS_CFG_LOG_PAGE_SZ(fs), + fs->lu_work); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + } + else + { + /* Page is deleted but not deleted in lookup. + * Scrap it - might seem unnecessary as we will + * erase this block, but we might get aborted + */ + + spiffs_gcinfo("Wipe objndx=%04x:%04x page=%04x\n", + id, p_hdr.spndx, cur_pgndx); + + ret = spiffs_page_delete(fs, cur_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret); + return ret; + } + + new_data_pgndx = SPIFFS_OBJ_ID_FREE; + } + + /* Update memory representation of object index page + * with new data page + */ + + if (gc.cur_objndx_spndx == 0) + { + /* Update object index header page */ + + ((FAR int16_t *)((FAR uint8_t *)objhdr + + sizeof(struct spiffs_pgobj_ixheader_s)))[p_hdr.spndx] = + new_data_pgndx; + + spiffs_gcinfo("Wrote page=%04x to objhdr entry=%04x in mem\n", + new_data_pgndx, + (int16_t)SPIFFS_OBJ_IX_ENTRY(fs, p_hdr.spndx)); + } + else + { + /* Update object index page */ + + ((FAR int16_t *)((FAR uint8_t *)objndx + + sizeof(spiffs_page_object_ix)))[SPIFFS_OBJ_IX_ENTRY(fs, p_hdr.spndx)] = + new_data_pgndx; + + spiffs_gcinfo("Wrote page=%04x to objndx entry=%04x in mem\n", + new_data_pgndx, + (int16_t)SPIFFS_OBJ_IX_ENTRY(fs, p_hdr.spndx)); + } + } + } + break; + + case MOVE_OBJ_IX: + /* Find and evacuate object index pages */ + + if (id != SPIFFS_OBJ_ID_DELETED && + id != SPIFFS_OBJ_ID_FREE && + (id & SPIFFS_OBJ_ID_IX_FLAG) != 0) + { + /* Found an index object id */ + + struct spiffs_page_header_s p_hdr; + int16_t new_pgndx; + + /* Load header */ + + ret = spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, + SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx), + sizeof(struct spiffs_page_header_s), + (FAR uint8_t *)&p_hdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + if (p_hdr.flags & SPIFFS_PH_FLAG_DELET) + { + /* Move page */ + + ret = spiffs_page_move(fs, 0, 0, id, &p_hdr, + cur_pgndx, &new_pgndx); + + spiffs_gcinfo("Move objndx=%04x:%04x page=%04x to %04x\n", + id, p_hdr.spndx, cur_pgndx, + new_pgndx); + + if (ret < 0) + { + ferr("ERROR: spiffs_page_move() failed: %d\n", ret); + return ret; + } + + spiffs_cb_object_event(fs, + (spiffs_page_object_ix *)&p_hdr, + SPIFFS_EV_IX_MOV, id, + p_hdr.spndx, new_pgndx, 0); + + /* Move wipes obj_lu, reload it */ + + ret = spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, + 0, + blkndx * SPIFFS_CFG_LOG_BLOCK_SZ(fs) + + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + } + else + { + /* Page is deleted but not deleted in lookup, scrap + * it - might seem unnecessary as we will erase this + * block, but we might get aborted + */ + + spiffs_gcinfo("Wipe objndx=%04x:%04x page=%04x\n", + id, p_hdr.spndx, cur_pgndx); + + ret = spiffs_page_delete(fs, cur_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_cb_object_event() failed: %d\n", ret); + return ret; + } + + spiffs_cb_object_event(fs, NULL, + SPIFFS_EV_IX_DEL, id, + p_hdr.spndx, + cur_pgndx, 0); + } + } + break; + + default: + scan = 0; + break; + } + + cur_entry++; + } + + obj_lookup_page++; /* No need to check scan variable here, + * obj_lookup_page is set in start of loop + */ + } + + + if (ret < 0) + { + break; + } + + /* State finalization and switch */ + + switch (gc.state) + { + case FIND_OBJ_DATA: + if (gc.objid_found) + { + struct spiffs_page_header_s p_hdr; + int16_t objndx_pgndx; + + /* Handle found data page. Find out corresponding objndx page + * and load it to memory + */ + + gc.stored_scan_entry_index = cur_entry; /* Push cursor */ + cur_entry = 0; /* Sestart scan from start */ + gc.state = MOVE_OBJ_DATA; + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, cur_pgndx), + sizeof(struct spiffs_page_header_s), + (FAR uint8_t *)&p_hdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cb_object_event() failed: %d\n", ret); + return ret; + } + + gc.cur_objndx_spndx = + SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, p_hdr.spndx); + + spiffs_gcinfo("Find objndx spndx=%04x\n", gc.cur_objndx_spndx); + + ret = spiffs_obj_lu_find_id_and_span(fs, + gc.cur_objid | SPIFFS_OBJ_ID_IX_FLAG, + gc.cur_objndx_spndx, 0, + &objndx_pgndx); + if (ret == -ENOENT) + { + /* On borked systems we might get an ERR_NOT_FOUND here - + * this is handled by simply deleting the page as it is not + * referenced from anywhere + */ + + spiffs_gcinfo("objndx not found! Wipe page %04x\n", + gc.cur_data_pgndx); + + ret = spiffs_page_delete(fs, gc.cur_data_pgndx); + if (ret < 0) + { + ferr("ERROR: spiffs_page_delete() failed: %d\n", ret); + return ret; + } + + /* Then we restore states and continue scanning for data + * pages + */ + + cur_entry = gc.stored_scan_entry_index; /* pop cursor */ + gc.state = FIND_OBJ_DATA; + break; + } + + if (ret < 0) + { + return ret; + } + + spiffs_gcinfo("Found object index at page=%04x\n", objndx_pgndx); + + ret = spiffs_cache_read(fs, + SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, + SPIFFS_PAGE_TO_PADDR(fs, objndx_pgndx), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->work); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read() failed: %d\n", ret); + return ret; + } + + /* Cannot allow a gc if the presumed index in fact is no + * index, a check must run or lot of data may be lost + */ + + SPIFFS_VALIDATE_OBJIX(objndx->p_hdr, + gc.cur_objid | SPIFFS_OBJ_ID_IX_FLAG, + gc.cur_objndx_spndx); + gc.cur_objndx_pgndx = objndx_pgndx; + } + else + { + /* No more data pages found, passed thru all block, start + * evacuating object indices + */ + + gc.state = MOVE_OBJ_IX; + cur_entry = 0; /* Restart entry scan index */ + } + break; + + case MOVE_OBJ_DATA: + { + int16_t new_objndx_pgndx; + + /* Store modified objndx (hdr) page residing in memory now that all + * data pages belonging to this object index and residing in the + * block we want to evacuate + */ + + gc.state = FIND_OBJ_DATA; + cur_entry = gc.stored_scan_entry_index; /* pop cursor */ + + if (gc.cur_objndx_spndx == 0) + { + /* Store object index header page */ + + ret = spiffs_object_update_index_hdr(fs, 0, + gc.cur_objid | SPIFFS_OBJ_ID_IX_FLAG, + gc.cur_objndx_pgndx, fs->work, 0, + 0, &new_objndx_pgndx); + + spiffs_gcinfo("Store modified objhdr page=%04x:%04x\n", + new_objndx_pgndx, 0); + + if (ret < 0) + { + ferr("ERROR: spiffs_object_update_index_hdr() failed: %d\n", ret); + return ret; + } + } + else + { + /* Store object index page */ + + ret = spiffs_page_move(fs, 0, fs->work, + gc.cur_objid | SPIFFS_OBJ_ID_IX_FLAG, + 0, gc.cur_objndx_pgndx, + &new_objndx_pgndx); + + spiffs_gcinfo("Store modified objndx page=%04x:%04x\n", + new_objndx_pgndx, objndx->p_hdr.spndx); + + if (ret < 0) + { + ferr("ERROR: spiffs_page_move() failed: %d\n", ret); + return ret; + } + + spiffs_cb_object_event(fs, + (FAR spiffs_page_object_ix *)fs->work, + SPIFFS_EV_IX_UPD, gc.cur_objid, + objndx->p_hdr.spndx, + new_objndx_pgndx, 0); + } + } + break; + + case MOVE_OBJ_IX: + /* Scanned through all blocks, no more object indices found - our + * work here is done + */ + + gc.state = FINISHED; + break; + + default: + cur_entry = 0; + break; + } + + spiffs_gcinfo("state=%d\n", gc.state); + } + + return ret; +} +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_gc_quick + * + * Description: + * Searches for blocks where all entries are deleted - if one is found, + * the block is erased. Compared to the non-quick gc, the quick one ensures + * that no updates are needed on existing objects on pages that are erased. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * max_free_pages - The maximum pages to free + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. -ENODATA is returned if no blocks were deleted. + * + ****************************************************************************/ + +int spiffs_gc_quick(FAR struct spiffs_s *fs, uint16_t max_free_pages) +{ + FAR int16_t *obj_lu_buf = (int16_t *) fs->lu_work; + uint32_t cur_block_addr = 0; + uint32_t blocks = fs->geo.neraseblocks; + int16_t cur_block = 0; + int entries_per_page = (SPIFFS_CFG_LOG_PAGE_SZ(fs) / sizeof(int16_t)); + int cur_entry = 0; + int ret = OK; + + spiffs_gcinfo("max_free_pages=%u\n", max_free_pages); +#ifdef CONFIG_SPIFFS_GCDBG + fs->stats_gc_runs++; +#endif + + /* Find fully deleted blocks */ + + while (ret == OK && blocks--) + { + uint16_t deleted_pages_in_block = 0; + uint16_t free_pages_in_block = 0; + int obj_lookup_page = 0; + + /* Check each object lookup page */ + + while (ret == OK && + obj_lookup_page < (int)SPIFFS_OBJ_LOOKUP_PAGES(fs)) + { + int entry_offset = obj_lookup_page * entries_per_page; + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, 0, + cur_block_addr + + SPIFFS_PAGE_TO_PADDR(fs, obj_lookup_page), + SPIFFS_CFG_LOG_PAGE_SZ(fs), fs->lu_work); + + /* Check each entry */ + + while (ret == OK && + cur_entry - entry_offset < entries_per_page && + cur_entry < + (int)(SPIFFS_PAGES_PER_BLOCK(fs) - + SPIFFS_OBJ_LOOKUP_PAGES(fs))) + { + int16_t id = obj_lu_buf[cur_entry - entry_offset]; + + if (id == SPIFFS_OBJ_ID_DELETED) + { + deleted_pages_in_block++; + } + else if (id == SPIFFS_OBJ_ID_FREE) + { + /* Kill scan, go for next block */ + + free_pages_in_block++; + if (free_pages_in_block > max_free_pages) + { + obj_lookup_page = SPIFFS_OBJ_LOOKUP_PAGES(fs); + ret = 1; /* Kill object lu loop */ + break; + } + } + else + { + /* Kill scan, go for next block */ + + obj_lookup_page = SPIFFS_OBJ_LOOKUP_PAGES(fs); + ret = 1; /* Kill object lu loop */ + break; + } + + cur_entry++; + } + + obj_lookup_page++; + } + + if (ret == 1) + { + ret = OK; + } + + if (ret == OK && + deleted_pages_in_block + free_pages_in_block == + SPIFFS_PAGES_PER_BLOCK(fs) - SPIFFS_OBJ_LOOKUP_PAGES(fs) && + free_pages_in_block <= max_free_pages) + { + /* Found a fully deleted block */ + + fs->stats_p_deleted -= deleted_pages_in_block; + ret = spiffs_gc_erase_block(fs, cur_block); + return ret; + } + + cur_entry = 0; + cur_block++; + cur_block_addr += SPIFFS_CFG_LOG_BLOCK_SZ(fs); + } + + if (ret == OK) + { + /* No deleted blocks */ + + ret = -ENODATA; + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_gc_check + * + * Description: + * Checks if garbage collecting is necessary. If so a candidate block is + * found, cleansed and erased + * + * This function will try to make room for given amount of bytes in the + * filesystem by moving pages and erasing blocks. + * + * If it is physically impossible, err_no will be set to -ENOSPC. If + * there already is this amount (or more) of free space, SPIFFS_gc will + * silently return. It is recommended to call statfs() before invoking + * this method in order to determine what amount of bytes to give. + * + * NB: the garbage collector is automatically called when spiffs needs free + * pages. The reason for this function is to give possibility to do background + * tidying when user knows the system is idle. + * + * Input Parameters: + * fs the file system struct + * len amount of data that should be freed + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. -ENODATA is returned if no blocks were deleted. + * + ****************************************************************************/ + +int spiffs_gc_check(FAR struct spiffs_s *fs, off_t len) +{ + int32_t free_pages = + (SPIFFS_PAGES_PER_BLOCK(fs) - + SPIFFS_OBJ_LOOKUP_PAGES(fs)) * (fs->geo.neraseblocks - 2) - + fs->stats_p_allocated - fs->stats_p_deleted; + int tries = 0; + int ret; + + if (fs->free_blocks > 3 && + (int32_t) len < free_pages * (int32_t) SPIFFS_DATA_PAGE_SIZE(fs)) + { + return OK; + } + + uint32_t needed_pages = + (len + SPIFFS_DATA_PAGE_SIZE(fs) - 1) / SPIFFS_DATA_PAGE_SIZE(fs); + +#if 0 + if (fs->free_blocks <= 2 && (int32_t)needed_pages > free_pages) + { + spiffs_gcinfo("Full freeblk=%d" needed=%d" free=%d dele=%d\n", + fs->free_blocks, needed_pages, free_pages, + fs->stats_p_deleted); + return -ENOSPC; + } +#endif + + if ((int32_t) needed_pages > (int32_t)(free_pages + fs->stats_p_deleted)) + { + spiffs_gcinfo("Full freeblk=%d needed=%d free=%d dele=%d\n", + fs->free_blocks, needed_pages, free_pages, + fs->stats_p_deleted); + return -ENOSPC; + } + + do + { + FAR int16_t *cands; + int count; + int16_t cand; + int32_t prev_free_pages = free_pages; + + spiffs_gcinfo("#%d: run gc free_blocks=%d pfree=%d pallo=%d pdele=%d [%d] len=%d of %d\n", + tries, fs->free_blocks, free_pages, + fs->stats_p_allocated, fs->stats_p_deleted, + (free_pages + fs->stats_p_allocated + fs->stats_p_deleted), + len, (uint32_t)(free_pages * SPIFFS_DATA_PAGE_SIZE(fs))); + + /* If the fs is crammed, ignore block age when selecting candidate - kind + * of a bad state + */ + + ret = spiffs_gc_find_candidate(fs, &cands, &count, free_pages <= 0); + if (ret < 0) + { + ferr("ERROR: spiffs_gc_find_candidate() failed: %d\n", ret); + return ret; + } + + if (count == 0) + { + spiffs_gcinfo("No candidates, return\n"); + return (int32_t) needed_pages < free_pages ? OK : -ENOSPC; + } + +#ifdef CONFIG_SPIFFS_GCDBG + fs->stats_gc_runs++; +#endif + cand = cands[0]; + + ret = spiffs_gc_clean(fs, cand); + + spiffs_gcinfo("Cleaning block %d, result=%d\n", cand, ret); + + if (ret < 0) + { + ferr("ERROR: spiffs_gc_clean() failed: %d\n", ret); + return ret; + } + + ret = spiffs_gc_epage_stats(fs, cand); + if (ret < 0) + { + ferr("ERROR: spiffs_gc_epage_stats() failed: %d\n", ret); + return ret; + } + SPIFFS_CHECK_RES(ret); + + ret = spiffs_gc_erase_block(fs, cand); + if (ret < 0) + { + ferr("ERROR: spiffs_gc_erase_block() failed: %d\n", ret); + return ret; + } + + free_pages = (SPIFFS_PAGES_PER_BLOCK(fs) - + SPIFFS_OBJ_LOOKUP_PAGES(fs)) * (fs->geo.neraseblocks - 2) - + fs->stats_p_allocated - fs->stats_p_deleted; + + if (prev_free_pages <= 0 && prev_free_pages == free_pages) + { + /* Abort early to reduce wear, at least tried once */ + + spiffs_gcinfo("Early abort, no result on gc when fs crammed\n"); + break; + } + + } + while (++tries < CONFIG_SPIFFS_GC_MAXRUNS && + (fs->free_blocks <= 2 || + (int32_t) len > free_pages * (int32_t) SPIFFS_DATA_PAGE_SIZE(fs))); + + free_pages = + (SPIFFS_PAGES_PER_BLOCK(fs) - + SPIFFS_OBJ_LOOKUP_PAGES(fs)) * (fs->geo.neraseblocks - 2) - + fs->stats_p_allocated - fs->stats_p_deleted; + + if ((int32_t) len > free_pages * (int32_t) SPIFFS_DATA_PAGE_SIZE(fs)) + { + ret = -ENOSPC; + } + + spiffs_gcinfo("Finished, %d dirty, blocks, %d free, %d pages free, %d tries, ret=%d\n", + fs->stats_p_allocated + fs->stats_p_deleted, fs->free_blocks, + free_pages, tries, ret); + + return ret; +} diff --git a/fs/spiffs/src/spiffs_gc.h b/fs/spiffs/src/spiffs_gc.h new file mode 100644 index 00000000000..deebf222964 --- /dev/null +++ b/fs/spiffs/src/spiffs_gc.h @@ -0,0 +1,133 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_gc.h + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +#ifndef __FS_SPIFFS_SRC_SPIFFS_GC_H +#define __FS_SPIFFS_SRC_SPIFFS_GC_H + +#if defined(__cplusplus) +extern "C" +{ +#endif + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include + +/**************************************************************************** + * Pre-processor Definitions + ****************************************************************************/ + +/* Debug */ + +#ifdef CONFIG_SPIFFS_GCDBG +# ifdef CONFIG_CPP_HAVE_VARARGS +# define spiffs_gcinfo(format, ...) _info(format, ##__VA_ARGS__) +# else +# define spiffs_gcinfo _info +# endif +#else +# ifdef CONFIG_CPP_HAVE_VARARGS +# define spiffs_gcinfo(format, ...) +# else +# define spiffs_gcinfo (void) +# endif +#endif + +/**************************************************************************** + * Public Function Prototypes + ****************************************************************************/ + +struct spiffs_s; /* Forward reference */ + +/**************************************************************************** + * Name: spiffs_gc_quick + * + * Description: + * Searches for blocks where all entries are deleted - if one is found, + * the block is erased. Compared to the non-quick gc, the quick one ensures + * that no updates are needed on existing objects on pages that are erased. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * max_free_pages - The maximum pages to free + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. -ENOSPC is returned if no blocks were deleted. + * + ****************************************************************************/ + +int spiffs_gc_quick(FAR struct spiffs_s *fs, uint16_t max_free_pages); + +/**************************************************************************** + * Name: spiffs_gc_check + * + * Description: + * Checks if garbage collecting is necessary. If so a candidate block is + * found, cleansed and erased + * + * This function will try to make room for given amount of bytes in the + * filesystem by moving pages and erasing blocks. + * + * If it is physically impossible, err_no will be set to -ENOSPC. If + * there already is this amount (or more) of free space, SPIFFS_gc will + * silently return. It is recommended to call statfs() before invoking + * this method in order to determine what amount of bytes to give. + * + * NB: the garbage collector is automatically called when spiffs needs free + * pages. The reason for this function is to give possibility to do background + * tidying when user knows the system is idle. + * + * Input Parameters: + * fs the file system struct + * len amount of data that should be freed + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. -ENODATA is returned if no blocks were deleted. + * + ****************************************************************************/ + +int spiffs_gc_check(FAR struct spiffs_s *fs, off_t len); + +#if defined(__cplusplus) +} +#endif + +#endif /* __FS_SPIFFS_SRC_SPIFFS_GC_H */ diff --git a/fs/spiffs/src/spiffs_mtd.c b/fs/spiffs/src/spiffs_mtd.c new file mode 100644 index 00000000000..797de03bac1 --- /dev/null +++ b/fs/spiffs/src/spiffs_mtd.c @@ -0,0 +1,189 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_mtd.c + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include +#include +#include + +#include + +#include "spiffs.h" +#include "spiffs_mtd.h" + +/**************************************************************************** + * Private Data + ****************************************************************************/ + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_mtd_write + * + * Description: + * Write data to FLASH memory + * + * Input Parameters: + * fs - A reference to the volume structure + * offset - The physical offset to write to + * len - The number of bytes to write + * src - A reference to the bytes to be written + * + * Returned Value: + * On success, the number of bytes written is returned. On failure, a + * negated errno value is returned. + * + ****************************************************************************/ + +ssize_t spiffs_mtd_write(FAR struct spiffs_s *fs, off_t offset, size_t len, + FAR const uint8_t *src) +{ + int16_t blksize; + off_t blkstart; + off_t blkend; + ssize_t ret; + + DEBUGASSERT(fs != NULL && fs->mtd != NULL && src != NULL); + +#warning REVISIT: What are units of offset and len? + +#ifdef CONFIG_MTD_BYTE_WRITE + ret = MTD_WRITE(fs->mtd, offset, len, src); + if (ret < 0) +#endif + { + blksize = fs->geo.blocksize; + DEBUGASSERT(offset = offset % blksize); + DEBUGASSERT(len = len % blksize); + + blkstart = offset / blksize; /* Truncates to floor */ + blkend = (offset + len + blksize - 1) / blksize; /* Rounds up to ceil */ + + ret = MTD_BWRITE(fs->mtd, blkstart, blkend - blkstart, src); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_mtd_read + * + * Description: + * Read data from FLASH memory + * + * Input Parameters: + * fs - A reference to the volume structure + * offset - The physical offset to read from + * len - The number of bytes to read + * dest - The user provide location to store the bytes read from FLASH. + * + * Returned Value: + * On success, the number of bytes read is returned. On failure, a + * negated errno value is returned. + * + ****************************************************************************/ + +ssize_t spiffs_mtd_read(FAR struct spiffs_s *fs, off_t offset, size_t len, + FAR uint8_t *dest) +{ + int16_t blksize; + off_t blkstart; + off_t blkend; + ssize_t ret; + + DEBUGASSERT(fs != NULL && fs->mtd != NULL && dest != NULL); + +#warning REVISIT: What are units of offset and len? + + ret = MTD_READ(fs->mtd, offset, len, dest); + if (ret < 0) + { + blksize = fs->geo.blocksize; + DEBUGASSERT(offset = offset % blksize); + DEBUGASSERT(len = len % blksize); + + blkstart = offset / blksize; /* Truncates to floor */ + blkend = (offset + len + blksize - 1) / blksize; /* Rounds up to ceil */ + + ret = MTD_BREAD(fs->mtd, blkstart, blkend - blkstart, dest); + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_mtd_erase + * + * Description: + * Read data from FLASH memory + * + * Input Parameters: + * fs - A reference to the volume structure + * offset - The physical offset to begin erasing + * len - The number of bytes to erase + * + * Returned Value: + * On success, the number of bytes erased is returned. On failure, a + * negated errno value is returned. + * + ****************************************************************************/ + +ssize_t spiffs_mtd_erase(FAR struct spiffs_s *fs, off_t offset, size_t len) +{ + int16_t blksize; + off_t blkstart; + off_t blkend; + + DEBUGASSERT(fs != NULL && fs->mtd != NULL); + +#warning REVISIT: What are units of offset and len? + + blksize = fs->geo.erasesize; + DEBUGASSERT(offset = offset % blksize); + DEBUGASSERT(len = len % blksize); + + blkstart = offset / blksize; /* Truncates to floor */ + blkend = (offset + len + blksize - 1) / blksize; /* Rounds up to ceil */ + + return MTD_ERASE(fs->mtd, blkstart, blkend - blkstart); +} diff --git a/fs/spiffs/src/spiffs_mtd.h b/fs/spiffs/src/spiffs_mtd.h new file mode 100644 index 00000000000..60213c98111 --- /dev/null +++ b/fs/spiffs/src/spiffs_mtd.h @@ -0,0 +1,131 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_mtd.h + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +#ifndef __FS_SPIFFS_SRC_SPIFFS_MTD_H +#define __FS_SPIFFS_SRC_SPIFFS_MTD_H + +#if defined(__cplusplus) +extern "C" +{ +#endif + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include + +/**************************************************************************** + * Pre-processor Definitions + ****************************************************************************/ + +#define SPIFFS_CFG_EBLOCK_COUNT(fs) ((fs)->geo.neraseblocks) +#define SPIFFS_CFG_LOG_PAGE_SZ(fs) ((fs)->geo.blocksize) +#define SPIFFS_CFG_LOG_BLOCK_SZ(fs) ((fs)->geo.blocksize) +#define SPIFFS_CFG_PHYS_SZ(fs) ((fs)->phys_size) +#define SPIFFS_CFG_PHYS_ERASE_SZ(fs) ((fs)->geo.erasesize) + +/**************************************************************************** + * Public Function Prototypes + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_mtd_write + * + * Description: + * Write data to FLASH memory + * + * Input Parameters: + * fs - A reference to the volume structure + * offset - The physical offset to write to + * len - The number of bytes to write + * src - A reference to the bytes to be written + * + * Returned Value: + * On success, the number of bytes written is returned. On failure, a + * negated errno value is returned. + * + ****************************************************************************/ + +ssize_t spiffs_mtd_write(FAR struct spiffs_s *fs, off_t offset, size_t len, + FAR const uint8_t *src); + +/**************************************************************************** + * Name: spiffs_mtd_read + * + * Description: + * Read data from FLASH memory + * + * Input Parameters: + * fs - A reference to the volume structure + * offset - The physical offset to read from + * len - The number of bytes to read + * dest - The user provide location to store the bytes read from FLASH. + * + * Returned Value: + * On success, the number of bytes read is returned. On failure, a + * negated errno value is returned. + * + ****************************************************************************/ + +ssize_t spiffs_mtd_read(FAR struct spiffs_s *fs, off_t offset, size_t len, + FAR uint8_t *dest); + +/**************************************************************************** + * Name: spiffs_mtd_erase + * + * Description: + * Read data from FLASH memory + * + * Input Parameters: + * fs - A reference to the volume structure + * offset - The physical offset to begin erasing + * len - The number of bytes to erase + * + * Returned Value: + * On success, the number of bytes erased is returned. On failure, a + * negated errno value is returned. + * + ****************************************************************************/ + +ssize_t spiffs_mtd_erase(FAR struct spiffs_s *fs, off_t offset, size_t len); + +#if defined(__cplusplus) +} +#endif + +#endif /* __FS_SPIFFS_SRC_SPIFFS_MTD_H */ diff --git a/fs/spiffs/src/spiffs_vfs.c b/fs/spiffs/src/spiffs_vfs.c new file mode 100644 index 00000000000..950a0dded9f --- /dev/null +++ b/fs/spiffs/src/spiffs_vfs.c @@ -0,0 +1,1817 @@ +/**************************************************************************** + * fs/spiffs/spiffs.c + * Interface between SPIFFS and the NuttX VFS + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * Includes logic taken from 0.3.7 of SPIFFS by Peter Andersion. That + * version was originally released under the MIT license. + * + * Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976@gmail.com) + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "spiffs.h" +#include "spiffs_core.h" +#include "spiffs_cache.h" +#include "spiffs_gc.h" +#include "spiffs_check.h" + +/**************************************************************************** + * Pre-processor Definitions + ****************************************************************************/ + +#define spiffs_lock_volume(fs) (spiffs_lock_reentrant(&fs->exclsem)) +#define spiffs_unlock_volume(fs) (spiffs_unlock_reentrant(&fs->exclsem)) + +/**************************************************************************** + * Private Function Prototypes + ****************************************************************************/ + +/* SPIFFS helpers */ + +static void spiffs_lock_reentrant(FAR struct spiffs_sem_s *sem); +static void spiffs_unlock_reentrant(FAR struct spiffs_sem_s *sem); + +/* File system operations */ + +static int spiffs_open(FAR struct file *filep, FAR const char *relpath, + int oflags, mode_t mode); +static int spiffs_close(FAR struct file *filep); +static ssize_t spiffs_read(FAR struct file *filep, FAR char *buffer, + size_t buflen); +static ssize_t spiffs_write(FAR struct file *filep, FAR const char *buffer, + size_t buflen); +static off_t spiffs_seek(FAR struct file *filep, off_t offset, int whence); +static int spiffs_ioctl(FAR struct file *filep, int cmd, unsigned long arg); + +static int spiffs_sync(FAR struct file *filep); +static int spiffs_dup(FAR const struct file *oldp, FAR struct file *newp); +static int spiffs_fstat(FAR const struct file *filep, FAR struct stat *buf); +static int spiffs_truncate(FAR struct file *filep, off_t length); + +static int spiffs_opendir(FAR struct inode *mountpt, FAR const char *relpath, + FAR struct fs_dirent_s *dir); +static int spiffs_closedir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir); +static int spiffs_readdir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir); +static int spiffs_rewinddir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir); +static int spiffs_bind(FAR struct inode *mtdinode, FAR const void *data, + FAR void **handle); +static int spiffs_unbind(FAR void *handle, FAR struct inode **mtdinode, + unsigned int flags); +static int spiffs_statfs(FAR struct inode *mountpt, FAR struct statfs *buf); +static int spiffs_unlink(FAR struct inode *mountpt, FAR const char *relpath); +static int spiffs_mkdir(FAR struct inode *mountpt, FAR const char *relpath, + mode_t mode); +static int spiffs_rmdir(FAR struct inode *mountpt, FAR const char *relpath); +static int spiffs_rename(FAR struct inode *mountpt, FAR const char *oldrelpath, + FAR const char *newrelpath); +static int spiffs_stat(FAR struct inode *mountpt, FAR const char *relpath, + FAR struct stat *buf); + +/**************************************************************************** + * Public Data + ****************************************************************************/ + +const struct mountpt_operations spiffs_operations = +{ + spiffs_open, /* open */ + spiffs_close, /* close */ + spiffs_read, /* read */ + spiffs_write, /* write */ + spiffs_seek, /* seek */ + spiffs_ioctl, /* ioctl */ + + spiffs_sync, /* sync */ + spiffs_dup, /* dup */ + spiffs_fstat, /* fstat */ + spiffs_truncate, /* truncate */ + + spiffs_opendir, /* opendir */ + spiffs_closedir, /* closedir */ + spiffs_readdir, /* readdir */ + spiffs_rewinddir, /* rewinddir */ + + spiffs_bind, /* bind */ + spiffs_unbind, /* unbind */ + spiffs_statfs, /* statfs */ + + spiffs_unlink, /* unlink */ + spiffs_mkdir, /* mkdir */ + spiffs_rmdir, /* rmdir */ + spiffs_rename, /* rename */ + spiffs_stat, /* stat */ +}; + +/**************************************************************************** + * Private Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_lock_reentrant + ****************************************************************************/ + +static void spiffs_lock_reentrant(FAR struct spiffs_sem_s *rsem) +{ + pid_t me; + + /* Do we already hold the semaphore? */ + + me = getpid(); + if (me == rsem->holder) + { + /* Yes... just increment the count */ + + rsem->count++; + DEBUGASSERT(rsem->count > 0); + } + + /* Take the semaphore (perhaps waiting) */ + + else + { + int ret; + + do + { + ret = nxsem_wait(&rsem->sem); + + /* The only case that an error should occur here is if the wait + * was awakened by a signal. + */ + + DEBUGASSERT(ret == OK || ret == -EINTR); + } + while (ret == -EINTR); + + /* No we hold the semaphore */ + + rsem->holder = me; + rsem->count = 1; + } +} + +/**************************************************************************** + * Name: spiffs_unlock_reentrant + ****************************************************************************/ + +static void spiffs_unlock_reentrant(FAR struct spiffs_sem_s *rsem) +{ + DEBUGASSERT(rsem->holder == getpid()); + + /* Is this our last count on the semaphore? */ + + if (rsem->count > 1) + { + /* No.. just decrement the count */ + + rsem->count--; + } + + /* Yes.. then we can really release the semaphore */ + + else + { + rsem->holder = SPIFFS_NO_HOLDER; + rsem->count = 0; + nxsem_post(&rsem->sem); + } +} + +/**************************************************************************** + * Name: spiffs_readdir_callback + ****************************************************************************/ + +static int spiffs_consistency_check(FAR struct spiffs_s *fs) +{ + int status; + int ret = OK; + + status = spiffs_check_luconsistency(fs); + if (status < 0) + { + fwarn("WARNING spiffs_check_luconsistency failed: %d\n", status); + if (ret == OK) + { + ret = status; + } + } + + status = spiffs_check_objidconsistency(fs); + if (status < 0) + { + fwarn("WARNING spiffs_check_objidconsistency failed: %d\n", status); + if (ret == OK) + { + ret = status; + } + } + + status = spiffs_check_pgconsistency(fs); + if (status < 0) + { + fwarn("WARNING spiffs_check_pgconsistency failed: %d\n", status); + if (ret == OK) + { + ret = status; + } + } + + status = spiffs_obj_lu_scan(fs); + if (status < 0) + { + fwarn("WARNING spiffs_obj_lu_scan failed: %d\n", status); + if (ret == OK) + { + ret = status; + } + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_readdir_callback + ****************************************************************************/ + +static int spiffs_readdir_callback(FAR struct spiffs_s *fs, + int16_t objid, int16_t blkndx, int ix_entry, + FAR const void *user_const, + FAR void *user_var) +{ + struct spiffs_pgobj_ixheader_s objhdr; + int16_t pgndx; + int ret; + + if (objid == SPIFFS_OBJ_ID_FREE || objid == SPIFFS_OBJ_ID_DELETED || + (objid & SPIFFS_OBJ_ID_IX_FLAG) == 0) + { + return SPIFFS_VIS_COUNTINUE; + } + + pgndx = SPIFFS_OBJ_LOOKUP_ENTRY_TO_PIX(fs, blkndx, ix_entry); + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU2 | SPIFFS_OP_C_READ, + 0, SPIFFS_PAGE_TO_PADDR(fs, pgndx), + sizeof(struct spiffs_pgobj_ixheader_s), + (FAR uint8_t *) & objhdr); + if (ret < 0) + { + ferr("ERROR: spiffs_cache_read failed: %d\n", ret); + return ret; + } + + if ((objid & SPIFFS_OBJ_ID_IX_FLAG) && + objhdr.p_hdr.spndx == 0 && + (objhdr.p_hdr.flags & (SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_FINAL | + SPIFFS_PH_FLAG_IXDELE)) == + (SPIFFS_PH_FLAG_DELET | SPIFFS_PH_FLAG_IXDELE)) + { + FAR struct fs_dirent_s *dir = (FAR struct fs_dirent_s *)user_var; + FAR struct dirent *entryp; + + DEBUGASSERT(dir != NULL); + entryp = &dir->fd_dir; + + strncpy(entryp->d_name, (FAR char *)objhdr.name, NAME_MAX + 1); + entryp->d_type = objhdr.type; + return OK; + } + + return SPIFFS_VIS_COUNTINUE; +} + +/**************************************************************************** + * Name: spiffs_open + ****************************************************************************/ + +static int spiffs_open(FAR struct file *filep, FAR const char *relpath, + int oflags, mode_t mode) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + off_t offset; + int16_t pgndx; + int ret; + + finfo("relpath: %s oflags; %04x\n", relpath, oflags); + DEBUGASSERT(filep->f_priv == NULL && filep->f_inode != NULL); + + /* Get the mountpoint inode reference from the file structure and the + * mountpoint private data from the inode structure + */ + + inode = filep->f_inode; + fs = inode->i_private; + + DEBUGASSERT(fs != NULL); + + /* Skip over any leading directory separators (shouldn't be any) */ + + for (; *relpath == '/'; relpath++) + { + } + + /* Check the length of the relative path */ + + if (strlen(relpath) > CONFIG_SPIFFS_NAME_MAX - 1) + { + return -ENAMETOOLONG; + } + + /* Allocate a new file object with a reference count of one. */ + + fobj = (FAR struct spiffs_file_s *)kmm_zalloc(sizeof(struct spiffs_file_s)); + if (fobj == NULL) + { + return -ENOMEM; + } + + fobj->crefs = 1; + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Check of the file object already exists */ + + ret = spiffs_find_objhdr_pgndx(fs, + (FAR const uint8_t *)relpath, + &pgndx); + if (ret < 0 && (oflags & O_CREAT) == 0) + { + /* It does not exist and we were not asked to create it */ + + goto errout_with_fileobject; + } + else if (ret == OK && (oflags & (O_CREAT | O_EXCL)) == (O_CREAT | O_EXCL)) + { + /* O_CREAT and O_EXCL and file exists - fail */ + + ret = -EEXIST; + goto errout_with_fileobject; + } + else if ((oflags & O_CREAT) != 0 && ret == -ENOENT) + { + int16_t objid; + + /* The file does not exist. We need to create the it. */ + + ret = spiffs_obj_lu_find_free_obj_id(fs, &objid, 0); + if (ret < 0) + { + goto errout_with_fileobject; + } + + ret = spiffs_object_create(fs, objid, (FAR const uint8_t *)relpath, + DTYPE_FILE, &pgndx); + if (ret < 0) + { + goto errout_with_fileobject; + } + + /* Since we created the file, we don't need to truncate it */ + + oflags &= ~O_TRUNC; + } + else if (ret < 0) + { + goto errout_with_fileobject; + } + + /* Open the file */ + + ret = spiffs_object_open_bypage(fs, pgndx, fobj, oflags, mode); + if (ret < 0) + { + goto errout_with_fileobject; + } + + /* Truncate the file to zero length */ + + if ((oflags & O_TRUNC) != 0) + { + ret = spiffs_object_truncate(fs, fobj, 0, false); + if (ret < 0) + { + goto errout_with_fileobject; + } + } + + /* Save the struct spiffs_file_s instance as the file private data */ + + filep->f_priv = fobj; + + /* In write/append mode, we need to set the file pointer to the end of the + * file. + */ + + offset = 0; + if ((oflags & (O_APPEND | O_WROK)) == (O_APPEND | O_WROK)) + { + offset = fobj->size == SPIFFS_UNDEFINED_LEN ? 0 : fobj->size; + } + + /* Save the file position */ + + filep->f_pos = offset; + + /* Add the new file object to the tail of the open file list */ + + dq_addlast((FAR dq_entry_t *)fobj, &fs->objq); + + spiffs_unlock_volume(fs); + return OK; + +errout_with_fileobject: + kmm_free(fobj); + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_close + ****************************************************************************/ + +static int spiffs_close(FAR struct file *filep) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + + finfo("filep: %p\n", filep); + DEBUGASSERT(filep->f_priv != NULL && filep->f_inode != NULL); + + /* Get the mountpoint inode reference from the file structure and the + * volume state data from the inode structure + */ + + inode = filep->f_inode; + fs = inode->i_private; + DEBUGASSERT(fs != NULL); + + /* Recover our private data from the struct file instance */ + + fobj = filep->f_priv; + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Decrement the reference count on the file */ + + DEBUGASSERT(fobj->crefs > 0); + if (fobj->crefs > 0) + { + fobj->crefs--; + } + + filep->f_priv = NULL; + + /* If the reference count decremented to zero and the file has been + * unlinked, then free resources related to the open file. + */ + + if (fobj->crefs == 0 && (fobj->flags & SFO_FLAG_UNLINKED) != 0) + { + /* Free the file object while we hold the lock? Weird but this + * should be safe because the object is unlinked and could not + * have any other references. + */ + + spiffs_fobj_free(fs, fobj); + return OK; + } + + /* Release the lock on the file system */ + + spiffs_unlock_volume(fs); + return OK; +} + +/**************************************************************************** + * Name: spiffs_read + ****************************************************************************/ + +static ssize_t spiffs_read(FAR struct file *filep, FAR char *buffer, + size_t buflen) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + ssize_t nread; + + finfo("filep: %p buffer: %p buflen: %lu\n", + filep, buffer, (unsigned long)buflen); + DEBUGASSERT(filep->f_priv != NULL && filep->f_inode != NULL); + + /* Get the mountpoint inode reference from the file structure and the + * volume state data from the inode structure + */ + + inode = filep->f_inode; + fs = inode->i_private; + DEBUGASSERT(fs != NULL); + + /* Recover the file object state from the struct file instance */ + + fobj = filep->f_priv; + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Read from FLASH */ + + nread = spiffs_fobj_read(fs, fobj, buffer, buflen, filep->f_pos); + if (nread > 0) + { + filep->f_pos += nread; + } + + /* Release the lock on the file system */ + + spiffs_unlock_volume(fs); + return nread; +} + +/**************************************************************************** + * Name: spiffs_write + ****************************************************************************/ + +static ssize_t spiffs_write(FAR struct file *filep, FAR const char *buffer, + size_t buflen) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + ssize_t nwritten; + off_t offset; + int ret; + + finfo("filep: %p buffer: %p buflen: %lu\n", + filep, buffer, (unsigned long)buflen); + DEBUGASSERT(filep->f_priv != NULL && filep->f_inode != NULL); + + /* Get the mountpoint inode reference from the file structure and the + * volume state data from the inode structure + */ + + inode = filep->f_inode; + fs = inode->i_private; + DEBUGASSERT(fs != NULL); + + /* Recover the file object state from the struct file instance */ + + fobj = filep->f_priv; + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Verify that the file was opened with write access */ + + if ((fobj->oflags & O_WROK) == 0) + { + ret = -EACCES; + goto errout_with_lock; + } + + /* Write to FLASH (or cache) */ + + offset = filep->f_pos; + + if (fobj->cache_page == 0) + { + /* See if object ID is associated with cache already */ + + fobj->cache_page = spiffs_cache_page_get_byobjid(fs, fobj); + } + + if ((fobj->flags & O_DIRECT) == 0) + { + if (buflen < (size_t)SPIFFS_CFG_LOG_PAGE_SZ(fs)) + { + /* Small write, try to cache it */ + + bool alloc_cpage = true; + if (fobj->cache_page != NULL) + { + /* We have a cached page for this object already, check cache + * page boundaries + */ + + if (offset < fobj->cache_page->offset || + offset > fobj->cache_page->offset + fobj->cache_page->size || + offset + buflen > fobj->cache_page->offset + SPIFFS_CFG_LOG_PAGE_SZ(fs)) + { + /* Boundary violation, write back cache first and allocate + * new + */ + + spiffs_cacheinfo("Cache page=%d for fobj ID=%d " + "Boundary violation, offset=%d size=%d\n", + fobj->cache_page->cpndx, fobj->objid, + fobj->cache_page->offset, fobj->cache_page->size); + nwritten = spiffs_fobj_write(fs, fobj, + spiffs_get_cache_page(fs, spiffs_get_cache(fs), + fobj->cache_page->cpndx), + fobj->cache_page->offset, + fobj->cache_page->size); + spiffs_cache_page_release(fs, fobj->cache_page); + if (nwritten < 0) + { + ret = (int)nwritten; + goto errout_with_lock; + } + } + else + { + /* Writing within cache */ + + alloc_cpage = false; + } + } + + if (alloc_cpage) + { + fobj->cache_page = spiffs_cache_page_allocate_byobjid(fs, fobj); + if (fobj->cache_page) + { + fobj->cache_page->offset = offset; + fobj->cache_page->size = 0; + + spiffs_cacheinfo("Allocated cache page %d for fobj %d\n", + fobj->cache_page->cpndx, fobj->objid); + } + } + + if (fobj->cache_page) + { + FAR struct spiffs_cache_s *cache; + FAR uint8_t *cpage_data; + off_t offset_in_cpage; + + offset_in_cpage = offset - fobj->cache_page->offset; + spiffs_cacheinfo("Storing to cache page %d for fobj %d offset=%d:%d buflen=%d\n", + fobj->cache_page->cpndx, fobj->objid, offset, + offset_in_cpage, buflen); + cache = spiffs_get_cache(fs); + cpage_data = spiffs_get_cache_page(fs, cache, fobj->cache_page->cpndx); + + memcpy(&cpage_data[offset_in_cpage], buffer, buflen); + fobj->cache_page->size = MAX(fobj->cache_page->size, offset_in_cpage + buflen); + + nwritten = buflen; + goto success_with_lock; + } + else + { + nwritten = spiffs_fobj_write(fs, fobj, buffer, offset, buflen); + if (nwritten < 0) + { + ret = (int)nwritten; + goto errout_with_lock; + } + + goto success_with_lock; + } + } + else + { + /* Big write, no need to cache it - but first check if there is a + * cached write already + */ + + if (fobj->cache_page) + { + /* Write back cache first */ + + spiffs_cacheinfo("Cache page=%d for fobj ID=%d " + "Boundary violation, offset=%d size=%d\n", + fobj->cache_page->cpndx, fobj->objid, + fobj->cache_page->offset, fobj->cache_page->size); + + nwritten = spiffs_fobj_write(fs, fobj, + spiffs_get_cache_page(fs, + spiffs_get_cache(fs), + fobj->cache_page->cpndx), + fobj->cache_page->offset, + fobj->cache_page->size); + spiffs_cache_page_release(fs, fobj->cache_page); + + if (nwritten < 0) + { + ret = (int)nwritten; + goto errout_with_lock; + } + + /* Data written below */ + } + } + } + + nwritten = spiffs_fobj_write(fs, fobj, buffer, offset, buflen); + if (nwritten < 0) + { + ret = (int)nwritten; + goto errout_with_lock; + } + +success_with_lock: + /* Update the file position */ + + filep->f_pos += nwritten; + + /* Release our access to the volume */ + + spiffs_unlock_volume(fs); + return nwritten; + +errout_with_lock: + spiffs_unlock_volume(fs); + return (ssize_t)ret; +} + +/**************************************************************************** + * Name: spiffs_seek + ****************************************************************************/ + +static off_t spiffs_seek(FAR struct file *filep, off_t offset, int whence) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + int16_t data_spndx; + int16_t objndx_spndx; + off_t fsize; + off_t pos; + int ret; + + finfo("filep=%p offset=%ld whence=%d\n", filep, (long)offset, whence); + DEBUGASSERT(filep->f_priv != NULL && filep->f_inode != NULL); + + /* Get the mountpoint inode reference from the file structure and the + * volume state data from the inode structure + */ + + inode = filep->f_inode; + fs = inode->i_private; + DEBUGASSERT(fs != NULL); + + /* Recover the file object state from the struct file instance */ + + fobj = filep->f_priv; + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Get the new file offset */ + + spiffs_fflush_cache(fs, fobj); + + fsize = fobj->size == SPIFFS_UNDEFINED_LEN ? 0 : fobj->size; + + /* Map the offset according to the whence option */ + + switch (whence) + { + case SEEK_SET: /* The offset is set to offset bytes. */ + pos = offset; + break; + + case SEEK_CUR: /* The offset is set to its current location plus + * offset bytes. */ + pos = offset + filep->f_pos; + break; + + case SEEK_END: /* The offset is set to the size of the file plus + * offset bytes. */ + pos = fsize + offset; + break; + + default: + return -EINVAL; + } + + /* Verify the resulting file position */ + + if (pos < 0) + { + ret = -EINVAL; + goto errout_with_lock; + } + + /* Attempts to set the position beyond the end of file should + * work if the file is open for write access. + * + * REVISIT: This simple implementation has no per-open storage that + * would be needed to retain the open flags. + */ + + if (pos > fsize) + { + filep->f_pos = fsize; + ret = -ENOSYS; + goto errout_with_lock; + } + + /* Set up for the new file position */ + + + data_spndx = (pos > 0 ? (pos - 1) : 0) / SPIFFS_DATA_PAGE_SIZE(fs); + objndx_spndx = SPIFFS_OBJ_IX_ENTRY_SPAN_IX(fs, data_spndx); + + if (fobj->objndx_spndx != objndx_spndx) + { + int16_t pgndx; + + ret = spiffs_obj_lu_find_id_and_span(fs, fobj->objid | SPIFFS_OBJ_ID_IX_FLAG, + objndx_spndx, 0, &pgndx); + if (ret < 0) + { + goto errout_with_lock; + } + + fobj->objndx_spndx = objndx_spndx; + fobj->objndx_pgndx = pgndx; + } + + filep->f_pos = pos; + spiffs_unlock_volume(fs); + return pos; + +errout_with_lock: + spiffs_unlock_volume(fs); + return (off_t)ret; +} + +/**************************************************************************** + * Name: spiffs_ioctl + ****************************************************************************/ + +static int spiffs_ioctl(FAR struct file *filep, int cmd, unsigned long arg) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + int ret; + + finfo("filep=%p cmd=%d arg=%ld\n", filep, cmd, (long)arg); + DEBUGASSERT(filep->f_priv != NULL && filep->f_inode != NULL); + + /* Get the mountpoint inode reference from the file structure and the + * volume state data from the inode structure + */ + + inode = filep->f_inode; + fs = inode->i_private; + DEBUGASSERT(fs != NULL); + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Handle the IOCTL according tot he command */ + + switch (cmd) + { + /* Run a consistency check on the file system media. + * IN: None + * OUT: None + */ + + case BIOC_CHECK: + { + ret = spiffs_consistency_check(fs); + } + break; + + /* Force reformatting of media. All data will be lost. + * IN: None + * OUT: None + */ + + case BIOC_FORMAT: + { + /* Check if the MTD driver supports the MTDIOC_BULKERASE command */ + + ret = MTD_IOCTL(fs->mtd, MTDIOC_BULKERASE, 0); + if (ret < 0) + { + /* No.. we will have to erase a block at a time */ + + int16_t blkndx = 0; + while (blkndx < fs->geo.neraseblocks) + { + fs->max_erase_count = 0; + ret = spiffs_erase_block(fs, blkndx); + if (ret < 0) + { + return ret; + } + + blkndx++; + } + } + } + break; + + /* Run garbage collection. + * IN: On entry holds the number of bytes to be recovered. + * OUT: None + */ + + case BIOC_GC: + { + ret = spiffs_gc_check(fs, (size_t)arg); + } + break; + + default: + /* Pass through to the contained MTD driver */ + + ret = MTD_IOCTL(fs->mtd, cmd, arg); + break; + } + + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_sync + * + * Description: Synchronize the file state on disk to match internal, in- + * memory state. + * + ****************************************************************************/ + +static int spiffs_sync(FAR struct file *filep) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + int ret; + + finfo("filep=%p\n", filep); + DEBUGASSERT(filep->f_priv != NULL && filep->f_inode != NULL); + + /* Get the mountpoint inode reference from the file structure and the + * volume state data from the inode structure + */ + + inode = filep->f_inode; + fs = inode->i_private; + DEBUGASSERT(fs != NULL); + + /* Recover the file object state from the struct file instance */ + + fobj = filep->f_priv; + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Flush all cached write data */ + + ret = spiffs_fflush_cache(fs, fobj); + + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_dup + ****************************************************************************/ + +static int spiffs_dup(FAR const struct file *oldp, FAR struct file *newp) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + + finfo("Dup %p->%p\n", oldp, newp); + DEBUGASSERT(oldp->f_priv != NULL && oldp->f_inode != NULL && + newp->f_priv == NULL && newp->f_inode != NULL); + + /* Get the mountpoint inode reference from the file structure and the + * volume state data from the inode structure + */ + + inode = oldp->f_inode; + fs = inode->i_private; + DEBUGASSERT(fs != NULL); + + /* Recover our private data from the struct file instance */ + + fobj = oldp->f_priv; + + /* Increment the reference count (atomically)*/ + + spiffs_lock_volume(fs); + fobj->crefs++; + spiffs_unlock_volume(fs); + + /* Save a copy of the file object as the dup'ed file. */ + + newp->f_priv = fobj; + return OK; +} + +/**************************************************************************** + * Name: spiffs_fstat + * + * Description: + * Obtain information about an open file associated with the file + * descriptor 'fobj', and will write it to the area pointed to by 'buf'. + * + ****************************************************************************/ + +static int spiffs_fstat(FAR const struct file *filep, FAR struct stat *buf) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + int ret; + + finfo("filep=%p buf=%p\n", filep, buf); + DEBUGASSERT(filep->f_priv != NULL && filep->f_inode != NULL && buf != NULL); + + /* Get the mountpoint inode reference from the file structure and the + * volume state data from the inode structure + */ + + inode = filep->f_inode; + fs = inode->i_private; + DEBUGASSERT(fs != NULL); + + /* Recover the file object state from the struct file instance */ + + fobj = filep->f_priv; + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Flush the cache and perform the common stat() operation */ + + spiffs_fflush_cache(fs, fobj); + + ret = spiffs_stat_pgndx(fs, fobj->objhdr_pgndx, fobj->objid, buf); + if (ret < 0) + { + ferr("ERROR: spiffs_stat_pgndx failed: %d\n", ret); + } + + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_truncate + ****************************************************************************/ + +static int spiffs_truncate(FAR struct file *filep, off_t length) +{ + FAR struct inode *inode; + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + off_t fsize; + int ret; + + finfo("filep: %p length: %ld\n", filep, (long)length); + DEBUGASSERT(filep->f_priv != NULL && filep->f_inode != NULL && length >= 0); + + /* Get the mountpoint inode reference from the file structure and the + * volume state data from the inode structure + */ + + inode = filep->f_inode; + fs = inode->i_private; + DEBUGASSERT(fs != NULL); + + /* Recover the file object state from the struct file instance */ + + fobj = filep->f_priv; + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* REVISIT: spiffs_object_truncate() can only truncate to smaller sizes. */ + + ret = spiffs_object_truncate(fs, fobj, length, false); + if (ret < 0) + { + ferr("ERROR: spiffs_object_truncate failed: %d/n", ret); + } + + /* Check if we need to reset the file pointer. Probably could use + * 'length', but let's use the authoritative file file size for the + * comparison. + */ + + fsize = fobj->size == SPIFFS_UNDEFINED_LEN ? 0 : fobj->size; + + if (ret >= 0 && fsize < filep->f_pos) + { + /* Reset the file pointer to the new end-of-file position */ + + filep->f_pos = fsize; + } + + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_opendir + ****************************************************************************/ + +static int spiffs_opendir(FAR struct inode *mountpt, FAR const char *relpath, + FAR struct fs_dirent_s *dir) +{ + finfo("mountpt: %p relpath: %s dir: %p\n", + mountpt, relpath, dir); + + DEBUGASSERT(mountpt != NULL && relpath != NULL && dir != NULL); + + /* Initialize for traversal of the 'directory' */ + + dir->u.spiffs.block = 0; + dir->u.spiffs.entry = 0; + return OK; +} + +/**************************************************************************** + * Name: spiffs_closedir + ****************************************************************************/ + +static int spiffs_closedir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir) +{ + finfo("mountpt: %p dir: %p\n", mountpt, dir); + DEBUGASSERT(mountpt != NULL && dir != NULL); + + /* There is nothing to be done */ + + return OK; +} + +/**************************************************************************** + * Name: spiffs_readdir + ****************************************************************************/ + +static int spiffs_readdir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir) +{ + FAR struct spiffs_s *fs; + int16_t blkndx; + int entry; + int ret; + + finfo("mountpt: %p dir: %p\n", mountpt, dir); + DEBUGASSERT(mountpt != NULL && dir != NULL); + + /* Get the mountpoint private data from the inode structure */ + + fs = mountpt->i_private; + DEBUGASSERT(fs != NULL); + + /* Lock the SPIFFS volume */ + + spiffs_lock_volume(fs); + + /* And visit the next file object */ + + ret = spiffs_foreach_objlu(fs, dir->u.spiffs.block, dir->u.spiffs.entry, + SPIFFS_VIS_NO_WRAP, 0, spiffs_readdir_callback, + NULL, dir, &blkndx, &entry); + if (ret == OK) + { + dir->u.spiffs.block = blkndx; + dir->u.spiffs.entry = entry + 1; + } + + /* Release the lock on the file system */ + + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_rewinddir + ****************************************************************************/ + +static int spiffs_rewinddir(FAR struct inode *mountpt, + FAR struct fs_dirent_s *dir) +{ + finfo("mountpt: %p dir: %p\n", mountpt, dir); + DEBUGASSERT(mountpt != NULL && dir != NULL); + + /* Reset as when opendir() was called. */ + + dir->u.spiffs.block = 0; + dir->u.spiffs.entry = 0; + + return OK; +} + +/**************************************************************************** + * Name: spiffs_bind + ****************************************************************************/ + +static int spiffs_bind(FAR struct inode *mtdinode, FAR const void *data, + FAR void **handle) +{ + FAR struct spiffs_s *fs; + FAR struct mtd_dev_s *mtd; + FAR uint8_t *work; + size_t cache_size; + size_t cache_max; + size_t work_size; + size_t addrmask; + int ret; + + finfo("mtdinode: %p data: %p handle: %p\n", mtdinode, data, handle); + DEBUGASSERT(mtdinode == NULL && handle != NULL); + + /* Extract the MTD interface reference */ + + DEBUGASSERT(INODE_IS_MTD(mtdinode) && mtdinode->u.i_mtd != NULL); + mtd = mtdinode->u.i_mtd; + + /* Create an instance of the SPIFFS file system */ + + fs = (FAR struct spiffs_s *)kmm_zalloc(sizeof(struct spiffs_s)); + if (fs == NULL) + { + ferr("ERROR: Failed to allocate volume structure\n"); + return -ENOMEM; + } + + fs->mtd = mtd; + + /* Get the MTD geometry */ + + ret = MTD_IOCTL(mtd, MTDIOC_GEOMETRY, (unsigned long)((uintptr_t)&fs->geo)); + if (ret < 0) + { + ferr("ERROR: MTD_IOCTL(MTDIOC_GEOMETRY) failed: %d\n", ret); + goto errout_with_volume; + } + + fs->phys_size = fs->geo.neraseblocks * fs->geo.erasesize; + + /* Get the aligned cache size */ + + addrmask = (sizeof(FAR void *) - 1); + cache_size = (CONFIG_SPIFFS_CACHE_SIZE + addrmask) & ~addrmask; + + /* Don't let the cache size exceed the maximum that is needed */ + + cache_max = SPIFFS_CFG_LOG_PAGE_SZ(fs) << 5; + if (cache_size > cache_max) + { + cache_size = cache_max; + } + + /* Allocate the cache */ + + fs->cache_size = cache_size; + fs->cache = (FAR void *)kmm_malloc(cache_size); + + if (fs->cache == NULL) + { + ferr("ERROR: Failed to allocate volume structure\n"); + ret = -ENOMEM; + goto errout_with_volume; + } + + spiffs_cache_initialize(fs); + + /* Allocate the memory work buffer comprising 2*config->page_size bytes + * used throughout all file system operations. + * + * NOTE: Currently page size is equivalent to block size. + */ + + work_size = SPIFFS_CFG_LOG_PAGE_SZ(fs) << 1; + work = (FAR uint8_t *)kmm_malloc(work_size); + + if (work == NULL) + { + ferr("ERROR: Failed to allocate work buffer\n"); + ret = -ENOMEM; + goto errout_with_cache; + } + + fs->work = &work[0]; + fs->lu_work = &work[work_size >> 1]; + fs->config_magic = SPIFFS_SUPER_MAGIC; + + /* Check the file system */ + + ret = spiffs_obj_lu_scan(fs); + if (ret < 0) + { + ferr("ERROR: spiffs_obj_lu_scan() failed: %d\n", ret); + goto errout_with_work; + } + + finfo("page index byte len: %u\n", + (unsigned int)SPIFFS_CFG_LOG_PAGE_SZ(fs)); + finfo("object lookup pages: %u\n", + (unsigned int)SPIFFS_OBJ_LOOKUP_PAGES(fs)); + finfo("page pages per block: %u\n", + (unsigned int)SPIFFS_PAGES_PER_BLOCK(fs)); + finfo("page header length: %u\n", + (unsigned int)sizeof(struct spiffs_page_header_s)); + finfo("object header index entries: %u\n", + (unsigned int)SPIFFS_OBJ_HDR_IX_LEN(fs)); + finfo("object index entries: %u\n", + (unsigned int)SPIFFS_OBJ_IX_LEN(fs)); + finfo("free blocks: %u\n", + (unsigned int)fs->free_blocks); + +#ifdef CONFIG_SPIFFS_CHECK_ONMOUNT + /* Perform the full consistency check */ + + ret = spiffs_consistency_check(fs); + if (ret < 0) + { + fwarn("WARNING: File system is damaged: %d\n", ret); + } +#endif + + /* Return the new file system handle */ + + *handle = (FAR void *)fs; + return OK; + +errout_with_work: + kmm_free(fs->work); + +errout_with_cache: + kmm_free(fs->cache); + +errout_with_volume: + kmm_free(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_unbind + ****************************************************************************/ + +static int spiffs_unbind(FAR void *handle, FAR struct inode **mtdinode, + unsigned int flags) +{ + FAR struct spiffs_s *fs = (FAR struct spiffs_s *)handle; + FAR struct spiffs_file_s *fobj; + int ret; + + finfo("handle: %p mtdinode: %p flags: %02x\n", + handle, mtdinode, flags); + DEBUGASSERT(fs != NULL); + + /* Lock the file system */ + + spiffs_lock_volume(fs); + + /* Are there open file system? If so, are we being forced to unmount? */ + + if (!dq_empty(&fs->objq) && (flags & MNT_FORCE) == 0) + { + fwarn("WARNING: Open files and umount not forced\n"); + ret = -EBUSY; + goto errout_with_lock; + } + + /* Release all of the open file objects... Very scary stuff. */ + + while ((fobj = (FAR struct spiffs_file_s *)dq_peek(&fs->objq)) != NULL) + { + /* Free the file object */ + + spiffs_fobj_free(fs, fobj); + } + + /* Free allocated working buffers */ + + if (fs->work != NULL) + { + kmm_free(fs->work); + } + + if (fs->cache != NULL) + { + kmm_free(fs->cache); + } + + /* Free the volume memory (note that the semaphore is now stale!) */ + + nxsem_destroy(&fs->exclsem.sem); + kmm_free(fs); + ret = OK; + +errout_with_lock: + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_statfs + ****************************************************************************/ + +static int spiffs_statfs(FAR struct inode *mountpt, FAR struct statfs *buf) +{ + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + uint32_t pages_per_block; + uint32_t blocks; + uint32_t obj_lupages; + uint32_t data_pgsize; + uint32_t ndata_pages; + uint32_t nfile_objs; + uint32_t used; + + finfo("mountpt: %p buf: %p\n", mountpt, buf); + DEBUGASSERT(mountpt != NULL && buf != NULL); + + /* Get the mountpoint private data from the inode structure */ + + fs = mountpt->i_private; + DEBUGASSERT(fs != NULL); + + /* Lock the SPIFFS volume */ + + spiffs_lock_volume(fs); + + /* Collect some statistics */ + + pages_per_block = SPIFFS_PAGES_PER_BLOCK(fs); + blocks = fs->geo.neraseblocks; + obj_lupages = SPIFFS_OBJ_LOOKUP_PAGES(fs); + data_pgsize = SPIFFS_DATA_PAGE_SIZE(fs); + + /* -2 for spare blocks, +1 for emergency page */ + + ndata_pages = (blocks - 2) * (pages_per_block - obj_lupages) + 1; + used = fs->stats_p_allocated * data_pgsize; + + /* Count the number of file objects */ + + nfile_objs = 0; + for (fobj = (FAR struct spiffs_file_s *)dq_peek(&fs->objq); + fobj != NULL; + fobj = (FAR struct spiffs_file_s *)dq_next((FAR dq_entry_t *)fobj)) + { + nfile_objs++; + } + + /* Fill in the statfs structure */ + + buf->f_type = SPIFFS_SUPER_MAGIC; + buf->f_namelen = CONFIG_SPIFFS_NAME_MAX - 1; + buf->f_bsize = data_pgsize; + buf->f_blocks = ndata_pages; + buf->f_bfree = ndata_pages - used; + buf->f_bavail = buf->f_bfree; + buf->f_files = nfile_objs; + buf->f_ffree = buf->f_bfree; /* SWAG */ + + /* Release the lock on the file system */ + + spiffs_unlock_volume(fs); + return OK; +} + +/**************************************************************************** + * Name: spiffs_unlink + ****************************************************************************/ + +static int spiffs_unlink(FAR struct inode *mountpt, FAR const char *relpath) +{ + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + int16_t pgndx; + int ret; + + finfo("mountpt: %p relpath: %s\n", mountpt, relpath); + DEBUGASSERT(mountpt != NULL && relpath != NULL); + + if (strlen(relpath) > CONFIG_SPIFFS_NAME_MAX - 1) + { + return -ENAMETOOLONG; + } + + /* Get the file system structure from the inode reference. */ + + fs = mountpt->i_private; + DEBUGASSERT(fs != NULL); + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Find the page index to the object header associated with this path */ + + ret = spiffs_find_objhdr_pgndx(fs, (FAR const uint8_t *)relpath, &pgndx); + if (ret < OK) + { + fwarn("WARNING: No objhdr found for relpath '%s': %d\n", relpath, ret); + goto errout_with_lock; + } + else if (ret != -ENOENT) + { + ferr("ERROR: spiffs_find_objhdr_pgndx failed: %d\n", ret); + goto errout_with_lock; + } + + /* Check to see if there is an open file reference for the object at this + * page index. + */ + + ret = spiffs_find_fobj_bypgndx(fs, pgndx, &fobj); + if (ret >= 0) + { + /* If so, then we cannot unlink the file now. Just mark the file as + * 'unlinked' so that it can be removed when the file object is + * released. + */ + + fobj->flags |= SFO_FLAG_UNLINKED; + } + else + { + /* Otherwise, we will ne to re-open the file */ + /* Allocate new file object */ + + fobj = (FAR struct spiffs_file_s *)kmm_zalloc(sizeof(struct spiffs_file_s)); + if (fobj == NULL) + { + fwarn("WARNING: Failed to allocate fobjs\n"); + ret = -ENOMEM; + goto errout_with_lock; + } + + /* Use the page index to open the file */ + + ret = spiffs_object_open_bypage(fs, pgndx, fobj, 0, 0); + if (ret < 0) + { + ferr("ERROR: spiffs_object_open_bypage failed: %d\n", ret); + kmm_free(fobj); + goto errout_with_lock; + } + + /* Now we can remove the file by truncating it to zero length */ + + ret = spiffs_object_truncate(fs, fobj, 0, true); + kmm_free(fobj); + + if (ret < 0) + { + ferr("ERROR: spiffs_object_truncate failed: %d\n", ret); + goto errout_with_lock; + } + } + + /* Release the lock on the volume */ + + spiffs_unlock_volume(fs); + return OK; + +errout_with_lock: + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_mkdir + ****************************************************************************/ + +static int spiffs_mkdir(FAR struct inode *mountpt, FAR const char *relpath, + mode_t mode) +{ + finfo("mountpt: %p relpath: %s mode: %04x\n", mountpt, relpath, mode); + DEBUGASSERT(mountpt != NULL && relpath != NULL); + + /* Directories are not supported */ + + return -ENOSYS; +} + +/**************************************************************************** + * Name: spiffs_rmdir + ****************************************************************************/ + +static int spiffs_rmdir(FAR struct inode *mountpt, FAR const char *relpath) +{ + finfo("mountpt: %p relpath: %s\n", mountpt, relpath); + DEBUGASSERT(mountpt != NULL && relpath != NULL); + + /* Directories are not supported */ + + return -ENOSYS; +} + +/**************************************************************************** + * Name: spiffs_rename + ****************************************************************************/ + +static int spiffs_rename(FAR struct inode *mountpt, FAR const char *oldrelpath, + FAR const char *newrelpath) +{ + FAR struct spiffs_s *fs; + FAR struct spiffs_file_s *fobj; + int16_t oldpgndx; + int16_t newpgndx; + int ret; + + finfo("mountpt: %p oldrelpath: %s newrelpath: %s\n", + mountpt, oldrelpath, newrelpath); + DEBUGASSERT(mountpt != NULL && oldrelpath != NULL && newrelpath != NULL); + + /* Get the file system structure from the inode reference. */ + + fs = mountpt->i_private; + DEBUGASSERT(fs != NULL); + + if (strlen(newrelpath) > CONFIG_SPIFFS_NAME_MAX - 1 || + strlen(oldrelpath) > CONFIG_SPIFFS_NAME_MAX - 1) + { + return -ENAMETOOLONG; + } + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Get the page index of the object header for the oldrelpath */ + + ret = spiffs_find_objhdr_pgndx(fs, (FAR const uint8_t *)oldrelpath, + &oldpgndx); + if (ret < 0) + { + fwarn("WARNING: spiffs_find_objhdr_pgndx failed: %d\n"); + goto errout_with_lock; + } + + /* Check if there is any file object corresponding to the newrelpath */ + + ret = spiffs_find_objhdr_pgndx(fs, (FAR const uint8_t *)newrelpath, + &newpgndx); + if (ret == -ENOENT) + { + ret = OK; + } + else if (ret == OK) + { + ret = -EEXIST; + } + + if (ret < 0) + { + goto errout_with_lock; + } + + /* Allocate new file object. NOTE: The file could already be open. */ + + fobj = (FAR struct spiffs_file_s *)kmm_zalloc(sizeof(struct spiffs_file_s)); + if (fobj == NULL) + { + return -ENOMEM; + } + + /* Use the page index to open the file */ + + ret = spiffs_object_open_bypage(fs, oldpgndx, fobj, 0, 0); + if (ret < 0) + { + goto errout_with_fobj; + } + + /* Then update the file name */ + + ret = spiffs_object_update_index_hdr(fs, fobj, fobj->objid, + fobj->objhdr_pgndx, 0, + (FAR const uint8_t *)newrelpath, 0, + &newpgndx); + +errout_with_fobj: + kmm_free(fobj); + +errout_with_lock: + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Name: spiffs_stat + ****************************************************************************/ + +static int spiffs_stat(FAR struct inode *mountpt, FAR const char *relpath, + FAR struct stat *buf) +{ + FAR struct spiffs_s *fs; + int16_t pgndx; + int ret; + + finfo("mountpt=%p relpath=%s buf=%p\n", mountpt, relpath, buf); + DEBUGASSERT(mountpt != NULL && relpath != NULL && buf != NULL); + + if (strlen(relpath) > CONFIG_SPIFFS_NAME_MAX - 1) + { + return -ENAMETOOLONG; + } + + /* Get the file system structure from the inode reference. */ + + fs = mountpt->i_private; + DEBUGASSERT(fs != NULL); + + /* Get exclusive access to the file system */ + + spiffs_lock_volume(fs); + + /* Find the object associated with this relative path */ + + ret = spiffs_find_objhdr_pgndx(fs, (FAR const uint8_t *)relpath, &pgndx); + if (ret < 0) + { + goto errout_with_lock; + } + + /* And get information about the object */ + + ret = spiffs_stat_pgndx(fs, pgndx, 0, buf); + if (ret < 0) + { + ferr("ERROR: spiffs_stat_pgndx failed: %d\n", ret); + } + +errout_with_lock: + spiffs_unlock_volume(fs); + return ret; +} + +/**************************************************************************** + * Public Functions + ****************************************************************************/ diff --git a/fs/spiffs/src/spiffs_volume.c b/fs/spiffs/src/spiffs_volume.c new file mode 100644 index 00000000000..a2c16e1f30e --- /dev/null +++ b/fs/spiffs/src/spiffs_volume.c @@ -0,0 +1,470 @@ +/**************************************************************************** + * fs/spiffs.h/spiffs_volume.c + * SPIFFS Utility Functions for Volume and File Object Support + * + * Copyright (C) 2018 Gregory Nutt. All rights reserved. + * Author: Gregory Nutt + * + * This is a port of version 0.3.7 of SPIFFS by Peter Andersion. That + * version was originally released under the MIT license but is here re- + * released under the NuttX BSD license. + * + * Copyright (c) 2013-2017 Peter Andersson (pelleplutt1976@gmail.com) + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * 3. Neither the name NuttX nor the names of its contributors may be + * used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, + * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS + * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED + * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN + * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + * + ****************************************************************************/ + +/**************************************************************************** + * Included Files + ****************************************************************************/ + +#include + +#include +#include +#include + +#include + +#include "spiffs.h" +#include "spiffs_core.h" +#include "spiffs_cache.h" + +/**************************************************************************** + * Public Functions + ****************************************************************************/ + +/**************************************************************************** + * Name: spiffs_stat_pgndx + * + * Description: + * Checks if there are any cached writes for the object ID associated with + * given file object. If so, these writes are flushed. + * + * Input Parameters: + * fobj - A reference to the file object to flush + * + * Returned Value: + * On success, then number of bytes flushed is returned. A negated errno + * value is returned on any failure. + * + ****************************************************************************/ + +int spiffs_stat_pgndx(FAR struct spiffs_s *fs, int16_t pgndx, int16_t objid, + FAR struct stat *buf) +{ + struct spiffs_pgobj_ixheader_s objhdr; + uint32_t obj_id_addr; + int16_t ndx; + mode_t mode; + int ret; + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_IX | SPIFFS_OP_C_READ, objid, + SPIFFS_PAGE_TO_PADDR(fs, pgndx), + sizeof(struct spiffs_pgobj_ixheader_s), + (FAR uint8_t *) & objhdr); + if (ret < 0) + { + return ret; + } + + obj_id_addr = + SPIFFS_BLOCK_TO_PADDR(fs, SPIFFS_BLOCK_FOR_PAGE(fs, pgndx)) + + SPIFFS_OBJ_LOOKUP_ENTRY_FOR_PAGE(fs, pgndx) * sizeof(int16_t); + + ret = spiffs_cache_read(fs, SPIFFS_OP_T_OBJ_LU | SPIFFS_OP_C_READ, objid, + obj_id_addr, sizeof(int16_t), (FAR uint8_t *)&ndx); + if (ret < 0) + { + return ret; + } + + /* Build the struct stat */ + + mode = S_IRWXO | S_IRWXG | S_IRWXU; /* Assume all permissions */ + mode |= S_IFREG; /* Assume regular file */ + + /* REVISIT: Should the file object type derive from objhdr.type? */ + + memset(buf, 0, sizeof(struct stat)); + buf->st_mode = mode; + buf->st_size = objhdr.size == SPIFFS_UNDEFINED_LEN ? 0 : objhdr.size; + buf->st_blksize = fs->geo.blocksize; + buf->st_blocks = fs->phys_size / fs->geo.blocksize; + + return ret; +} + +/**************************************************************************** + * Name: spiffs_find_fobj_bypgndx + * + * Description: + * Given the page index of the object header, find the corresponding file + * object instance. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * pgndx - The page index to match + * ppfobj - A user provided location in which to return the matching file + * file object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_find_fobj_bypgndx(FAR struct spiffs_s *fs, int16_t pgndx, + FAR struct spiffs_file_s **ppfobj) +{ + FAR struct spiffs_file_s *fobj; + int ret = -ENOENT; + + for (fobj = (FAR struct spiffs_file_s *)dq_peek(&fs->objq); + fobj != NULL; + fobj = (FAR struct spiffs_file_s *)dq_next((FAR dq_entry_t *)fobj)) + { + if (fobj->objhdr_pgndx == pgndx) + { + ret = OK; + break; + } + } + + if (ppfobj != NULL) + { + *ppfobj = fobj; + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_find_fobj_byobjid + * + * Description: + * Given a object ID, find the corresponding file object instance + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * objid - The object ID to match + * ppfobj - A user provided location in which to return the matching file + * file object instance + * + * Returned Value: + * Zero (OK) is returned on success; A negated errno value is returned on + * any failure. + * + ****************************************************************************/ + +int spiffs_find_fobj_byobjid(FAR struct spiffs_s *fs, int16_t objid, + FAR struct spiffs_file_s **ppfobj) +{ + FAR struct spiffs_file_s *fobj; + int ret = -ENOENT; + + for (fobj = (FAR struct spiffs_file_s *)dq_peek(&fs->objq); + fobj != NULL; + fobj = (FAR struct spiffs_file_s *)dq_next((FAR dq_entry_t *)fobj)) + { + if (fobj->objid == objid) + { + ret = OK; + break; + } + } + + if (ppfobj != NULL) + { + *ppfobj = fobj; + } + + return ret; +} + +/**************************************************************************** + * Name: spiffs_fflush_cache + * + * Description: + * Checks if there are any cached writes for the object ID associated with + * given file object. If so, these writes are flushed. + * + * Input Parameters: + * fs - A reference to the SPIFFS volume object instance + * fobj - A reference to the file object to flush + * + * Returned Value: + * On success, then number of bytes flushed is returned. A negated errno + * value is returned on any failure. + * + ****************************************************************************/ + +ssize_t spiffs_fflush_cache(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj) +{ + ssize_t nwritten = 0; + + /* If we were asked to use direct hardware accessed, then don't bother + * flushing the cache. + */ + + if ((fobj->flags & O_DIRECT) == 0) + { + if (fobj->cache_page == 0) + { + /* See if object ID is associated with cache already */ + + fobj->cache_page = spiffs_cache_page_get_byobjid(fs, fobj); + } + + if (fobj->cache_page) + { + spiffs_cacheinfo("Flushing cache page %d for objid=%d offset=%d size=%d\n", + fobj->cache_page->cpndx, fobj->objid, + fobj->cache_page->offset, fobj->cache_page->size); + + nwritten = + spiffs_fobj_write(fs, fobj, + spiffs_get_cache_page(fs, spiffs_get_cache(fs), + fobj->cache_page->cpndx), + fobj->cache_page->offset, fobj->cache_page->size); + if (nwritten < 0) + { + ferr("ERROR: spiffs_fobj_write failed %d\n", (int)nwritten); + } + + spiffs_cache_page_release(fs, fobj->cache_page); + } + } + + return nwritten; +} + +/**************************************************************************** + * Name: spiffs_fobj_write + * + * Description: + * Write to a file object + * + * Input Parameters: + * fs - A reference to the volume structure + * fobj - A reference to the file object to write to + * buffer - The data to be written + * offset - The FLASH offset to be written + * len - The number of bytes to be written + * + * Returned Value: + * On success, then number of bytes written is returned. A negated errno + * value is returned on any failure. + * + ****************************************************************************/ + +ssize_t spiffs_fobj_write(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, + FAR const void *buffer, off_t offset, size_t len) +{ + ssize_t remaining = len; + ssize_t nwritten = 0; + + if (fobj->size != SPIFFS_UNDEFINED_LEN && offset < fobj->size) + { + FAR const uint8_t *tmp; + ssize_t wrsize; + + wrsize = MIN((ssize_t)(fobj->size - offset), len); + nwritten = spiffs_object_modify(fs, fobj, offset, + (FAR uint8_t *)buffer, wrsize); + if (nwritten <= 0) + { + return nwritten; + } + + remaining -= nwritten; + + tmp = (FAR const uint8_t *)buffer; + tmp += nwritten; + buffer = tmp; + offset += nwritten; + } + + if (remaining > 0) + { + ssize_t nappend; + nappend = spiffs_object_append(fs, fobj, offset, + (FAR uint8_t *)buffer, remaining); + if (nappend < 0) + { + return (ssize_t)nappend; + } + + nwritten += nappend; + } + + return (ssize_t)nwritten; +} + +/**************************************************************************** + * Name: spiffs_fobj_read + * + * Description: + * Read from a file object + * + * Input Parameters: + * fs - A reference to the volume structure + * fobj - A reference to the file object to read from + * buffer - The location that the data is read to + * offset - The FLASH offset to be read + * len - The number of bytes to be read + * fpos - The file position to read from + * + * Returned Value: + * On success, then number of bytes written is returned. A negated errno + * value is returned on any failure. + * + ****************************************************************************/ + +ssize_t spiffs_fobj_read(FAR struct spiffs_s *fs, + FAR struct spiffs_file_s *fobj, FAR void *buffer, + size_t buflen, off_t fpos) +{ + ssize_t nread; + + /* Make sure that read access is supported */ + + if ((fobj->flags & O_RDONLY) == 0) + { + return -EACCES; + } + + /* Handle the special case of zero-length files */ + + if (fobj->size == SPIFFS_UNDEFINED_LEN && buflen > 0) + { + /* Return zero (meaning EOF) */ + + return 0; + } + + spiffs_fflush_cache(fs, fobj); + + /* Check for attempts to read beyond the end of the file */ + + if (fpos + buflen >= fobj->size) + { + /* Truncate */ + + buflen = fobj->size - fpos; + + /* Check if we are already at or beyond the end of the file */ + + if (buflen <= 0) + { + /* Return zero (meaning EOF) */ + + nread = 0; + } + else + { + /* Then read from the file object */ + + nread = spiffs_object_read(fs, fobj, fpos, buflen, + (FAR uint8_t *)buffer); + } + } + else + { + /* Reading within file size */ + + nread = spiffs_object_read(fs, fobj, fpos, buflen, + (FAR uint8_t *)buffer); + } + + return nread; +} + +/**************************************************************************** + * Name: spiffs_fobj_free + * + * Description: + * Free all resources used a file object + * + * Input Parameters: + * fs - A reference to the volume structure + * fobj - A reference to the file object to be removed + * + * Returned Value: + * None + * + ****************************************************************************/ + +void spiffs_fobj_free(FAR struct spiffs_s *fs, FAR struct spiffs_file_s *fobj) +{ + FAR struct spiffs_file_s *curr; + int ret; + + /* Flush any buffered write data */ + + ret = spiffs_fflush_cache(fs, fobj); + if (ret < 0) + { + ferr("ERROR: spiffs_fflush_cache failed: %d\n", ret); + } + + /* Remove the file object from the list of file objects in the volume + * structure. + */ + + for (curr = (FAR struct spiffs_file_s *)dq_peek(&fs->objq); + curr != NULL; + curr = (FAR struct spiffs_file_s *)dq_next((FAR dq_entry_t *)curr)) + { + /* Is this the entry we are searching for? */ + + if (curr == fobj) + { + /* Yes, remove it from the list of of file objects */ + + dq_rem((FAR dq_entry_t *)curr, &fs->objq); + } + } + + DEBUGASSERT(curr != NULL); + + /* Now we can remove the file by truncating it to zero length */ + + ret = spiffs_object_truncate(fs, fobj, 0, true); + if (ret < 0) + { + ferr("ERROR: spiffs_object_truncate failed: %d\n", ret); + } + + /* Then free the file object itself (which contains the lock we hold) */ + + kmm_free(fobj); +} diff --git a/include/fcntl.h b/include/fcntl.h index 6145c11fd6b..12ad7bbf51c 100644 --- a/include/fcntl.h +++ b/include/fcntl.h @@ -65,6 +65,7 @@ #define O_SYNC (1 << 7) /* Synchronize output on write */ #define O_DSYNC O_SYNC /* Equivalent to OSYNC in NuttX */ #define O_BINARY (1 << 8) /* Open the file in binary (untranslated) mode. */ +#define O_DIRECT (1 << 9) /* Avoid caching, write directly to hardware */ /* Unsupported, but required open flags */ diff --git a/include/nuttx/fs/dirent.h b/include/nuttx/fs/dirent.h index 278f8c74f8e..208a6c04b71 100644 --- a/include/nuttx/fs/dirent.h +++ b/include/nuttx/fs/dirent.h @@ -172,6 +172,16 @@ struct fs_smartfsdir_s }; #endif +#ifdef CONFIG_FS_SPIFFS +/* SPIFFS is an SPI-oriented FLASH file system originally by Peter Andersson */ + +struct fs_spiffsdir_s +{ + int16_t block; /* Current block */ + int entry; /* Current entry */ +}; +#endif + #ifdef CONFIG_FS_UNIONFS /* The Union File System can be used to merge to different mountpoints so * that they appear as a single merged directory. @@ -276,6 +286,9 @@ struct fs_dirent_s #ifdef CONFIG_FS_SMARTFS struct fs_smartfsdir_s smartfs; #endif +#ifdef CONFIG_FS_SPIFFS + struct fs_spiffsdir_s spiffs; +#endif #ifdef CONFIG_FS_UNIONFS struct fs_unionfsdir_s unionfs; #endif diff --git a/include/nuttx/fs/ioctl.h b/include/nuttx/fs/ioctl.h index 5d1dd8f9d07..e1e026ae7b2 100644 --- a/include/nuttx/fs/ioctl.h +++ b/include/nuttx/fs/ioctl.h @@ -46,6 +46,7 @@ /**************************************************************************** * Pre-processor Definitions ****************************************************************************/ + /* General ioctl definitions ************************************************/ /* Each NuttX ioctl commands are uint16_t's consisting of an 8-bit type * identifier and an 8-bit command number. All command type identifiers are @@ -240,7 +241,7 @@ * the block with specific debug * command and data. * OUT: None. */ -#define BIOC_GEOMETRY _BIOC(0x000c) /* Used only by BCH to return the +#define BIOC_GEOMETRY _BIOC(0x000c) /* Used only by BCH to return the * geometry of the contained block * driver. * IN: Pointer to writable instance @@ -248,6 +249,18 @@ * to return geometry. * OUT: Data return in user-provided * buffer. */ +#define BIOC_CHECK _BIOC(0x000d) /* Run a consistency check on the + * file system media. + * IN: None + * OUT: None */ +#define BIOC_FORMAT _BIOC(0x000e) /* Force reformatting of media. All + * data will be lost. + * IN: None + * OUT: None */ +#define BIOC_GC _BIOC(0x000f) /* Run garbage collection. + * IN: On entry holds the number + * of bytes to be recovered. + * OUT: None */ /* NuttX MTD driver ioctl definitions ***************************************/ diff --git a/include/sys/statfs.h b/include/sys/statfs.h index e59c8bb1ef8..8211b53ddd0 100644 --- a/include/sys/statfs.h +++ b/include/sys/statfs.h @@ -95,6 +95,7 @@ #define XENIX_SUPER_MAGIC 0x012ff7b4 #define XFS_SUPER_MAGIC 0x58465342 #define _XIAFS_SUPER_MAGIC 0x012fd16d +#define SPIFFS_SUPER_MAGIC 0x20090315 /* NuttX specific file-systems */