LittleFS always calls us with properly aligned and sized reads and
writes so we don't need to keep track of partial blocks (and they were
broken anyway).
Also fixes an issue where we would read too much of a JEDEC NOR page
when doing multi-page reads, though this had no effect except to waste
time since they are designed for it.
Doing the modulus in the loop body made it incorrect for the next loop
if doing a multi-page read. However, LittleFS doesn't seem to do these
with our cache size and filesytem usage.
Fix by only modifying the argument when sending the command.
The `wait_until_device_is_ready` and thus `write_enable` functions both
check for death before doing anything, and the flash access functions
call these before doing anything.
64K blocks are supported by all chips and have a higher bytes-per-second
erase rate than 4K "sectors". The block counts previously were in 64K
units so they don't change.
Also cleans up the calculations a bit.
The address in the `JEDEC_READ_DATA` command for JEDEC NOR chips is
absolute, not relative like for W25NXX. Fix erroneous reads by only
using relative `JEDEC_READ_DATA` address on W25NXX.
Match the chips which use the JEDEC driver in AP_Logger. Other chips are
(or should be) compatible. They are all NOR; NAND chips seem to use a
slightly different protocol described by AP_FILESYSTEM_FLASH_W25NXX.
Studied by copying and pasting the equations into a simple program to
step through each file offset and print the results.
* The equations return nonsense results for small file positions. This
is expected (and noted by the LittleFS author) as block 0 does not
have any pointers according to the LittleFS design; it's entirely
filled with data. Therefore, check and avoid calculating them when we
know we're in the first block.
* This function always assumes we are writing a full block, so
`block_size - block_offset <= nbytes` is always true.
* `block_size == block_offset` can never be true, `block_offset` is at
maximum `block_size-1`. Therefore we can remove the logic to sync when
they are equal and we will always calculate the result to be
`block_size - block_offset`.
We will reserve BOARD_FLASH_SIZE for the internal flash on stm32 flash processors, use HAL_PROGRAM_SIZE_LIMIT_KB in the general code base.
Notable change here is that boards with external flash will start to get features only available with more than 2MB of program storage
The file handle and object are always allocated together. Combine them
to avoid overhead from extra allocations and additional error cases.
Also simplify by using strdup for the pathname.
In Standard C, the first three file descriptors are usually standard in,
out, and error. However, ArduPilot doesn't have a concept of this and
other backends (such as LittleFS) don't bother to reject them.
Remove this logic to simplify implementation and allow use of more open
files.
lseek must return the current file position. Previously, the littlefs
version always returned 0, which broke terrain I/O as it checks that the
position returned is the one it seeked to. Fix to return the current
position, which is correctly returned from littlefs.
This problem was originally and incorrectly diagnosed as an issue with
littlefs seeking past the end of the file, but this functionality works
fine and fixing the incorrect return completely fixes terrain.
Terrain functionality was verified using `TERRAIN_DEBUG` on
KakuteH7Mini-Nand running sim on HW. Terrain data is correctly
downloaded from the GCS and loaded from the filesystem after reboot.
Lua opens scripts to load them into memory, then the logger opens them
after to stream them into the dataflash log. When loading multiple large
Lua scripts from ROMFS, decompression takes a significant amount of
time. This creates the opportunity for the Lua interpreter and logging
threads to both be inside `AP_Filesystem_ROMFS::open()` decompressing a
file.
If this happens, the function can return the same `fd` for two different
calls as the `fd` is chosen before decompression starts, but only marked
as being used after that finishes. The read pointers then stomp on each
other, so Lua loads garbled scripts (usually resulting in a syntax
error) and the logger dumps garbled data.
Fix the issue by locking before searching for a free record (or marking
a record as free). Apply the same fix to directories as well. This
doesn't protect against using the same `fd`/`dirp` from multiple
threads, but that behavior is to be discouraged anyway and is not the
root cause here.
optimize configured defaults on littlefs and address review comments
support lseek() in littlefs in a way that enables terrain to work
rename file and directory structures in littlefs
code in littlefs glue should be C++ rather than C
check for strdup failure on littlefs
use correct read status for nor flash
implement format on littlefs
optimize device calls in littlefs flash usage
check for fileops allowed in littlefs
littlefs optimization and support for mtime
The delays will be canceled on return by the EXPECT_DELAY_MS(3000)
destructor at the start of the function. The current behavior will
unexpectedly cancel delays from higher levels up the stack and is likely
not what was intended.
normally fgets is on a buffered FILE handle. For AP_Filesystem we use
an unbuffered file descriptor. This means we were reading one byte at
a time from the file
this uses lseek to make fgets() much more efficient by reading the max
buffer size at a time in the file
Improves safety of use and clarity of users. Termination is not
included in the reported size to avoid changing user behavior or
misrepresenting the file contents.
Adds virtual directory entries for these virtual filesystems in /
RTL> ftp list
RTL> Listing /
D @MISSION
D @PARAM
D @ROMFS
D @SYS
D APM
D log
V5_BT.dfu 10541
bootlog.txt 297
dataman 350216
message-intervals-chan0.txt 7
Total size 352.60 kByte
ftp list @MISSION
RTL> Listing @MISSION
LIST: OP seq:7 sess:2 opcode:129 req_opcode:3 size:2 bc:0 ofs:0 plen=2 [2]
ftp list @ROMFS
RTL> Listing @ROMFS
bootloader.bin 16448
hwdef.dat 5743
io_firmware.bin 40880
Total size 61.59 kByte
This PR also makes us *much* more lenient in what we accept for looking at virtual filesystems, so
ftp list @SYS
ftp list /@SYS
ftp list @SYS/
ftp list /@SYS/
should all work
Thomas Watson
Peter Barker
Iampete1
Simon Hancock
Eric Katzfey
Richard Allen
ARg
muramura
Andrew Tridgell