172 Commits

Include renames

Author	SHA1	Message	Date
Peter Barker	3708936c62	global: correct shebang lines to refernce python3 ... also remove the odd encoding line which is not required in Py3	2025-07-22 10:45:57 +10:00
Peter Barker	51c956d9aa	AP_Filesystem: disallow setting of readonly/internal parameters via mavftp	2025-05-12 13:01:02 +10:00
Thomas Watson	13519aaabb	AP_Filesystem: LittleFS: drop LFS_FLASH_BLOCKS_PER_BLOCK support ... Not needed for our flashes. Cleans up the code a bit.	2025-04-03 14:06:02 +11:00
Thomas Watson	86e4487048	AP_Filesystem: LittleFS: simplify flash read and program functions ... LittleFS always calls us with properly aligned and sized reads and writes so we don't need to keep track of partial blocks (and they were broken anyway). Also fixes an issue where we would read too much of a JEDEC NOR page when doing multi-page reads, though this had no effect except to waste time since they are designed for it.	2025-04-03 14:06:02 +11:00
Thomas Watson	f5f53b586c	AP_Filesystem: LittleFS: fix JEDEC NOR page read fix ... Doing the modulus in the loop body made it incorrect for the next loop if doing a multi-page read. However, LittleFS doesn't seem to do these with our cache size and filesytem usage. Fix by only modifying the argument when sending the command.	2025-04-03 14:06:02 +11:00
Thomas Watson	cc905249a7	AP_Filesystem: LittleFS: remove redundant death checks ... The `wait_until_device_is_ready` and thus `write_enable` functions both check for death before doing anything, and the flash access functions call these before doing anything.	2025-04-03 14:06:02 +11:00
Thomas Watson	064f4998e7	AP_Filesystem: LittleFS: always wait for device before initing	2025-04-03 14:06:02 +11:00
Thomas Watson	d6bda7362f	AP_Filesystem: LittleFS: tune for JEDEC NOR ... Decreasing metadata size from the default of a full block reduces large pauses that cause log drops.	2025-03-04 11:46:15 +11:00
Thomas Watson	7e90503245	AP_Filesystem: LittleFS: use 64K blocks for JEDEC_NOR ... 64K blocks are supported by all chips and have a higher bytes-per-second erase rate than 4K "sectors". The block counts previously were in 64K units so they don't change. Also cleans up the calculations a bit.	2025-03-04 11:46:15 +11:00
Thomas Watson	abea5309f3	AP_Filesystem: LittleFS: add erase counts to debug ... Also fix that we are talking about page writes instead of block writes.	2025-03-04 11:46:15 +11:00
Thomas Watson	25792f9997	AP_Filesystem: LittleFS: fix JEDEC NOR page reads ... The address in the `JEDEC_READ_DATA` command for JEDEC NOR chips is absolute, not relative like for W25NXX. Fix erroneous reads by only using relative `JEDEC_READ_DATA` address on W25NXX.	2025-03-04 11:46:15 +11:00
Thomas WatsonandAndy Piper	f5858ebca0	AP_Filesystem: LittleFS: support GD25Q16E flash ... This really needs to be shared with AP_Logger and AP_FlashIFace. Co-authored-by: Andy Piper <github@andypiper.com>	2025-03-04 11:46:15 +11:00
Thomas Watson	3532d6d5ad	AP_Filesystem: AP_FILESYSTEM_FLASH_W25QXX -> AP_FILESYSTEM_FLASH_JEDEC_NOR ... Match the chips which use the JEDEC driver in AP_Logger. Other chips are (or should be) compatible. They are all NOR; NAND chips seem to use a slightly different protocol described by AP_FILESYSTEM_FLASH_W25NXX.	2025-03-04 11:46:15 +11:00
Thomas Watson	89cc25ca81	AP_Filesystem: LittleFS: clarify fsync bytes logic ... Studied by copying and pasting the equations into a simple program to step through each file offset and print the results. * The equations return nonsense results for small file positions. This is expected (and noted by the LittleFS author) as block 0 does not have any pointers according to the LittleFS design; it's entirely filled with data. Therefore, check and avoid calculating them when we know we're in the first block. * This function always assumes we are writing a full block, so `block_size - block_offset <= nbytes` is always true. * `block_size == block_offset` can never be true, `block_offset` is at maximum `block_size-1`. Therefore we can remove the logic to sync when they are equal and we will always calculate the result to be `block_size - block_offset`.	2025-03-03 09:24:50 -05:00
Thomas Watson	9cc6e25fd4	AP_Filesystem: LittleFS: inline fsync bytes calculation ... Rename _write_offset to file_pos and blocksize to block_size. Also remove the unused return value.	2025-03-03 09:24:50 -05:00
Peter Barker	6338d10b5f	global: create and use HAL_PROGRAM_SIZE_LIMIT_KB ... We will reserve BOARD_FLASH_SIZE for the internal flash on stm32 flash processors, use HAL_PROGRAM_SIZE_LIMIT_KB in the general code base. Notable change here is that boards with external flash will start to get features only available with more than 2MB of program storage	2025-02-26 10:06:39 +11:00
Thomas Watson	71be1194cb	AP_Filesystem: FATFS: clean up error handling ... Remove redundant checks of the stream, cases that could never occur (both before and after combining allocations), and clarify errno propagation.	2025-02-25 11:17:50 +11:00
Thomas Watson	b9cad62acf	AP_Filesystem: FATFS: combine file allocations ... The file handle and object are always allocated together. Combine them to avoid overhead from extra allocations and additional error cases. Also simplify by using strdup for the pathname.	2025-02-25 11:17:50 +11:00
Thomas Watson	e696a83371	AP_Filesystem: LittleFS: avoid leaking directory on close error	2025-02-22 09:46:33 -05:00
Thomas Watson	0246b03424	AP_Filesystem: LittleFS: use NEW_NOTHROW instead of new ... Unfortunate CI did not catch this...	2025-02-22 09:46:33 -05:00
Thomas Watson	7a6861e28d	AP_Filesystem: FATFS: drop tty check logic ... In Standard C, the first three file descriptors are usually standard in, out, and error. However, ArduPilot doesn't have a concept of this and other backends (such as LittleFS) don't bother to reject them. Remove this logic to simplify implementation and allow use of more open files.	2025-02-18 12:33:37 +11:00
Thomas Watson	ec7ece3a4d	AP_Filesystem: littlefs: drop singleton	2025-02-17 16:33:51 +11:00
Thomas Watson	ed2c42ede2	AP_Filesystem: new API for performing optimal fsyncs	2025-02-17 16:33:51 +11:00
Thomas Watson	182646a7e4	AP_Filesystem: littlefs: debug warn of misaligned fsyncs	2025-02-17 16:33:51 +11:00
Thomas Watson	fb8f3488d5	AP_Filesystem: littlefs: fix lseek ... lseek must return the current file position. Previously, the littlefs version always returned 0, which broke terrain I/O as it checks that the position returned is the one it seeked to. Fix to return the current position, which is correctly returned from littlefs. This problem was originally and incorrectly diagnosed as an issue with littlefs seeking past the end of the file, but this functionality works fine and fixing the incorrect return completely fixes terrain. Terrain functionality was verified using `TERRAIN_DEBUG` on KakuteH7Mini-Nand running sim on HW. Terrain data is correctly downloaded from the GCS and loaded from the filesystem after reboot.	2025-02-14 13:29:12 +01:00
Thomas Watson	63afcae8a7	AP_Filesystem: ROMFS: fix open race conditions ... Lua opens scripts to load them into memory, then the logger opens them after to stream them into the dataflash log. When loading multiple large Lua scripts from ROMFS, decompression takes a significant amount of time. This creates the opportunity for the Lua interpreter and logging threads to both be inside `AP_Filesystem_ROMFS::open()` decompressing a file. If this happens, the function can return the same `fd` for two different calls as the `fd` is chosen before decompression starts, but only marked as being used after that finishes. The read pointers then stomp on each other, so Lua loads garbled scripts (usually resulting in a syntax error) and the logger dumps garbled data. Fix the issue by locking before searching for a free record (or marking a record as free). Apply the same fix to directories as well. This doesn't protect against using the same `fd`/`dirp` from multiple threads, but that behavior is to be discouraged anyway and is not the root cause here.	2025-02-01 16:51:49 +11:00
Andy Piper	c971bb6850	AP_Filesystem: allow for logical blocks bigger than physical blocks in littlefs. ... optimize configured defaults on littlefs and address review comments support lseek() in littlefs in a way that enables terrain to work rename file and directory structures in littlefs code in littlefs glue should be C++ rather than C check for strdup failure on littlefs	2025-01-21 11:10:31 +11:00
Thomas Watson	90ac6bdc9e	AP_Filesystem: lock littlefs between opendir and closedir ... Avoids issues with (alleged) corruption if writes happen between.	2025-01-21 11:10:31 +11:00
Andy Piper	f22e1ff38b	AP_Filesystem: support mutiple reads/writes in littlefs ... improve performance by avoiding block validation on writes	2025-01-21 11:10:31 +11:00
Andy Piper	0c541b72fd	AP_Filesystem: provide SITL implementation for littlefs	2025-01-21 11:10:31 +11:00
Andy Piper	8a992740f5	AP_Filesystem: add littlefs singleton for sync block use	2025-01-21 11:10:31 +11:00
Andy Piper	acf1b40dca	AP_Filesystem: reduce metadata_max on W25N flash	2025-01-21 11:10:31 +11:00
Andy Piper	6390285598	AP_Filesystem: add littlefs based filesystem support ... use correct read status for nor flash implement format on littlefs optimize device calls in littlefs flash usage check for fileops allowed in littlefs littlefs optimization and support for mtime	2025-01-21 11:10:31 +11:00
Iampete1	78d9e5a875	AP_Filesystem: Don't try and rename between backends	2025-01-05 13:33:42 +11:00
Thomas Watson	c57ac3b916	AP_Filesystem: FATFS: remove redundant delay expectations ... The delays will be canceled on return by the EXPECT_DELAY_MS(3000) destructor at the start of the function. The current behavior will unexpectedly cancel delays from higher levels up the stack and is likely not what was intended.	2024-12-31 11:20:21 +11:00
Simon Hancock	17f18f7fc5	AP_Filesystem: Add details of withdefaults query param to README	2024-12-15 08:43:42 +11:00
Eric Katzfey	7f4a20981a	AP_Filesystem: Added conditional compiling to errno declarations. QURT HAL already has this in errno.h.	2024-11-17 20:38:25 +11:00
Richard Allen	9696081a85	AP_Filesystem: support port SITL to OpenBSD	2024-11-06 14:17:31 +11:00
Thomas Watson	380d8ca6fc	AP_Filesystem: remove dependency on tmpnam ... tmpnam is never linked and not necessary, and naming it/redefining it introduces problems on platforms that do have a definition.	2024-09-24 22:49:18 +10:00
Peter Barker	a8cc071d62	AP_Filesystem: rearrange to put mission filesystem last ... We're developing loops in our config files, move include to after base FILESYSTEM_READING/FILESYSTEM_WRITING defines	2024-09-24 11:59:13 +10:00
ARg	8204ff4d7c	AP_Filesystem: fix esp32 function datatypes ... Make them match the superclass and the header file.	2024-09-24 09:00:29 +10:00
muramura	e86d5e113f	AP_Filesystem: BOOL for binary types	2024-07-26 20:12:05 +10:00
Andrew Tridgell	f8ce4183b8	AP_Filesystem: support QURT with posix filesystem ... some features disabled	2024-07-12 15:56:48 +10:00
Andrew Tridgell	75ebf96adb	AP_Filesystem: allow for filesystem with no dtype	2024-07-11 09:25:05 +10:00
Andrew Tridgell	c2e52af1e2	AP_Filesystem: make fgets() much more efficient ... normally fgets is on a buffered FILE handle. For AP_Filesystem we use an unbuffered file descriptor. This means we were reading one byte at a time from the file this uses lseek to make fgets() much more efficient by reading the max buffer size at a time in the file	2024-07-11 09:24:38 +10:00
Andrew Tridgell	a241b13f45	AP_Filesystem: use NEW_NOTHROW for new(std::nothrow)	2024-06-04 09:20:21 +10:00
Thomas Watson	0ca37387be	AP_Filesystem: guarantee load_file() data is null-terminated ... Improves safety of use and clarity of users. Termination is not included in the reported size to avoid changing user behavior or misrepresenting the file contents.	2024-05-04 10:15:44 +10:00
Andrew Tridgell	8763084a9e	AP_Filesystem: avoid decompressing in directory listing ... this makes ROMFS much more efficient	2024-03-19 19:49:47 +11:00
Iampete1	8d7052887c	AP_Filesystem: fixup AP_FILESYSTEM_FILE_WRITING_ENABLED comment	2024-03-12 08:52:38 +11:00
Peter Barker	2cb5f434fd	AP_Filesystem: don't show directory entries for empty @ filesystems	2024-02-28 18:47:22 +11:00