This comment converts the underlying IPC used by the UserFS from Unix domain local sockets to UDP LocalHost loopback sockets. The problem with the local sockets is that they do require operations on the top level psuedo-file system inode tree. That tree must be locked during certain traversals such as enumerate mountpoints or enumerating directory entries.

This conversion is unfortunate in the sense that Unix local domain sockets are relatively lightweight.  LocalHost UDP sockets are much heavier weight since they rely on the full UDP stack.  If anyone is up for a complete redesign, then using some shared memory and a POSIX message queue would be lightweight again.

This commit also fixes several bugs that were not testable before the inode tree deadlock.  I cannot say that the logic is 100% stable but it does not have basic functionality.

Squashed commit of the following:

    fs/userfs:  Order locking so that access to the shared I/O buffer is also locked.
    fs/userfs:  Converts to use LocalHost UDP loopback for IPC.

include/nuttx/fs/userfs.h

@@ -55,10 +55,9 @@
  * 1. The UserFS OS support will be instantiated when the UserFS is mounted
  *    based upon the configuration passed in the optional data of the
  *    mount command.
- * 2. The UserFS instance N will be configured to communicate on a Unix
- *    domain local socket with address:  /dev/userfsN where N is the same
- *    value as was when file system was created.  The Unix domain socket
- *    handles both client to server requests and server-to-client responses.
+ * 2. The UserFS server port number will be configured to communicate on a
+ *    LocalHost UDP socket with the server portof 0x83nn where nn is the
+ *    value that was provided when file system was created.
  * 3. The UserFs will receive system file system requests and forward them
  *    on the the MqUfsReqN to the user-space file system server
  *    (userfs_run()).  These requests may be accompanied by additional data in
@@ -66,12 +65,12 @@
  *    created.  This buffer would hold, for example, the data to be
  *    written that would accompany a write request.
  * 4. The user-space logic of userfs_run() listens at the other end of the
- *    Unix domain socket.  It will receive the requests and forward them
+ *    LocalHost socket.  It will receive the requests and forward them
  *    to the user file system implementation via the methods of struct
  *    userfs_operations_s
  * 5. Responses generated by the struct userfs_operations_s method will be
- *    returned to UserFS via the Unix domain socket.
- * 6. The UserFS kernel thread will listen on the Unix local domain socket
+ *    returned to UserFS via the LocalHost socket.
+ * 6. The UserFS kernel thread will listen on the LocalHost socket
  *    and will receive the user file system responses and forward them to
  *    the kernel-space file system client.
  */
@@ -104,19 +103,17 @@
  *   Input:  This function receives an pointer to a read-only instance of
  *           struct userfs_config_s that contains information needed to
  *           configure the UserFS instance.
- *   Output: On success the UserFS N instance is created.  N is non-negative
+ *   Output: On success the UserFS nn instance is created.  nn is non-negative
  *           and will be provided as the IOCTL return value on success.  On
  *           failure, ioctl() will return -1 with the errno variable set to
  *           indicate the cause of the failure.
  */
 
-/* Format statements that should be used in creating Unix domain addresses */
+/* This is the base value of the server port number.  The actual range is
+ * 0x8300 through 0x83ff.
+ */
 
-#define USERFS_SERVER_FMT      "/dev/userver%u"
-#define USERFS_SERVER_MAXLEN   (18)
-
-#define USERFS_CLIENT_FMT      "/dev/uclient%u"
-#define USERFS_CLIENT_MAXLEN   (18)
+#define USERFS_SERVER_PORTBASE 0x8300
 
 /* It looks like the maximum size of a request is 16 bytes.  We will allow a
  * little more for the maximum size of a request structure.
@@ -194,7 +191,7 @@ enum userfs_resp_e
 struct userfs_config_s
 {
   size_t mxwrite;        /* The max size of a write data */
-  int instance;          /* Instance number used to create unique naming */
+  uint16_t portno;       /* The server port number (host order) */
 };
 
 /* This structure identifies the user-space file system operations. */
@@ -561,13 +558,13 @@ int userfs_register(void);
  *
  *   1. It configures and creates the UserFS file system and
  *   2. Mounts the user file system at the provide mount point path.
- *   2. Receives file system requests on the Unix doamin local socket with
- *      address /dev/userfsN where N is the same as above,
+ *   2. Receives file system requests on the LocalHost socket with
+ *      server port 0x83nn where nn is the same as above,
  *   3. Received file system requests are marshaled and dispatch to the
  *      user file system via callbacks to the operations provided by
  *      "userops", and
- *   3. Returns file system responses generated by the callbacks via the
- *      same Unix domain local socket.
+ *   3. Returns file system responses generated by the callbacks to the
+ *      LocalHost client socket.
  *
  *   NOTE:  This is a user function that is implemented as part of the
  *   NuttX C library and is intended to be called by appliation logic.