Documentation: migrate "SMP Critical Sections" from wiki

links: https://cwiki.apache.org/confluence/display/NUTTX/SMP+Critical+Sections

Documentation/implementation/critical_sections.rst

@@ -0,0 +1,163 @@
+=================
+Critical Sections
+=================
+
+Single CPU Critical Sections
+============================
+
+OS Interfaces
+-------------
+
+Before we talk about SMP Critical Sections let's first review the internal OS
+interfaces avaiable and what they do in the single CPU case:
+
+* ``up_irq_save()`` (and its companion, ``up_irq_restore()``). These simple
+  interfaces just enable and disable interrupts globally. This is the simplest
+  way to establish a critical section in the single CPU case. It does have
+  side-effects to real-time behavior as discussed elsewhere.
+
+* ``up_irq_save()`` should never be called directly, however. Instead, the wrapper
+  macros enter_critical_section() (and its companion ``leave_critical_section()``)
+  or ``spin_lock_irqsave()`` (and ``spin_unlock_irqrestore()``) should be used.
+  In the single CPU case, these macros are defined to be simply ``up_irq_save()``
+  (or ``up_irq_save()``). Rather than being called directly, they should always
+  be called indirectly through these macros so that the code will function in the
+  SMP environment as well.
+
+* Finally, there is ``sched_lock()`` (and ``sched_unlock()``) that disable (and
+  enable) pre-emption. That is, ``sched_lock()`` will lock your kernel thread in
+  place and prevent other tasks from running. Interrupts are still enabled, but
+  other tasks cannot run.
+
+
+Using sched_lock() for Critical Sections – **DON'T**
+----------------------------------------------------
+
+In the single CPU case, ``sched_lock()`` can do a pretty good job of establishing a
+critical section too. After all, if no other tasks can run on the single CPU,
+then that task has pretty much exclusive access to all resources (provided that
+those resources are not shared with interrupt handlers). However, ``sched_lock()``
+must never be used to establish a critical section because it does not work the
+same way in the SMP case. In the SMP case, locking the scheduer does not provide
+any kind of exclusive access to resources. Tasks running on other CPUs are still
+free to do whatever they wish.
+
+SMP Critical Sections
+=====================
+
+``up_irq_save()`` and ``up_irq_restore()``
+------------------------------------------
+
+As mentioned, ``up_irq_save()`` and ``up_irq_restore()`` should never be called
+directly. That is because the behavior is different in multiple CPU systems. In
+the multiple CPU case, these functions only enable (or disable) interrupts on the
+local CPU. They have no effect on interrupts in the other CPUs and hence really
+accomplish very little. Certainly they do not provide a critical section in any
+sense.
+
+``enter_critical_section()`` and ``leave_critical_section()``
+-------------------------------------------------------------
+
+**spinlocks**
+
+In order to establish a critical section, we also need to employ spinlocks. Spins
+locks are simply loops that execute in one processor. If processor A sets spinlock
+x, then processor B would have to wait for the spinlock like:
+
+.. code-block:: C
+
+  while (test_and_set(x))
+   {
+   }
+
+Where test and set is an atomic operation that sets the value of a memory location
+but also returns its previous value. Here we are talking about atomic in terms of
+memory bus operations: The testing and setting of the memory location must be atomic
+with respect to other bus operations. Special hardware support of some kind is
+necessary to implement ``test_and_set()`` logic.
+
+When Task A released the lock x, Task B will successfully take the spinlock and
+continue.
+
+**Implementation**
+
+Without going into the details of the implementation of ``enter_critical_section()``
+suffice it to say that it (1) disables interrupts on the local CPU and (2) uses
+spinlocks to assure exclusive access to a code sequence across all CPUs.
+
+NOTE that a critical section is indeed created: While within the critical section,
+the code does have exclusive access to the resource being protected. However the
+behavior is really very different:
+
+* In the single CPU case, disable interrupts stops all possible activity from any
+  other task. The single CPU becomes single threaded and un-interruptible.
+* In the SMP case, tasks continue to run on other CPUs. It is only when those other
+  tasks attempt to enter a code sequence protected by the critical section that those
+  tasks on other CPUs will be stopped. They will be stopped waiting on a spinlock.
+
+``spin_lock_irqsave()`` and ``spin_unlock_irqrestore()``
+--------------------------------------------------------
+
+**Generic Interrupt Controller (GIC)**
+
+ARM provides a special, optional sub-system called MPCore that provides
+multi-core support. One MPCore component is the Generic Interrupt Controller
+or GIC. The GIC supports 16 inter-processor interrupts and is a key component for
+implementing SMP on those platforms. The are called Software Generated Interrupts
+or SGIs.
+
+One odd behavior of the GIC is that the SGIs cannot be disabled (at least not
+using the standard ARM global interrupt disable logic). So disabling local
+interrupts does not prevent these GIC interrupts.
+
+This causes numerous complexities and significant overhead in establishing a
+critical section.
+
+**ARMv7-M NVIC**
+
+The GIC is available in all recent ARM architectures. However, most embedded
+ARM7-M multi-core CPUs just incorporate the inter-processor interrupts as a
+normal interrupt that is mask-able via the NVIC (each CPU will have its own NVIC).
+
+This means in those cases, the critical section logic can be greatly simplified.
+
+**Implementation**
+
+For the case of the GIC with no support for disabling interrupts,
+``spin_lock_irqsave()`` and ``spin_unlock_irqstore()`` are equivalent to
+``enter_critical_section()`` and ``leave_critical_section()``. In is only in the
+case where inter-processor interrupts can be disabled that there is a difference.
+
+In that case, ``spin_lock_irqsave()`` will disable local interrupts and take
+a spinlock. This is really very simple and efficient implementation of a critical
+section.
+
+There are two important things to note, however:
+
+* The logic within this critical section must never suspend! For example, if
+  code were to call ``spin_lock_irqsave()`` then ``sleep()``, then the sleep
+  would occur with the spinlock in the lock state and the whole system could
+  be blocked. Rather, ``spin_lock_irqsave()`` can only be used with straight
+  line code.
+
+* This is a different critical section than the one established via
+  ``enter_critical_section()``. Taking one critical section, does not prevent
+  logic on another CPU from taking the other critical section and the result
+  is that you make not have the protection that you think you have.
+
+``sched_lock()`` and ``sched_unlock()``
+---------------------------------------
+
+Other than some details, the SMP ``sched_lock()`` works much like it does in
+the single CPU case. Here are the caveats:
+
+* As in the single CPU case, the case that calls ``sched_lock()`` is locked
+  in place and cannot be suspected.
+
+* However, tasks will continue to run on other CPUs so ``sched_lock()`` cannot
+  be used as a critical section.
+
+* Tasks on other CPUs are also locked in place. However, they may opt to suspend
+  themselves at any time (say, via ``sleep()``). In that case, only the CPU's
+  IDLE task will be permitted to run.
+

Documentation/implementation/index.rst

@@ -7,3 +7,4 @@ Implementation Details
    :caption: Contents:
    
    processes_vs_tasks.rst
+   critical_sections.rst