add clarifications; add Free call to swapper

src/ch10-00-swap-overview.md

@@ -134,6 +134,7 @@ The `swapper` must also implement the following opcodes:
 - `ReadFromSwap`: a memory message that retrieves & decrypts a page from swap, and copies it to the lent page. The `offset` & `valid` fields encode the original PID and virtual address.
 - `AllocateAdvisory`: a scalar message that informs the swapper that a page in free RAM was allocated to a given PID and virtual address
 - `Trim`: a request from the kernel to free up N pages. Normally the kernel would not call this, as the swapper should be pre-emptively clearing space, but it is provided as a last-ditch method in case of an OOM.
+- `Free`: a scalar message that informs the swapper that a page was de-allocated by a process.
 
 #### Flags and States
 
@@ -152,10 +153,16 @@ When `swap` is enabled, the flags have the following meaning:
 
 From the standpoint of memory management, a page can only have the following states:
 
-- Allocated: `V` set, `P` may not be set
-- Fault: No flags are set, or `S` is set and `V` is not set
-- Swapped: `P` is set. `V` is *not* set. `S` may also not be set.
-- Reserved: `V` is *not* set, and at least one other flag is set except for `S`
+- `Allocated`: `V` set, `P` may not be set
+- `Fault`: No flags are set, or `S` is set and `V` is not set
+- `Swapped`: `P` is set. `V` is *not* set. `S` may also not be set. Upon access to this page, the kernel allocates a resident page and calls `ReadFromSwap` to fill it. The page will move to the `Allocated` state on conclusion.
+- `Reserved`: `V` is *not* set, `P` is *not* set, and at least one other flag is set except for `S`. A kernel allocates a resident page and zeros it. The page will move to the `Allocated` state on conclusion.
+
+Pages go from `Allocated` to `Swapped` based on the `swapper` observing that the kernel is low on memory, and calling a series of `EvictPage` calls to free up memory. It is always assumed that the kernel can allocate memory when necessary; as a last ditch the kernel can attempt to call `Trim` on the swapper, but this should only happen in extreme cases of memory pressure.
+
+Pages go from `Allocated` to `Reserved` when a process unmaps memory.
+
+When the `swapper` runs out of space, `WriteToSwap` panics with an OOM.
 
 #### RegisterSwapper Syscall
 
@@ -168,7 +175,7 @@ After registration, the kernel sends a message to the `swapper` with the locatio
 `EvictPage` is a syscall that only the `swapper` is allowed to call. It is a scalar `send` message, which contains the PID and address of the page to evict. Upon receipt, the kernel will:
 
 - Change into the requested PID's address space
-- Lookup teh physical address of the evicted page
+- Lookup the physical address of the evicted page
 - Clear the `V` bit and set the `P` bit of the evicted page's PTE
 - Change into the swapper's address space
 - Mutably lend the evicted physical page to the swapper with a `WriteToSwap` message

src/temp_notes.md

@@ -0,0 +1,68 @@
+
+# OLD NOTES
+
+Here is the flow for how page faults happen in a `swap` enabled kernel:
+
+1. Catch Store, Load or Instruction page fault (inside `kernel/src/arch/riscv/irq.c/trap_handler`)
+2. `ensure_page_exists_inner`:
+   1. checks if the address makes sense
+   2. If the PTE is marked invalid, check the physical address.
+      1. Do a single-pass, bottom-up search through memory for the first unallocated physical page.
+         1. If the physical address is not in the swap space:
+            1. Zero the page, map it, flush the MMU
+            2. Stick an `AllocAdvisory` message to the `swapper`
+         2. If the physical address is in the swap space:
+            1. Zero the page, map it, flush the MMU
+            2. Call `swapper` with `MoveToMain`, with this allocated page lended as an argument
+            3. Swapper decrypts the requested page and copies it to the lent page
+            4. Swapper decrements physical kernel memory tracker
+            5. Execution returns to the kernel
+3. Return from the page fault and resume execution
+4. `swapper` gains a quantum and handles `AllocAdvisory`
+   1. Kernel free memory tracker is decremented
+   2. If the free memory amount falls below the low water mark, find the least recently used pages, and use the `UncommitPage` syscall.
+      1. `EvictPage` is called with the new physical address for the data to be moved
+      2. Kernel notes the physical address of the page to be uncommitted
+      3. Kernel modifies the target process's PT so that the virtual page as invalid but points to the physical address
+      4. Kernel maps the uncommit physical page into the swapper's memory space as the return result of the `UncommitPage` syscall
+      5. The uncommit page is encrypted
+
+      6. Decrement the amount of memory available, 
+      7. If (1) fails, send a `MoveToSwap` message to the `swapper`
+      8. Activate the `swapper` context
+      9. `swapper` searches through swap for the first unallocated physical page
+      10. `swapper` calls `EvictPage` on the kernel
+
+
+For contrast, here is the flow for how pages faults happen in a kernel without `swap` enabled:
+
+1. Catch Store or Load page fault (inside `kernel/src/arch/riscv/irq.c/trap_handler`)
+2. If the PTE is marked invalid call `ensure_page_exists_inner`:
+   1. checks if the address makes sense
+   2. If it is demand-paged, allocate a new physical page with `kernel/src/mem.rs/MemoryManager::alloc_page()` -- this does a single-pass, bottom-up search through memory for the first unallocated physical page.
+   3. Zero the page, map it, flush the MMU
+3. Return from the page fault and resume execution
+
+# Random notes
+
+general idea of how to invoke the swapper:
+
+const HANDLER_PID: PID = PID::new(2);
+SystemServices::with_mut(|ss| {
+    // Disable all other IRQs and redirect into userspace
+    arch::irq::disable_all_irqs();
+    ss.make_callback_to(HANDLER_PID,
+                        function_address as *const usize,
+                        arg_0,
+                        arg_1,
+    )
+}).expect("couldn't switch to handler");
+ArchProcess::with_current_mut(|process| {
+    crate::arch::syscall::resume(current_pid().get() == 1, process.current_thread())
+})
+
+Make a special version of this, for the swapper to return to (i.e., allocate a new magic address for swapper return):
+https://github.com/betrusted-io/xous-core/blob/8622fbdf7a1d63924e9fc5fe86ef69e35a3122b1/kernel/src/arch/riscv/irq.rs#L303-L308
+
+This will need patching to go to that magic address:
+https://github.com/betrusted-io/xous-core/blob/8622fbdf7a1d63924e9fc5fe86ef69e35a3122b1/kernel/src/services.rs#L649-L670