code review from Josh and Robert

          --- old/usr/src/man/man9/vmem.9.man.txt
          +++ new/usr/src/man/man9/vmem.9.man.txt

   1    1  VMEM(9)                    Device Driver Interfaces                    VMEM(9)
   2    2  
   3    3  NAME
   4    4       vmem - virtual memory allocator
   5    5  
   6    6  DESCRIPTION
   7    7     Overview
   8    8       An address space is divided into a number of logically distinct pieces,
   9    9       or arenas: text, data, heap, stack, and so on.  Within these arenas we
  10   10       often subdivide further; for example, we use heap addresses not only for
  11   11       the kernel heap (kmem_alloc() space), but also for DVMA, bp_mapin(),
  12   12       /dev/kmem, and even some device mappings.
  13   13  
  14   14       The kernel address space, therefore, is most accurately described as a
  15   15       tree of arenas in which each node of the tree imports some subset of its
  16   16       parent.  The virtual memory allocator manages these arenas and supports

  17   17       their natural hierarchical structure.
  18   18  
  19   19     Arenas
  20   20       An arena is nothing more than a set of integers.  These integers most
  21   21       commonly represent virtual addresses, but in fact they can represent
  22   22       anything at all.  For example, we could use an arena containing the
  23   23       integers minpid through maxpid to allocate process IDs.  For uses of this
  24   24       nature, prefer id_space(9F) instead.
  25   25  
  26   26       vmem_create() and vmem_destroy() create and destroy vmem arenas.  In
  27      -     order to differentiate between arenas used for adresses and arenas used
       27 +     order to differentiate between arenas used for addresses and arenas used
  28   28       for identifiers, the VMC_IDENTIFIER flag is passed to vmem_create().
  29   29       This prevents identifier exhaustion from being diagnosed as general
  30   30       memory failure.
  31   31  
  32   32     Spans
  33   33       We represent the integers in an arena as a collection of spans, or
  34   34       contiguous ranges of integers.  For example, the kernel heap consists of
  35   35       just one span: [kernelheap, ekernelheap).  Spans can be added to an arena
  36      -     in two ways: explicitly, by vmem_add(), or implicitly, by importing, as
       36 +     in two ways: explicitly, by vmem_add(); or implicitly, by importing, as
  37   37       described in Imported Memory below.
  38   38  
  39   39     Segments
  40   40       Spans are subdivided into segments, each of which is either allocated or
  41   41       free.  A segment, like a span, is a contiguous range of integers.  Each
  42   42       allocated segment [addr, addr + size) represents exactly one
  43   43       vmem_alloc(size) that returned addr.  Free segments represent the space
  44   44       between allocated segments.  If two free segments are adjacent, we
  45   45       coalesce them into one larger segment; that is, if segments [a, b) and
  46   46       [b, c) are both free, we merge them into a single segment [a, c).  The

  47   47       segments within a span are linked together in increasing-address order so
  48   48       we can easily determine whether coalescing is possible.
  49   49  
  50   50       Segments never cross span boundaries.  When all segments within an
  51   51       imported span become free, we return the span to its source.
  52   52  
  53   53     Imported Memory
  54   54       As mentioned in the overview, some arenas are logical subsets of other
  55   55       arenas.  For example, kmem_va_arena (a virtual address cache that
  56   56       satisfies most kmem_slab_create() requests) is just a subset of
  57   57       heap_arena (the kernel heap) that provides caching for the most common
  58   58       slab sizes.  When kmem_va_arena runs out of virtual memory, it imports
  59   59       more from the heap; we say that heap_arena is the vmem source for
  60   60       kmem_va_arena. vmem_create() allows you to specify any existing vmem
  61   61       arena as the source for your new arena.  Topologically, since every arena
  62   62       is a child of at most one source, the set of all arenas forms a
  63   63       collection of trees.
  64   64  
  65   65     Constrained Allocations
  66   66       Some vmem clients are quite picky about the kind of address they want.
  67   67       For example, the DVMA code may need an address that is at a particular
  68   68       phase with respect to some alignment (to get good cache coloring), or
  69   69       that lies within certain limits (the addressable range of a device), or
  70   70       that doesn't cross some boundary (a DMA counter restriction) -- or all of
  71   71       the above.  vmem_xalloc() allows the client to specify any or all of
  72   72       these constraints.
  73   73  
  74   74     The Vmem Quantum
  75   75       Every arena has a notion of `quantum', specified at vmem_create() time,
  76   76       that defines the arena's minimum unit of currency.  Most commonly the
  77   77       quantum is either 1 or PAGESIZE, but any power of 2 is legal.  All vmem
  78   78       allocations are guaranteed to be quantum-aligned.
  79   79  
  80   80     Relationship to the Kernel Memory Allocator
  81   81       Every kmem cache has a vmem arena as its slab supplier.  The kernel
  82   82       memory allocator uses vmem_alloc() and vmem_free() to create and destroy
  83   83       slabs.
  84   84  
  85   85  SEE ALSO
  86   86       id_space(9F), vmem_add(9F), vmem_alloc(9F), vmem_contains(9F),
  87   87       vmem_create(9F), vmem_walk(9F)
  88   88  
  89   89  
  90   90       Jeff Bonwick and Jonathan Adams, "Magazines and vmem: Extending the Slab
  91   91       Allocator to Many CPUs and Arbitrary Resources.", Proceedings of the 2001
  92   92       Usenix Conference, http://www.usenix.org/event/usenix01/bonwick.html.
  93   93  
  94   94  illumos                        January 18, 2017                        illumos

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX