1 # CDDL HEADER START
   2 #
   3 # The contents of this file are subject to the terms of the
   4 # Common Development and Distribution License (the "License").
   5 # You may not use this file except in compliance with the License.
   6 #
   7 # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
   8 # or http://www.opensolaris.org/os/licensing.
   9 # See the License for the specific language governing permissions
  10 # and limitations under the License.
  11 #
  12 # When distributing Covered Code, include this CDDL HEADER in each
  13 # file and include the License file at usr/src/OPENSOLARIS.LICENSE.
  14 # If applicable, add the following below this CDDL HEADER, with the
  15 # fields enclosed by brackets "[]" replaced with your own identifying
  16 # information: Portions Copyright [yyyy] [name of copyright owner]
  17 #
  18 # CDDL HEADER END
  19 #
  20 #
  21 # Copyright 2008 Sun Microsystems, Inc.  All rights reserved.
  22 # Use is subject to license terms.
  23 #
  24 ================================================================================
  25 
  26 TITLE:  Kstats Specification for SDBC
  27 
  28 DATE:   10-28-2002
  29 
  30 AUTHOR: Chris Juhasz (chris.juhasz@sun.com)
  31 
  32 LOCATION: src/uts/common/ns/sdbc/cache_kstats_readme.txt
  33 ================================================================================
  34 
  35 The existing sd_stat cache statistical reporting mechanism has been expanded  
  36 with the kstat library reporting mechanism. The existing mechanism will probably
  37 eventually be phased out.  In general the statistics have fallen 
  38 into two general categories - "global" and "cd." The global stats reflect gross 
  39 behavior over all cached volumes, while "cd" stats reflect behavior particular 
  40 to each cached volume (or cache descriptor).
  41 
  42 The sdbc module makes use of two types of kstats.  For generic statistic
  43 reporting, "regular" kstat_named_t type kstats are used.  For timing-specific
  44 reporting, sdbc relies on the kstat_io_t type.
  45 
  46 For more information on kstats, see [1] in the References section.
  47 
  48 1.0 NAMING:
  49 ===========
  50 The names for the sdbc kstats are defined in src/uts/common/ns/sdbc/sd_misc.h
  51 
  52 2.0 REGULAR KSTATS:
  53 ===================
  54 The following are kstats of type kstat_named_t, used to gather generic
  55 statistics.
  56 
  57 These make use of the original statistics gathering mechanism for sdbc, 
  58 _sd_stats_t and _sd_shared_t structs, defined in 
  59 src/uts/common/ns/sdbc/sd_bcache.h.  The _sd_stats_t structure tracks 
  60 statistics that are global to the entire cache, while the _sd_shared_t struct 
  61 is used to track statistics particular to a cache descriptor (cd). 
  62 
  63 2.1 GLOBAL KSTATS:
  64 ~~~~~~~~~~~~~~~~~~
  65 This global kstat represents statistics which reflect the state of the entire
  66 cache, summed over all cache descriptors.
  67 
  68 2.1.1 Field Definitions:
  69 ------------------------
  70 The "global" kstat corresponds to fields in the _sd_stats_t structure.  The
  71 following table maps the name of the kstat field to its equivalent field in
  72 the _sd_stats_t structure, also providing a description where appropriate.
  73  
  74 KSTAT FIELD             _sd_stats_t     DESCRIPTION
  75 -----------             -----------     -----------
  76 sdbc_count              st_count        - number of opens for device
  77 sdbc_loc_count          st_loc_count    - number of open devices
  78 sdbc_rdhits             st_rdhits       - number of read hits
  79 sdbc_rdmiss             st_rdmiss       - number of read misses
  80 sdbc_wrhits             st_wrhits       - number of write hits
  81 sdbc_wrmiss             st_wrmiss       - number of write misses
  82 sdbc_blksize            st_blksize      - cache block size (in bytes)
  83 
  84 /* I'm not very sure what the next three fields track--we might take them out */
  85 sdbc_lru_blocks         st_lru_blocks
  86 sdbc_lru_noreq          st_lru_noreq
  87 sdbc_lru_req            st_lru_req
  88 
  89 sdbc_wlru_inq           st_wlru_inq     - number of write blocks
  90 sdbc_cachesize          st_cachesize    - cache size (in bytes)
  91 sdbc_numblocks          st_numblocks    - cache blocks
  92 sdbc_num_shared         MAXFILES*2      - number of shared structures (one for
  93                                           each cached volume)
  94                                           This number dictates the maximum 
  95                                           index size for shared stats and 
  96                                           names given below.
  97 sdbc_destaged           st_destaged     - number of bytes destaged to disk
  98                                           (flushed from the cache to disk).
  99 sdbc_wrcancelns         st_wrcancelns   - number of write cancellations
 100                                           (writes to cached blocks that are 
 101                                           already dirty).
 102 sdbc_nodehints          ---             - node hints (such as wrthru/nowrthru)
 103 
 104 All fields are read-only and are of type KSTAT_DATA_ULONG. Note that the
 105 "sdbc_wrcancelns" and "sdbc_destaged" are new, and have also been added to the
 106 _sd_stats_t struct.
 107 
 108 2.1.2 Naming characteristics:
 109 -----------------------------
 110 module:         SDBC_KSTAT_MODULE       "sdbc"  
 111 class:          SDBC_KSTAT_CLASS        "storedge"
 112 name:           SDBC_KSTAT_GSTATS       "global"
 113 instance #:     0 
 114 
 115 
 116 2.2 KSTATS (PER CACHE DESCRIPTOR):
 117 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 118 These "cd" kstats present statistics which reflect the state of a single cache 
 119 descriptor.  One of these kstats exists for each open cache descriptor.
 120 
 121 2.2.1 Field Definitions:
 122 ------------------------
 123 The "cd" kstats correspond to fields in the _sd_shared_t structure.  The
 124 following table maps the name of the kstat field to its equivalent field in
 125 the _sd_shared_t structure, also providing a description where appropriate.
 126 
 127 KSTAT FIELD             _sd_shared_t    DESCRIPTION
 128 -----------             ------------    -----------
 129 sdbc_vol_name           sh_filename     - last 16 characters of the volume name
 130 sdbc_alloc              sh_alloc        - is this allocated?
 131 sdbc_failed             sh_failed       - Disk failure status (0=ok,1= /o 
 132                                           error ,2= open failed)
 133 sdbc_cd                 sh_cd           - the cache descriptor. (for stats)
 134 sdbc_cache_read         sh_cache_read   - Number of FBA's read from cache
 135 sdbc_cache_write        sh_cache_write  - Number of FBA's written  to cache
 136 sdbc_disk_read          sh_disk_read    - Number of FBA's read from disk 
 137 sdbc_disk_write         sh_disk_write   - Number of FBA's written to disk
 138 sdbc_filesize           sh_filesize     - Filesize (in FBA's)
 139 sdbc_numdirty           sh_numdirty     - Number of dirty blocks
 140 sdbc_numio              sh_numio        - Number of blocks on way to disk
 141 sdbc_numfail            sh_numfail      - Number of blocks failed
 142 sdbc_flushloop          sh_flushloop    - Loops delayed so far
 143 sdbc_flag               sh_flag         - Flags visible to user programs 
 144 sdbc_destaged           sh_destaged     - number of bytes destaged to disk
 145                                           (flushed from the cache to disk).
 146 sdbc_cdhints            ---             - cd hints (such as wrthru/nowrthru)
 147 
 148 All fields are read-only kstat_named_t kstats, with data type KSTAT_DATA_ULONG.
 149 The instance number of the kstat corresponds to the cache descriptor number.   
 150 Note that the "sdbc_wrcancelns" and "sdbc_destaged" are new, and have also 
 151 been added to the _sd_shared_t struct.
 152 
 153 2.2.2 Naming characteristics:
 154 -----------------------------
 155 module:         SDBC_KSTAT_MODULE       "sdbc"  
 156 class:          SDBC_KSTAT_CLASS        "storedge"
 157 name:           SDBC_KSTAT_CDSTATS      "cd%d"  (%d = < cd number >)
 158 instance #:     < cache descriptor number > 
 159 
 160 3.0 I/O KSTATS:
 161 ===============
 162 The sdbc module now contains kstats of type kstat_io_t.  These are used to
 163 track timing through the cache.  As with the "regular" kstats, sdbc tracks
 164 global statistics, as well as those per cache descriptor.  Since kstat_io_t
 165 is a built-in kstat type, all are defined the same way. 
 166 
 167 3.0.1 Time-Gathering:
 168 ---------------------
 169 These kstat_io_t types provide two built-in time-gathering mechanisms, which it 
 170 refers to as "waitq" and "runq," where "waitq" is intended to be interpreted 
 171 as the amount of time a request spends in its pre-service state, and "runq" the 
 172 amount of time a request spends in its service state.  Transitions to the
 173 runq and the waitq must be  made via built-in functions, such as
 174 kstat_runq_enter() and kstat_runq_exit().  The relevant fields in the 
 175 kstat_io_t structure should not be considered explicitly.  (See comment below).
 176 The iostat(1M) utility may be used to gather timing-related information
 177 collected through this mechanism.
 178 
 179 Please note that sdbc does not use waitq.
 180 sdbc uses runq as follows:
 181 
 182 An I/O request transitions to the runq (both global, and per-cd) upon entering 
 183 the cache through _sd_read(), _sd_write(), or _sd_alloc_buf().  It
 184 transitions off the runq after the request has been serviced, either by the 
 185 cache, or as the result of disk I/O.  Thus, this allows a user to track the
 186 total time spent in the cache, which includes disk I/O time.
 187 
 188  
 189 3.0.2 kstat_io_t Fields:
 190 ------------------------
 191 These I/O kstats include the following fields:
 192 
 193         u_longlong_t    nread;          /* number of bytes read */
 194         u_longlong_t    nwritten;       /* number of bytes written */
 195         uint_t          reads;          /* number of read operations */
 196         uint_t          writes;         /* number of write operations */
 197 
 198 # The following fields are automatically updated by the built-in
 199 # kstat_waitq_enter(), kstat_waitq_exit(), kstat_runq_enter() and
 200 # kstat_runq_exit() functions.
 201 
 202         hrtime_t wtime;         /* cumulative wait (pre-service) time */
 203         hrtime_t wlentime;      /* cumulative wait length*time product */
 204         hrtime_t wlastupdate;   /* last time wait queue changed */
 205         hrtime_t rtime;         /* cumulative run (service) time */
 206         hrtime_t rlentime;      /* cumulative run length*time product */
 207         hrtime_t rlastupdate;   /* last time run queue changed */
 208 
 209         uint_t  wcnt;           /* count of elements in wait state */
 210         uint_t  rcnt;           /* count of elements in run state */
 211 
 212 For more information, refer to [2] in the References section. 
 213 
 214 3.1 GLOBAL IO KSTATS:
 215 ~~~~~~~~~~~~~~~~~~~~~
 216 sdbc includes "global" I/O kstats which track the timings through the cache as 
 217 a whole, taking into account all cache descriptors.  The fields definitions
 218 are built-in, as explained above.
 219 
 220 3.1.1 Naming characteristics:
 221 -----------------------------
 222 module:         SDBC_KSTAT_MODULE       "sdbc"  
 223 class:                                  "disk"
 224 name:           SDBC_IOKSTAT_GSTATS     "gsdbc"
 225 instance #:     0
 226 
 227 3.2 IO KSTATS (PER CACHE DESCRIPTOR):
 228 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 229 These "cd" I/O kstats present statistics which reflect the state of a single 
 230 cache descriptor.  One of these I/O kstats exists for each open cache 
 231 descriptor. The fields definitions are built-in, as explained above.
 232 
 233 3.2.1 Naming characteristics:
 234 -----------------------------
 235 module:         SDBC_KSTAT_MODULE       "sdbc"  
 236 class:                                  "disk"
 237 name:           SDBC_IOKSTAT_STATS      "sdbc%d" (%d = < cd number >) 
 238 instance #:     < cache descriptor number > 
 239 
 240 4.0 DYNMEM KSTATS:
 241 ==================
 242 The sdbc module also a "regular" kstat to track dynamic memory
 243 allocation in the cache.  These are "global" statistics.
 244 
 245 Its fields can be divided logically between behavior variables, and statistical 
 246 variable
 247 
 248 4.1 Field Definitions: 
 249 ~~~~~~~~~~~~~~~~~~~~~~
 250 
 251 4.1.1 Behavior Variables:
 252 -------------------------
 253 sdbc_monitor_dynmem --- D0=monitor thread shutdown in the console window
 254                         D1=print deallocation thread stats to the console 
 255                         window
 256                         D2=print more deallocation thread stats to the console 
 257                         window
 258                         (usage: setting a value of 6 = 2+4 sets D1 and D2)
 259 sdbc_max_dyn_list ----- 1 to ?: sets the maximum host/parasite list length
 260                         (A length of 1 prevents any multipage allocations from 
 261                         occuring and effectively removes the concept of 
 262                         host/parasite.)
 263 sdbc_cache_aging_ct1 -- 1 to 255: fully aged count (everything but meta and 
 264                         holdover)
 265 sdbc_cache_aging_ct2 -- 1 to 255: fully aged count for meta-data entries
 266 sdbc_cache_aging_ct3 -- 1 to 255: fully aged count for holdovers
 267 sdbc_cache_aging_sec1 - 1 to 255: sleep level 1 for 100% to pcnt1 free cache 
 268                         entries
 269 sdbc_cache_aging_sec2 - 1 to 255: sleep level 2 for pcnt1 to pcnt2 free cache 
 270                         entries
 271 sdbc_cache_aging_sec3 - 1 to 255: sleep level 3 for pcnt2 to 0% free cache 
 272                         entries
 273 sdbc_cache_aging_pcnt1- 0 to 100: cache free percent for transition from 
 274                         sleep1 to sleep2
 275 sdbc_cache_aging_pcnt2- 0 to 100: cache free percent for transition from 
 276                         sleep2 to sleep3
 277 sdbc_max_holds_pcnt --- 0 to 100: max percent of cache entries to be maintained 
 278                         as holdovers
 279 
 280 4.1.2 Statistical Variables:
 281 ----------------------------
 282 Cache Stats (per wake cycle) (r/w):
 283 sdbc_alloc_ct --------- total allocations performed
 284 sdbc_dealloc_ct ------- total deallocations performed
 285 sdbc_history ---------- current hysterisis flag setting
 286 sdbc_nodatas ---------- cache entries w/o memory assigned
 287 sdbc_candidates ------- cache entries ready to be aged or released
 288 sdbc_deallocs --------- cache entries w/memory deallocated and requeued
 289 sdbc_hosts ------------ number of host cache entries
 290 sdbc_pests ------------ number of parasitic cache entries
 291 sdbc_metas ------------ number of meta-data cache entries
 292 sdbc_holds ------------ number of holdovers (fully aged w/memory and requeued)
 293 sdbc_others ----------- number of not [host, pests or metas]
 294 sdbc_notavail --------- number of cache entries to bypass (nodatas+'in use by 
 295                         other processes')
 296 sdbc_process_directive- D0=1 wake thread
 297                         D1=1 temporaily accelerate aging (set the hysterisis
 298                         flag)
 299 sdbc_simplect --------- simple count of the number of times the kstat update 
 300                         routine has been called (used for debugging)         
 301 
 302 The behavior fields (along with the "sdbc_process_directive" field) may be both 
 303 read and written.  The remaining statistical fields are read-only. 
 304 
 305 For more information, please refer to [3] in the References section.
 306 
 307 4.2 Naming characteristics:
 308 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 309 module:         SDBC_KSTAT_MODULE       "sdbc"  
 310 class:          SDBC_KSTAT_CLASS        "storedge"
 311 name:           SDBC_KSTAT_DYNMEM       "dynmem"
 312 instance #:     0       
 313 
 314 5.0 REFERENCES FOR FURTHER READING:
 315 ===================================
 316 1. generic kstat information: kstat(1M), <sys/include/kstat.h>
 317 2. kstat_io_t information: kstat_io(9S), kstat_queue(9F)
 318 3. sdbc dynamic memory implementation:
 319 <ds[3,4]>/src/uts/common/ns/sdbc/dynmem_readme.txt