1 '\" te 2 .\" Copyright 1999 Sun Microsystems, Inc. All Rights Reserved. 3 .\" The contents of this file are subject to the terms of the Common Development and Distribution License (the "License"). You may not use this file except in compliance with the License. 4 .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. See the License for the specific language governing permissions and limitations under the License. 5 .\" When distributing Covered Code, include this CDDL HEADER in each file and include the License file at usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your own identifying information: Portions Copyright [yyyy] [name of copyright owner] 6 .TH TNF_KERNEL_PROBES 4 "May 16, 2020" 7 .SH NAME 8 tnf_kernel_probes \- TNF kernel probes 9 .SH DESCRIPTION 10 The set of probes (trace instrumentation points) available in the standard 11 kernel. The probes log trace data to a kernel trace buffer in Trace Normal 12 Form (TNF). Kernel probes are controlled by \fBprex\fR(1). A snapshot of the 13 kernel trace buffer can be made using \fBtnfxtract\fR(1) and examined using 14 \fBtnfdump\fR(1). 15 .sp 16 .LP 17 Each probe has a \fIname\fR and is associated with a set of symbolic 18 \fIkeys\fR, or \fIcategories\fR. These are used to select and control probes 19 from \fBprex\fR(1). A probe that is enabled for tracing generates a \fBTNF\fR 20 record, called an \fIevent record\fR. An event record contains two common 21 members and may contain other probe-specific data members. 22 .SS "Common Members" 23 .in +2 24 .nf 25 \fBtnf_probe_event\fR \fItag\fR 26 \fBtnf_time_delta\fR \fItime_delta\fR 27 .fi 28 .in -2 29 30 .sp 31 .ne 2 32 .na 33 \fB\fItag\fR\fR 34 .ad 35 .RS 14n 36 Encodes \fBTNF\fR references to two other records: 37 .sp 38 .ne 2 39 .na 40 \fB\fItag\fR\fR 41 .ad 42 .RS 12n 43 Describes the layout of the event record. 44 .RE 45 46 .sp 47 .ne 2 48 .na 49 \fB\fIschedule\fR\fR 50 .ad 51 .RS 12n 52 Identifies the writing thread and also contains a 64-bit base time in 53 nanoseconds. 54 .RE 55 56 .RE 57 58 .sp 59 .ne 2 60 .na 61 \fB\fItime_delta\fR\fR 62 .ad 63 .RS 14n 64 A 32-bit time offset from the base time; the sum of the two times is the actual 65 time of the event. 66 .RE 67 68 .SS "Threads" 69 .SS "\fBthread_create\fR" 70 .in +2 71 .nf 72 \fBtnf_kthread_id\fR \fItid\fR 73 \fBtnf_pid\fR \fIpid\fR 74 \fBtnf_symbol\fR \fIstart_pc\fR 75 .fi 76 .in -2 77 78 .sp 79 .LP 80 Thread creation event. 81 .sp 82 .ne 2 83 .na 84 \fB\fItid\fR\fR 85 .ad 86 .RS 12n 87 The thread identifier for the new thread. 88 .RE 89 90 .sp 91 .ne 2 92 .na 93 \fB\fIpid\fR\fR 94 .ad 95 .RS 12n 96 The process identifier for the new thread. 97 .RE 98 99 .sp 100 .ne 2 101 .na 102 \fB\fIstart_pc\fR\fR 103 .ad 104 .RS 12n 105 The kernel address of its start routine. 106 .RE 107 108 .SS "\fBthread_state\fR" 109 .in +2 110 .nf 111 \fBtnf_kthread_id\fR \fItid\fR 112 \fBtnf_microstate\fR \fIstate\fR 113 .fi 114 .in -2 115 116 .sp 117 .LP 118 Thread microstate transition events. 119 .sp 120 .ne 2 121 .na 122 \fB\fItid\fR\fR 123 .ad 124 .RS 9n 125 Optional; if it is absent, the event is for the writing thread, otherwise the 126 event is for the specified thread. 127 .RE 128 129 .sp 130 .ne 2 131 .na 132 \fB\fIstate\fR\fR 133 .ad 134 .RS 9n 135 Indicates the thread state: 136 .RS +4 137 .TP 138 .ie t \(bu 139 .el o 140 Running in user mode. 141 .RE 142 .RS +4 143 .TP 144 .ie t \(bu 145 .el o 146 Running in system mode. 147 .RE 148 .RS +4 149 .TP 150 .ie t \(bu 151 .el o 152 Asleep waiting for a user-mode lock. 153 .RE 154 .RS +4 155 .TP 156 .ie t \(bu 157 .el o 158 Asleep on a kernel object. 159 .RE 160 .RS +4 161 .TP 162 .ie t \(bu 163 .el o 164 Runnable (waiting for a cpu). 165 .RE 166 .RS +4 167 .TP 168 .ie t \(bu 169 .el o 170 Stopped. 171 .RE 172 The values of this member are defined in <\fBsys/msacct.h\fR>. Note that to 173 reduce trace output, transitions between the \fIsystem\fR and \fIuser\fR 174 microstates that are induced by system calls are not traced. This information 175 is implicit in the system call entry and exit events. 176 .RE 177 178 .SS "thread_exit" 179 Thread termination event for writing thread. This probe has no data members 180 other than the common members. 181 .SS "Scheduling" 182 \fB\fR 183 .SS "thread_queue" 184 .in +2 185 .nf 186 \fBtnf_kthread_id\fR \fItid\fR 187 \fBtnf_cpuid\fR \fIcpuid\fR 188 \fBtnf_long\fR \fIpriority\fR 189 \fBtnf_ulong\fR \fIqueue_length\fR 190 .fi 191 .in -2 192 193 .sp 194 .LP 195 Thread scheduling events. These are triggered when a runnable thread is placed 196 on a dispatch queue. 197 .sp 198 .ne 2 199 .na 200 \fB\fIcpuid\fR\fR 201 .ad 202 .RS 16n 203 Specifies the cpu to which the queue is attached. 204 .RE 205 206 .sp 207 .ne 2 208 .na 209 \fB\fIpriority\fR\fR 210 .ad 211 .RS 16n 212 The (global) dispatch priority of the thread. 213 .RE 214 215 .sp 216 .ne 2 217 .na 218 \fB\fIqueue_length\fR\fR 219 .ad 220 .RS 16n 221 The current length of the cpu's dispatch queue. 222 .RE 223 224 .SS "Blocking" 225 .SS "\fBthread_block\fR" 226 .in +2 227 .nf 228 \fBtnf_opaque\fR \fIreason\fR 229 \fBtnf_symbols\fR \fIstack\fR 230 .fi 231 .in -2 232 233 .sp 234 .LP 235 Thread blockage event. This probe captures a partial stack backtrace when the 236 current thread blocks. 237 .sp 238 .ne 2 239 .na 240 \fB\fIreason\fR\fR 241 .ad 242 .RS 11n 243 The address of the object on which the thread is blocking. 244 .RE 245 246 .sp 247 .ne 2 248 .na 249 \fB\fIsymbols\fR\fR 250 .ad 251 .RS 11n 252 References a \fBTNF\fR array of kernel addresses representing the PCs on the 253 stack at the time the thread blocks. 254 .RE 255 256 .SS "System Calls" 257 .SS "\fBsyscall_start\fR" 258 .in +2 259 .nf 260 \fBtnf_sysnum\fR \fIsysnum\fR 261 .fi 262 .in -2 263 264 .sp 265 .LP 266 System call entry event. 267 .sp 268 .ne 2 269 .na 270 \fB\fIsysnum\fR\fR 271 .ad 272 .RS 10n 273 The system call number. The writing thread implicitly enters the \fIsystem\fR 274 microstate with this event. 275 .RE 276 277 .SS "\fBsyscall_end\fR" 278 .in +2 279 .nf 280 \fBtnf_long\fR \fIrval1\fR 281 \fBtnf_long\fR \fIrval2\fR 282 \fBtnf_long\fR \fIerrno\fR 283 .fi 284 .in -2 285 286 .sp 287 .LP 288 System call exit event. 289 .sp 290 .ne 2 291 .na 292 \fB\fIrval1\fR and \fIrval2\fR\fR 293 .ad 294 .RS 19n 295 The two return values of the system call 296 .RE 297 298 .sp 299 .ne 2 300 .na 301 \fB\fIerrno\fR\fR 302 .ad 303 .RS 19n 304 The error return. 305 .RE 306 307 .sp 308 .LP 309 The writing thread implicitly enters the \fIuser\fR microstate with this event. 310 .SS "Page Faults" 311 .SS "\fBaddress_fault\fR" 312 .in +2 313 .nf 314 \fBtnf_opaque\fR \fIaddress\fR 315 \fBtnf_fault_type\fR \fIfault_type\fR 316 \fBtnf_seg_access\fR \fIaccess\fR 317 .fi 318 .in -2 319 320 .sp 321 .LP 322 Address-space fault event. 323 .sp 324 .ne 2 325 .na 326 \fB\fIaddress\fR\fR 327 .ad 328 .RS 14n 329 Gives the faulting virtual address. 330 .RE 331 332 .sp 333 .ne 2 334 .na 335 \fB\fIfault_type\fR\fR 336 .ad 337 .RS 14n 338 Gives the fault type: invalid page, protection fault, software requested 339 locking or unlocking. 340 .RE 341 342 .sp 343 .ne 2 344 .na 345 \fB\fIaccess\fR\fR 346 .ad 347 .RS 14n 348 Gives the desired access protection: read, write, execute or create. The values 349 for these two members are defined in <\fBvm/seg_enum.h\fR>. 350 .RE 351 352 .SS "\fBmajor_fault\fR" 353 .in +2 354 .nf 355 \fBtnf_opaque\fR \fIvnode\fR 356 \fBtnf_offset\fR \fIoffset\fR 357 .fi 358 .in -2 359 360 .sp 361 .LP 362 Major page fault event. The faulting page is mapped to the file given by the 363 \fIvnode\fR member, at the given \fIoffset\fR into the file. (The faulting 364 virtual address is in the most recent \fBaddress_fault\fR event for the writing 365 thread.) 366 .SS "\fBanon_private\fR" 367 .in +2 368 .nf 369 \fBtnf_opaque\fR \fIaddress\fR 370 .fi 371 .in -2 372 373 .sp 374 .LP 375 Copy-on-write page fault event. 376 .sp 377 .ne 2 378 .na 379 \fB\fIaddress\fR\fR 380 .ad 381 .RS 11n 382 The virtual address at which the new page is mapped. 383 .RE 384 385 .SS "\fBanon_zero\fR" 386 .in +2 387 .nf 388 \fBtnf_opaque\fR \fIaddress\fR 389 .fi 390 .in -2 391 392 .sp 393 .LP 394 Zero-fill page fault event. 395 .sp 396 .ne 2 397 .na 398 \fB\fIaddress\fR\fR 399 .ad 400 .RS 11n 401 The virtual address at which the new page is mapped. 402 .RE 403 404 .SS "\fBpage_unmap\fR" 405 .in +2 406 .nf 407 \fBtnf_opaque\fR \fIvnode\fR 408 \fBtnf_offset\fR \fIoffset\fR 409 .fi 410 .in -2 411 412 .sp 413 .LP 414 Page unmapping event. This probe marks the unmapping of a file system page 415 from the system. 416 .sp 417 .ne 2 418 .na 419 \fB\fIvnode\fR and \fIoffset\fR\fR 420 .ad 421 .RS 20n 422 Identifies the file and offset of the page being unmapped. 423 .RE 424 425 .SS "Pageins and Pageouts" 426 .SS "\fBpagein\fR" 427 .in +2 428 .nf 429 \fBtnf_opaque\fR \fIvnode\fR 430 \fBtnf_offset\fR \fIoffset\fR 431 \fBtnf_size\fR \fIsize\fR 432 .fi 433 .in -2 434 435 .sp 436 .LP 437 Pagein start event. This event signals the initiation of pagein I/O. 438 .sp 439 .ne 2 440 .na 441 \fB\fIvnode\fRand\fIoffset\fR\fR 442 .ad 443 .RS 18n 444 Identifies the file and offset to be paged in. 445 .RE 446 447 .sp 448 .ne 2 449 .na 450 \fB\fIsize\fR\fR 451 .ad 452 .RS 18n 453 Specifies the number of bytes to be paged in. 454 .RE 455 456 .SS "\fBpageout\fR" 457 .in +2 458 .nf 459 \fBtnf_opaque\fR \fIvnode\fR 460 \fBtnf_ulong\fR \fIpages_pageout\fR 461 \fBtnf_ulong\fR \fIpages_freed\fR 462 \fBtnf_ulong\fR \fIpages_reclaimed\fR 463 .fi 464 .in -2 465 466 .sp 467 .LP 468 Pageout completion event. This event signals the completion of pageout I/O. 469 .sp 470 .ne 2 471 .na 472 \fB\fIvnode\fR\fR 473 .ad 474 .RS 19n 475 Identifies the file of the pageout request. 476 .RE 477 478 .sp 479 .ne 2 480 .na 481 \fB\fIpages_pageout\fR\fR 482 .ad 483 .RS 19n 484 The number of pages written out. 485 .RE 486 487 .sp 488 .ne 2 489 .na 490 \fB\fIpages_freed\fR\fR 491 .ad 492 .RS 19n 493 The number of pages freed after being written out. 494 .RE 495 496 .sp 497 .ne 2 498 .na 499 \fB\fIpages_reclaimed\fR\fR 500 .ad 501 .RS 19n 502 The number of pages reclaimed after being written out. 503 .RE 504 505 .SS "Page Daemon (Page Stealer)" 506 .SS "\fBpageout_scan_start\fR" 507 .in +2 508 .nf 509 \fBtnf_ulong\fR \fIpages_free\fR 510 \fBtnf_ulong\fR \fIpages_needed\fR 511 .fi 512 .in -2 513 514 .sp 515 .LP 516 Page daemon scan start event. This event signals the beginning of one 517 iteration of the page daemon. 518 .sp 519 .ne 2 520 .na 521 \fB\fIpages_free\fR\fR 522 .ad 523 .RS 16n 524 The number of free pages in the system. 525 .RE 526 527 .sp 528 .ne 2 529 .na 530 \fB\fIpages_needed\fR\fR 531 .ad 532 .RS 16n 533 The number of pages desired free. 534 .RE 535 536 .SS "\fBpageout_scan_end\fR" 537 .in +2 538 .nf 539 \fBtnf_ulong\fR \fIpages_free\fR 540 \fBtnf_ulong\fR \fIpages_scanned\fR 541 .fi 542 .in -2 543 544 .sp 545 .LP 546 Page daemon scan end event. This event signals the end of one iteration of the 547 page daemon. 548 .sp 549 .ne 2 550 .na 551 \fB\fIpages_free\fR\fR 552 .ad 553 .RS 17n 554 The number of free pages in the system. 555 .RE 556 557 .sp 558 .ne 2 559 .na 560 \fB\fIpages_scanned\fR\fR 561 .ad 562 .RS 17n 563 The number of pages examined by the page daemon. (Potentially more pages will 564 be freed when any queued pageout requests complete.) 565 .RE 566 567 .SS "Swapper" 568 .SS "\fBswapout_process\fR" 569 .in +2 570 .nf 571 \fBtnf_pid\fR \fIpid\fR 572 \fBtnf_ulong\fR \fIpage_count\fR 573 .fi 574 .in -2 575 576 .sp 577 .LP 578 Address space swapout event. This event marks the swapping out of a process 579 address space. 580 .sp 581 .ne 2 582 .na 583 \fB\fIpid\fR\fR 584 .ad 585 .RS 14n 586 Identifies the process. 587 .RE 588 589 .sp 590 .ne 2 591 .na 592 \fB\fIpage_count\fR\fR 593 .ad 594 .RS 14n 595 Reports the number of pages either freed or queued for pageout. 596 .RE 597 598 .SS "\fBswapout_lwp\fR" 599 .in +2 600 .nf 601 \fBtnf_pid\fR \fIpid\fR 602 \fBtnf_lwpid\fR \fIlwpid\fR 603 \fBtnf_kthread_id\fR \fItid\fR 604 \fBtnf_ulong\fR \fIpage_count\fR 605 .fi 606 .in -2 607 608 .sp 609 .LP 610 Light-weight process swapout event. This event marks the swapping out of an 611 \fBLWP\fR and its stack. 612 .sp 613 .ne 2 614 .na 615 \fB\fIpid\fR\fR 616 .ad 617 .RS 14n 618 The \fBLWP's\fR process identifier 619 .RE 620 621 .sp 622 .ne 2 623 .na 624 \fB\fIlwpid\fR\fR 625 .ad 626 .RS 14n 627 The \fBLWP\fR identifier 628 .RE 629 630 .sp 631 .ne 2 632 .na 633 \fB\fItid\fR \fImember\fR\fR 634 .ad 635 .RS 14n 636 The \fBLWP's\fR kernel thread identifier. 637 .RE 638 639 .sp 640 .ne 2 641 .na 642 \fB\fIpage_count\fR\fR 643 .ad 644 .RS 14n 645 The number of pages swapped out. 646 .RE 647 648 .SS "\fBswapin_lwp\fR" 649 .in +2 650 .nf 651 \fBtnf_pid\fR \fIpid\fR 652 \fBtnf_lwpid\fR \fIlwpid\fR 653 \fBtnf_kthread_id\fR \fItid\fR 654 \fBtnf_ulong\fR \fIpage_count\fR 655 .fi 656 .in -2 657 658 .sp 659 .LP 660 Light-weight process swapin event. This event marks the swapping in of an 661 \fBLWP\fR and its stack. 662 .sp 663 .ne 2 664 .na 665 \fB\fIpid\fR\fR 666 .ad 667 .RS 14n 668 The \fBLWP's\fR process identifier. 669 .RE 670 671 .sp 672 .ne 2 673 .na 674 \fB\fIlwpid\fR\fR 675 .ad 676 .RS 14n 677 The \fBLWP\fR identifier. 678 .RE 679 680 .sp 681 .ne 2 682 .na 683 \fB\fItid\fR\fR 684 .ad 685 .RS 14n 686 The \fBLWP's\fR kernel thread identifier. 687 .RE 688 689 .sp 690 .ne 2 691 .na 692 \fB\fIpage_count\fR\fR 693 .ad 694 .RS 14n 695 The number of pages swapped in. 696 .RE 697 698 .SS "Local I/O" 699 .SS "\fBstrategy\fR" 700 .in +2 701 .nf 702 \fBtnf_device\fR \fIdevice\fR 703 \fBtnf_diskaddr\fR \fIblock\fR 704 \fBtnf_size\fR \fIsize\fR 705 \fBtnf_opaque\fR \fIbuf\fR 706 \fBtnf_bioflags\fR \fI flags\fR 707 .fi 708 .in -2 709 710 .sp 711 .LP 712 Block I/O strategy event. This event marks a call to the \fBstrategy\fR(9E) 713 function of a block device driver. 714 .sp 715 .ne 2 716 .na 717 \fB\fIdevice\fR\fR 718 .ad 719 .RS 10n 720 Contains the major and minor numbers of the device. 721 .RE 722 723 .sp 724 .ne 2 725 .na 726 \fB\fIblock\fR\fR 727 .ad 728 .RS 10n 729 The logical block number to be accessed on the device. 730 .RE 731 732 .sp 733 .ne 2 734 .na 735 \fB\fIsize\fR\fR 736 .ad 737 .RS 10n 738 The size of the I/O request. 739 .RE 740 741 .sp 742 .ne 2 743 .na 744 \fB\fIbuf\fR\fR 745 .ad 746 .RS 10n 747 The kernel address of the \fBbuf\fR(9S) structure associated with the transfer. 748 .RE 749 750 .sp 751 .ne 2 752 .na 753 \fB\fIflags\fR\fR 754 .ad 755 .RS 10n 756 The \fBbuf\fR(9S) flags associated with the transfer. 757 .RE 758 759 .SS "\fBbiodone\fR" 760 .in +2 761 .nf 762 \fBtnf_device\fR \fIdevice\fR 763 \fBtnf_diskaddr\fR \fIblock\fR 764 \fBtnf_opaque\fR \fIbuf\fR 765 .fi 766 .in -2 767 768 .sp 769 .LP 770 Buffered I/O completion event. This event marks calls to the \fBbiodone\fR(9F) 771 function. 772 .sp 773 .ne 2 774 .na 775 \fB\fIdevice\fR\fR 776 .ad 777 .RS 10n 778 Contains the major and minor numbers of the device. 779 .RE 780 781 .sp 782 .ne 2 783 .na 784 \fB\fIblock\fR\fR 785 .ad 786 .RS 10n 787 The logical block number accessed on the device. 788 .RE 789 790 .sp 791 .ne 2 792 .na 793 \fB\fIbuf\fR\fR 794 .ad 795 .RS 10n 796 The kernel address of the \fBbuf\fR(9S) structure associated with the transfer. 797 .RE 798 799 .SS "\fBphysio_start\fR" 800 .in +2 801 .nf 802 \fBtnf_device\fR \fIdevice\fR 803 \fBtnf_offset\fR \fIoffset\fR 804 \fBtnf_size\fR \fIsize\fR 805 \fBtnf_bioflags\fR \fIrw\fR 806 .fi 807 .in -2 808 809 .sp 810 .LP 811 Raw I/O start event. This event marks entry into the \fBphysio\fR(9F) 812 function which performs unbuffered I/O. 813 .sp 814 .ne 2 815 .na 816 \fB\fIdevice\fR\fR 817 .ad 818 .RS 10n 819 Contains the major and minor numbers of the device of the transfer. 820 .RE 821 822 .sp 823 .ne 2 824 .na 825 \fB\fIoffset\fR\fR 826 .ad 827 .RS 10n 828 The logical offset on the device for the transfer. 829 .RE 830 831 .sp 832 .ne 2 833 .na 834 \fB\fIsize\fR\fR 835 .ad 836 .RS 10n 837 The number of bytes to be transferred. 838 .RE 839 840 .sp 841 .ne 2 842 .na 843 \fB\fIrw\fR\fR 844 .ad 845 .RS 10n 846 The direction of the transfer: read or write (see \fBbuf\fR(9S)). 847 .RE 848 849 .SS "\fBphysio_end\fR" 850 .in +2 851 .nf 852 \fBtnf_device\fR \fIdevice\fR 853 .fi 854 .in -2 855 856 .sp 857 .LP 858 Raw I/O end event. This event marks exit from the \fBphysio\fR(9F) function. 859 .sp 860 .ne 2 861 .na 862 \fB\fIdevice\fR\fR 863 .ad 864 .RS 10n 865 The major and minor numbers of the device of the transfer. 866 .RE 867 868 .SH USAGE 869 Use the \fBprex\fR utility to control kernel probes. The standard \fBprex\fR 870 commands to list and manipulate probes are available to you, along with 871 commands to set up and manage kernel tracing. 872 .sp 873 .LP 874 Kernel probes write trace records into a kernel trace buffer. You must copy the 875 buffer into a TNF file for post-processing; use the \fBtnfxtract\fR utility for 876 this. 877 .sp 878 .LP 879 You use the \fBtnfdump\fR utility to examine a kernel trace file. This is 880 exactly the same as examining a user-level trace file. 881 .sp 882 .LP 883 The steps you typically follow to take a kernel trace are: 884 .RS +4 885 .TP 886 1. 887 Become superuser (\fBsu\fR). 888 .RE 889 .RS +4 890 .TP 891 2. 892 Allocate a kernel trace buffer of the desired size (\fBprex\fR). 893 .RE 894 .RS +4 895 .TP 896 3. 897 Select the probes you want to trace and enable (\fBprex\fR). 898 .RE 899 .RS +4 900 .TP 901 4. 902 Turn kernel tracing on (\fBprex\fR). 903 .RE 904 .RS +4 905 .TP 906 5. 907 Run your application. 908 .RE 909 .RS +4 910 .TP 911 6. 912 Turn kernel tracing off (\fBprex\fR). 913 .RE 914 .RS +4 915 .TP 916 7. 917 Extract the kernel trace buffer (\fBtnfxtract\fR). 918 .RE 919 .RS +4 920 .TP 921 8. 922 Disable all probes (\fBprex\fR). 923 .RE 924 .RS +4 925 .TP 926 9. 927 Deallocate the kernel trace buffer (\fBprex\fR). 928 .RE 929 .RS +4 930 .TP 931 10. 932 Examine the trace file (\fBtnfdump\fR). 933 .RE 934 .sp 935 .LP 936 A convenient way to follow these steps is to use two shell windows; run an 937 interactive \fBprex\fR session in one, and run your application and 938 \fBtnfxtract\fR in the other. 939 .SH SEE ALSO 940 \fBprex\fR(1), \fBtnfdump\fR(1), \fBtnfxtract\fR(1), \fBlibtnfctl\fR(3TNF), 941 \fBTNF_PROBE\fR(3TNF), \fBtracing\fR(3TNF), \fBstrategy\fR(9E), 942 \fBbiodone\fR(9F), \fBphysio\fR(9F), \fBbuf\fR(9S)