1 .\" "
   2 .\" " The contents of this file are subject to the terms of the
   3 .\" " Common Development and Distribution License (the "License").
   4 .\" " You may not use this file except in compliance with the License.
   5 .\" "
   6 .\" " You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
   7 .\" " or http://www.opensolaris.org/os/licensing.
   8 .\" " See the License for the specific language governing permissions
   9 .\" " and limitations under the License.
  10 .\" "
  11 .\" " When distributing Covered Code, include this CDDL HEADER in each
  12 .\" " file and include the License file at usr/src/OPENSOLARIS.LICENSE.
  13 .\" " If applicable, add the following below this CDDL HEADER, with the
  14 .\" " fields enclosed by brackets "[]" replaced with your own identifying
  15 .\" " information: Portions Copyright [yyyy] [name of copyright owner]
  16 .\" "
  17 .\" " CDDL HEADER END
  18 .\" "
  19 .\" "Copyright (c) 1999, 2010, Oracle and/or its affiliates. All rights reserved.
  20 .\" "Copyright 2012 Joshua M. Clulow <josh@sysmgr.org>
  21 .\" "
  22 .TH nightly 1 "6 July 2010"
  23 .SH NAME
  24 .I nightly
  25 \- build an OS-Net consolidation overnight
  26 .SH SYNOPSIS
  27 \fBnightly [-in] [-V VERS] <env_file>\fP
  28 .LP
  29 .SH DESCRIPTION
  30 .IX "OS-Net build tools" "nightly" "" "\fBnightly\fP"
  31 .LP
  32 .I nightly,
  33 the mother of all build scripts,
  34 can bringover, build, archive, package, error check, and
  35 generally do everything it takes to
  36 turn OS/Net consolidation source code into useful stuff.
  37 It is customizable to permit you to run anything from a
  38 simple build to all of the cross-checking a gatekeeper
  39 needs.  The advantage to using
  40 .I nightly
  41 is that you build things correctly, consistently and
  42 automatically, with the best practices; building with
  43 .I nightly
  44 can mean never having to say you're sorry to your
  45 gatekeeper.
  46 .LP
  47 More
  48 specifically,
  49 .I nightly
  50 performs the following tasks, in order, if
  51 all these things are desired:
  52 .LP
  53 .RS
  54 .TP
  55 \(bu
  56 perform a "make clobber" to clean up old binaries
  57 .TP
  58 \(bu
  59 bringover from the identified parent gate/clone
  60 .TP
  61 \(bu
  62 perform non-DEBUG and DEBUG builds
  63 .TP
  64 \(bu
  65 list proto area files and compare with previous list
  66 .TP
  67 \(bu
  68 copy updated proto area to parent
  69 .TP
  70 \(bu
  71 list shared lib interface and compare with previous list
  72 .TP
  73 \(bu
  74 perform a "make lint" of the kernel and report errors
  75 .TP
  76 \(bu
  77 perform a "make check" to report hdrchk/cstyle errors
  78 .TP
  79 \(bu
  80 report the presence of any core files
  81 .TP
  82 \(bu
  83 check the ELF runtime attributes of all dynamic objects
  84 .TP
  85 \(bu
  86 check for unreferenced files
  87 .TP
  88 \(bu
  89 report on which proto area objects have changed (since the last build)
  90 .TP
  91 \(bu
  92 report the total build time
  93 .TP
  94 \(bu
  95 save a detailed log file for reference
  96 .TP
  97 \(bu
  98 mail the user a summary of the completed build
  99 .RE
 100 .LP
 101 The actions of the script are almost completely determined by
 102 the environment variables in the
 103 .I env
 104 file, the only necessary argument.  Ths only thing you really
 105 need to use 
 106 .I nightly
 107 is an
 108 .I env
 109 file that does what you want.
 110 .LP
 111 Like most of the other build tools in usr/src/tools, this script tends
 112 to change on a fairly regular basis; do not expect to be able to build
 113 OS/Net with a version of nightly significantly older than your source
 114 tree.  It has what is effectively a Consolidation Private relationship
 115 to other build tools and with many parts of the OS/Net makefiles,
 116 although it may also be used to build other consolidations.
 117 .LP
 118 .SH NIGHTLY_OPTIONS
 119 The environment variable NIGHTLY_OPTIONS controls the actions
 120 .I nightly
 121 will take as it proceeds.
 122 The -i, -n, +t and -V options may also be used from the command
 123 line to control the actions without editing your environment file.
 124 The -i and -n options complete the build more quickly by bypassing
 125 some actions. If NIGHTLY_OPTIONS is not set, then "-Bmt" build
 126 options will be used.
 127 
 128 .B Basic action options
 129 .TP 10
 130 .B \-D
 131 Do a build with DEBUG on (non-DEBUG is built by default)
 132 .TP
 133 .B \-F
 134 Do _not_ do a non-DEBUG build (use with -D to get just a DEBUG build)
 135 .TP
 136 .B \-M
 137 Do not run pmodes (safe file permission checker)
 138 .TP
 139 .B \-i
 140 Do an incremental build, suppressing the "make clobber" that by
 141 default removes all existing binaries and derived files.  From the
 142 command line, -i also suppresses the lint pass and the cstyle/hdrchk
 143 pass
 144 .TP
 145 .B \-n
 146 Suppress the bringover so that the build will start immediately with
 147 current source code
 148 .TP
 149 .B \-o
 150 Do an "old style" (pre-S10) build using root privileges to set OWNER
 151 and GROUP from the Makefiles.
 152 .TP
 153 .B \-p
 154 Create packages for regular install
 155 .TP
 156 .B \-U
 157 Update proto area in the parent workspace
 158 .TP
 159 .B \-u
 160 Update the parent workspace with files generated by the build, as follows.
 161 .RS
 162 .TP
 163 \(bu
 164 Copy proto_list_${MACH} and friends to usr/src in the parent.
 165 .TP
 166 \(bu
 167 When used with -f, build a usr/src/unrefmaster.out in
 168 the parent by merging all the usr/src/unref-${MACH}.out files in the
 169 parent.
 170 .TP
 171 \(bu
 172 When used with -A or -r, copy the contents of the resulting
 173 ELF-data.${MACH} directory to usr/src/ELF-data.${MACH} in the parent
 174 workspace.
 175 .RE
 176 .TP
 177 .B \-m
 178 Send mail to $MAILTO at end of build
 179 .TP
 180 .B \-t
 181 Build and use the tools in $SRC/tools (default setting).
 182 .TP
 183 .B \+t
 184 Use the build tools in "$ONBLD_TOOLS/bin".
 185 
 186 .LP
 187 .B Code checking options
 188 .TP 10
 189 .B \-A
 190 Check for ABI discrepancies in .so files.
 191 It is only required for shared object developers when there is an
 192 addition, deletion or change of interface in the .so files.
 193 .TP
 194 .B \-C
 195 Check for cstyle/hdrchk errors
 196 .TP
 197 .B \-f
 198 Check for unreferenced files.  Since the full workspace must be built
 199 in order to accurately identify unreferenced files, -f is ignored for
 200 incremental (-i) builds, or builds that do not include -l, and -p.
 201 .TP
 202 .B \-r
 203 Check the ELF runtime attributes of all dynamic objects
 204 .TP
 205 .B \-l
 206 Do "make lint" in $LINTDIRS (default: $SRC n)
 207 .TP
 208 .B \-N
 209 Do not run protocmp or checkpaths (note: this option is not
 210 recommended, especially in conjunction with the \-p option)
 211 .TP
 212 .B \-W
 213 Do not report warnings (for freeware gate ONLY)
 214 .TP
 215 .B \-w
 216 Report which proto area objects differ between this and the last build.
 217 See wsdiff(1) for details. Note that the proto areas used for comparison
 218 are the last ones constructed as part of the build. As an example, if both
 219 a non-debug and debug build are performed (in that order), then the debug
 220 proto area will be used for comparison (which might not be what you want).
 221 .LP
 222 .B Groups of options
 223 .TP 10
 224 .B \-G
 225 Gate keeper default group of options (-u)
 226 .TP
 227 .B \-I
 228 Integration engineer default group of options (-mpu)
 229 .TP
 230 .B \-R
 231 Default group of options for building a release (-mp)
 232 
 233 .LP
 234 .B Source Build options
 235 .TP 10
 236 .B \-S E | D | H
 237 Build the Export, Domestic, or Hybrid source product. Only Export and
 238 Domestic are truly buildable at this time.
 239 .TP 10
 240 .B \-S O
 241 Simulate an OpenSolaris build on a full tree.  This can be used by
 242 internal developers to ensure that they haven't broken the build for
 243 external developers.
 244 .LP
 245 Source build options only make sense for a full internal tree (open
 246 and closed source).  Only one source build option can be specified at
 247 a time.
 248 
 249 .LP
 250 .B Miscellaneous options
 251 .TP 10
 252 .B \-O
 253 generate deliverables for OpenSolaris.  Tarballs containing binaries
 254 of closed-source components are put in $CODEMGR_WS.
 255 .TP 10
 256 .B \-V VERS
 257 set the build version string to VERS, overriding VERSION
 258 .TP
 259 .B \-X
 260 Copies the proto area and packages from the IHV and IHV-bin gates into the
 261 nightly proto and package areas.  This is only available on i386.  See
 262 .B REALMODE ENVIRONMENT VARIABLES
 263 and
 264 .B BUILDING THE IHV WORKSPACE
 265 below.
 266 
 267 .LP
 268 .SH ENVIRONMENT VARIABLES
 269 .LP
 270 Here is a list of prominent environment variables that 
 271 .I nightly
 272 references and the meaning of each variable.
 273 .LP
 274 .RE
 275 .B CODEMGR_WS
 276 .RS 5
 277 The root of your workspace, including whatever metadata is kept by
 278 the source code management system.  This is the workspace in which the
 279 build will be done. 
 280 .RE
 281 .LP
 282 .B PARENT_WS
 283 .RS 5
 284 The root of the workspace that is the parent of the
 285 one being built.  This is particularly relevant for configurations
 286 with a main
 287 workspace and build workspaces underneath it; see the 
 288 \-u and \-U
 289 options as well as the PKGARCHIVE environment variable, for more
 290 information.
 291 .RE
 292 .LP
 293 .B BRINGOVER_WS
 294 .RS 5
 295 This is the workspace from which
 296 .I nightly
 297 will fetch sources to either populate or update your workspace;
 298 it defaults to $CLONE_WS.
 299 .RE
 300 .LP
 301 .B CLOSED_BRINGOVER_WS
 302 .RS 5
 303 A full Mercurial workspace has two repositories: one for open source
 304 and one for closed source.  If this variable is non-null, 
 305 .I nightly
 306 will pull from the repository that it names to get the closed source.
 307 It defaults to $CLOSED_CLONE_WS.
 308 .LP
 309 If $CODEMGR_WS already exists and contains only the open repository,
 310 .I nightly
 311 will ignore this variable; you'll need to pull the closed repository
 312 by hand if you want it.
 313 .RE
 314 .LP
 315 .B CLONE_WS
 316 .RS 5
 317 This is the workspace from which
 318 .I nightly
 319 will fetch sources by default.  This is
 320 often distinct from the parent, particularly if the parent is a gate.
 321 .RE
 322 .LP
 323 .B CLOSED_CLONE_WS
 324 .RS 5
 325 This is the default closed-source Mercurial repository that 
 326 .I nightly
 327 might pull from (see
 328 .B CLOSED_BRINGOVER_WS
 329 for details).
 330 .RE
 331 .LP
 332 .B SRC
 333 .RS 5
 334 Root of OS-Net source code, referenced by the Makefiles.  It is
 335 the starting point of build activity.  It should be expressed
 336 in terms of $CODEMGR_WS.
 337 .RE
 338 .LP
 339 .B ROOT
 340 .RS 5
 341 Root of the proto area for the build.  The makefiles direct
 342 installation of build products to this area and
 343 direct references to these files by builds of commands and other
 344 targets.  It should be expressed in terms of $CODEMGR_WS.
 345 .LP
 346 If $MULTI_PROTO is "no", $ROOT may contain a DEBUG or non-DEBUG
 347 build.  If $MULTI_PROTO is "yes", $ROOT contains the DEBUG build and
 348 $ROOT-nd contains the non-DEBUG build.
 349 .LP
 350 For OpenSolaris deliveries (\fB\-O\fR), $ROOT-closed contains a parallel
 351 proto area containing the DEBUG build of just usr/closed components, and
 352 $ROOT-nd-closed contains the non-DEBUG equivalent.
 353 .RE
 354 .LP
 355 .B TOOLS_ROOT
 356 .RS 5
 357 Root of the tools proto area for the build.  The makefiles direct
 358 installation of tools build products to this area.  Unless \fB+t\fR
 359 is part of $NIGHTLY_OPTIONS, these tools will be used during the
 360 build.
 361 .LP
 362 As built by nightly, this will always contain non-DEBUG objects.
 363 Therefore, this will always have a -nd suffix, regardless of
 364 $MULTI_PROTO.
 365 .RE
 366 .LP
 367 .B MACH
 368 .RS 5
 369 The instruction set architecture of the build machine as given
 370 by \fIuname -p\fP, e.g. sparc, i386.
 371 .RE
 372 .LP
 373 .B LOCKNAME
 374 .RS 5
 375 The name of the file used to lock out multiple runs of
 376 .IR nightly .
 377 This should generally be left to the default setting.
 378 .RE
 379 .LP
 380 .B ATLOG
 381 .RS 5
 382 The location of the log directory maintained by
 383 .IR nightly .
 384 This should generally be left to the default setting.
 385 .RE
 386 .LP
 387 .B LOGFILE
 388 .RS 5
 389 The name of the log file in the $ATLOG directory maintained by
 390 .IR nightly .
 391 This should generally be left to the default setting.
 392 .RE
 393 .LP
 394 .B STAFFER
 395 .RS 5
 396 The non-root account to use on the build machine for the
 397 bringover from the clone or parent workspace.
 398 This may not be the same identify used by the SCM.
 399 .RE
 400 .LP
 401 .B MAILTO
 402 .RS 5
 403 The address to be used to send completion e-mail at the end of
 404 the build (for the \-m option).
 405 .RE
 406 .LP
 407 .B MAILFROM
 408 .RS 5
 409 The address to be used for From: in the completion e-mail at the
 410 end of the build (for the \-m option).
 411 .RE
 412 .LP
 413 .B REF_PROTO_LIST
 414 .RS 5
 415 Name of file used with protocmp to compare proto area contents.
 416 .RE
 417 .LP
 418 .B PARENT_ROOT
 419 .RS 5
 420 The parent root, which is the destination for copying the proto
 421 area(s) when using the \-U option.
 422 .RE
 423 .LP
 424 .B PARENT_TOOLS_ROOT
 425 .RS 5
 426 The parent tools root, which is the destination for copying the tools
 427 proto area when using the \-U option.
 428 .RE
 429 .LP
 430 .B RELEASE
 431 .RS 5
 432 The release version number to be used; e.g., 5.10.1 (Note: this is set
 433 in Makefile.master and should not normally be overridden).
 434 .RE
 435 .LP
 436 .B VERSION
 437 .RS 5
 438 The version text string to be used; e.g., "onnv:`date '+%Y-%m-%d'`".
 439 .RE
 440 .LP
 441 .B RELEASE_DATE
 442 .RS 5
 443 The release date text to be used; e.g., October 2009. If not set in
 444 your environment file, then this text defaults to the output from
 445 $(LC_ALL=C date +"%B %Y"); e.g., "October 2009".
 446 .RE
 447 .LP
 448 .B INTERNAL_RELEASE_BUILD
 449 .RS 5
 450 See Makefile.master - but it mostly controls id strings. Generally,
 451 let
 452 .I nightly
 453 set this for you.
 454 .RE
 455 .LP
 456 .B RELEASE_BUILD
 457 .RS 5
 458 Define this to build a release with a non-DEBUG kernel. 
 459 Generally, let
 460 .I nightly
 461 set this for you based on its options.
 462 .RE
 463 .LP
 464 .B PKGARCHIVE
 465 .RS 5
 466 The destination for packages.  This may be relative to
 467 $CODEMGR_WS for private packages or relative to $PARENT_WS
 468 if you have different workspaces for different architectures
 469 but want one hierarchy of packages.
 470 .RE
 471 .LP
 472 .B MAKEFLAGS
 473 .RS 5
 474 Set default flags to make; e.g., -k to build all targets regardless of errors.
 475 .RE
 476 .LP
 477 .B UT_NO_USAGE_TRACKING
 478 .RS 5
 479 Disables usage reporting by listed Devpro tools. Otherwise it sends mail
 480 to some Devpro machine every time the tools are used.
 481 .RE
 482 .LP
 483 .B LINTDIRS
 484 .RS 5
 485 Directories to lint with the \-l option.
 486 .RE
 487 .LP
 488 .B BUILD_TOOLS
 489 .RS 5
 490 BUILD_TOOLS is the root of all tools including the compilers; e.g.,
 491 /ws/onnv-tools.  It is used by the makefile system, but not nightly.
 492 .RE
 493 .LP
 494 .B ONBLD_TOOLS
 495 .RS 5
 496 ONBLD_TOOLS is the root of all the tools that are part of SUNWonbld; e.g.,
 497 /ws/onnv-tools/onbld.  By default, it is derived from
 498 .BR BUILD_TOOLS .
 499 It is used by the makefile system, but not nightly.
 500 .RE
 501 .LP
 502 .B SPRO_ROOT
 503 .RS 5
 504 The gate-defined default location for the Sun compilers, e.g.
 505 /ws/onnv-tools/SUNWspro.  By default, it is derived from
 506 .BR BUILD_TOOLS .
 507 It is used by the makefile system, but not nightly.
 508 .RE
 509 .LP
 510 .B JAVA_ROOT
 511 .RS 5
 512 The location for the java compilers for the build, generally /usr/java.
 513 .RE
 514 .LP
 515 .B OPTHOME
 516 .RS 5
 517 The gate-defined default location of things formerly in /opt; e.g.,
 518 /ws/onnv-tools.  This is used by nightly, but not the makefiles.
 519 .RE
 520 .LP
 521 .B TEAMWARE
 522 .RS 5
 523 The gate-defined default location for the Teamware tools; e.g.,
 524 /ws/onnv-tools/SUNWspro.  By default, it is derived from
 525 .BR OPTHOME .
 526 This is used by nightly, but not the makefiles.  There is no
 527 corresponding variable for Mercurial or Subversion, which are assumed
 528 to be installed in the default path.
 529 .RE
 530 .LP
 531 .B OPEN_SRCDIR
 532 .RS 5
 533 The open source tree is copied to this directory when simulating an
 534 OpenSolaris build (\fB\-S O\fR).  It defaults to $CODEMGR_WS/open_src.
 535 .LP
 536 .RE
 537 .B ON_CLOSED_BINS
 538 .RS 5
 539 OpenSolaris builds do not contain the closed source tree.  Instead,
 540 the developer downloads a closed binaries tree and unpacks it.  
 541 .B ON_CLOSED_BINS
 542 tells nightly
 543 where to find these closed binaries, so that it can add them into the
 544 build.
 545 .LP
 546 .RE
 547 .B CHECK_PATHS
 548 .RS 5
 549 Normally, nightly runs the 'checkpaths' script to check for
 550 discrepancies among the files that list paths to other files, such as
 551 exception lists and req.flg.  Set this flag to 'n' to disable this
 552 check, which appears in the nightly output as "Check lists of files."
 553 .RE
 554 .LP
 555 .B CHECK_DMAKE
 556 .RS 5
 557 Nightly validates that the version of dmake encountered is known to be
 558 safe to use.  Set this flag to 'n' to disable this test, allowing any
 559 version of dmake to be used.
 560 .RE
 561 .LP
 562 .B MULTI_PROTO
 563 .RS 5
 564 If "no" (the default), 
 565 .I nightly
 566 will reuse $ROOT for both the DEBUG and non-DEBUG builds.  If "yes",
 567 the DEBUG build will go in $ROOT and the non-DEBUG build will go in
 568 $ROOT-nd.  Other values will be treated as "no".  Use of the
 569 .B \-O
 570 flag forces MULTI_PROTO to "yes".
 571 .RE
 572 .LP
 573 .SH NIGHTLY HOOK ENVIRONMENT VARIABLES
 574 .LP
 575 Several optional environment variables may specify commands to run at
 576 various points during the build.  Commands specified in the hook
 577 variable will be run in a subshell; command output will be appended to
 578 the mail message and log file.  If the hook exits with a non-zero
 579 status, the build is aborted immediately.  Environment variables
 580 defined in the environment file will be available.
 581 .LP
 582 .B SYS_PRE_NIGHTLY
 583 .RS 5
 584 Run just after the workspace lock is acquired.  This is reserved for
 585 per-build-machine customizations and should be set only in /etc/nightly.conf
 586 .RE
 587 .LP
 588 .B PRE_NIGHTLY
 589 .RS 5
 590 Run just after SYS_PRE_NIGHTLY.
 591 .RE
 592 .LP
 593 .B PRE_BRINGOVER
 594 .RS 5
 595 Run just before bringover is started; not run if no bringover is done.
 596 .RE
 597 .LP
 598 .B POST_BRINGOVER
 599 .RS 5
 600 Run just after bringover completes; not run if no bringover is done.
 601 .RE
 602 .LP
 603 .B POST_NIGHTLY
 604 .RS 5
 605 Run after the build completes, with the return status of nightly - one
 606 of "Completed", "Interrupted", or "Failed" - available in the
 607 environment variable NIGHTLY_STATUS.
 608 .RE
 609 .LP
 610 .B SYS_POST_NIGHTLY
 611 .RS 5
 612 This is reserved for per-build-machine customizations, and runs
 613 immedately after POST_NIGHTLY.
 614 .RE
 615 .LP
 616 .SH REALMODE ENVIRONMENT VARIABLES
 617 .LP
 618 The following environment variables referenced by
 619 .I nightly
 620 are only required when the -X option is used.
 621 .LP
 622 .RE
 623 .B IA32_IHV_WS
 624 .RS 5
 625 Reference to the IHV workspace containing IHV driver binaries.
 626 The IHV workspace must be fully built before starting the ON realmode build.
 627 .LP
 628 .RE
 629 .B IA32_IHV_ROOT
 630 .RS 5
 631 Reference to the IHV workspace proto area.
 632 The IHV workspace must be fully built before starting the ON realmode build.
 633 .LP
 634 .RE
 635 .B IA32_IHV_PKGS
 636 .RS 5
 637 Reference to the IHV workspace packages.  If this is empty or the directory
 638 is non-existent, then nightly will skip copying the packages.
 639 .LP
 640 .RE
 641 .B IA32_IHV_BINARY_PKGS
 642 .RS 5
 643 Reference to binary-only IHV packages.  If this is empty or the directory
 644 is non-existent, then nightly will skip copying the packages.
 645 .LP
 646 .RE
 647 .B SPARC_RM_PKGARCHIVE
 648 .RS 5
 649 Destination for sparc realmode package SUNWrmodu.
 650 Yes, this sparc package really is built on x86.
 651 .SH FILES
 652 .LP
 653 .RS 5
 654 /etc/nightly.conf
 655 .RE
 656 .LP
 657 If present, nightly executes this file just prior to executing the
 658 .I env
 659 file.  
 660 .SH BUILDING THE IHV WORKSPACE
 661 .LP
 662 The IHV workspace can be built with
 663 .I nightly.
 664 The recommended options are:
 665 .LP
 666 .RS 5
 667 NIGHTLY_OPTIONS="-pmWN"
 668 .RE
 669 .LP
 670 None of the realmode environment variables needed for ON realmode builds
 671 are required to build the IHV workspace.
 672 .SH EXAMPLES
 673 .LP
 674 Start with the example file in usr/src/tools/env/developer.sh
 675 (or gatekeeper.sh), copy to myenv and make your changes.
 676 .LP
 677 .PD 0
 678 # grep NIGHTLY_OPTIONS myenv
 679 .LP
 680 NIGHTLY_OPTIONS="-ACrlapDm"
 681 .LP
 682 export NIGHTLY_OPTIONS
 683 .LP
 684 # /opt/onbld/bin/nightly -i myenv
 685 .PD
 686 .LP
 687 .SH SEE ALSO
 688 .BR bldenv (1)