Nokia

Nokia nmake Product Builder

 

Nokia nmake Product Builder
nmake 16 Release Notes

Table of Contents

Released: July 2014

1. Introduction
1.1 Supported Platforms
1.2 Software Requirements
1.3 Customer Support

2. New Features and Significant Enhancements
2.1 New :MAKE: feature to make directories
2.2 New edit operator for PATH search
2.3 New edit operator for probe attributes
2.4 Probe attribute handling and support for probe file collisions
2.5 nmakelog and xmakelog optimizations
2.6 New --help command line option

3. Detailed List of Changes
3.1 Baserules
3.2 cpp
3.3 Engine
3.4 Operators
3.5 Probe
3.6 Build Log
3.7 Miscellaneous
3.8 Variables
3.9 Options

4. Changes Impacting nmake 14 Makefiles
4.1 Probe overflow files
4.2 Date format change in the version string

5. Known Problems and Remarks
5.1 Known Problems
5.2 Remarks


1. Introduction

This document announces the new release of nmake version 16. nmake is fully supported and is in wide production use throughout Alcatel-Lucent, AT&T, and elsewhere.

These nmake 16 Release Notes discuss in detail the new features, and highlight bug fixes, additional enhancements, and known problems.

1.1 Supported Platforms

The nmake 16 release has been ported to many UNIX-based and UNIX-like systems and is available for HP-UX, Linux, Solaris, and Windows (under SFU/Interix). For a current list of platforms and available distribution packages see the nmake 16 release page on the nmake web site, http://nmake.alcatel-lucent.com. nmake is generally upward compatible with later OS releases in a series (for example, the series Solaris 2.5 through 2.10). Contact the Customer Support helpdesk (see below) with any compatibility questions or requests for porting to other systems.

1.2 Software Requirements

nmake 16 provides dependency-based Java build support. This feature currently requires an external Java source scanner called JavaDeps to extract inter-modules dependencies from Java source. nmake 16 requires JavaDeps release lu2.2.4 or greater. The latest version is downloadable from the nmake JavaDeps page. Installation instructions are also on that page. THIS PACKAGE IS ONLY REQUIRED BY PROJECTS PERFORMING JAVA BUILDS USING THE :JAVA: ASSERTION OPERATOR. JavaDeps release lu2.2.4 is itself written in Java; it requires jdk1.2.1 or higher version on its PATH in order to run. (Note that this does not restrict the project being built to jdk1.2.1 or higher.)

The buildlog2html command requires Java runtime 1.5 or above. It will also use Graphviz dot and neato commands, if available, to generate diagrams showing build log structure. Graphviz versions 2.16 or above are known to work. buildlog2html generates an HTML log from a structured build log.

The Windows-based version of nmake is based on the Windows Services for UNIX (SFU/Interix) porting/development environment from Microsoft and requires installation of the SFU package in order to run. The SFU package must be obtained from Microsoft and installed following their installation procedure. For more information see the Support for Windows page.

1.3 Customer Support

We provide patch support, where code changes are required, for the latest 2 point releases of nmake (currently releases nmake 16 and 15). Customers using older releases can still acquire help in using the product, but genuine problems found in older releases will not be fixed. Such problems may require an upgrade to a current release or, when possible, a makefile work-around will be provided.

Fee-based services are also available. These include makefile rewrites, conversion of non-nmake makefiles to nmake, integration with vendor tools, build assessments, and porting nmake to new platforms.

Contact Customer Support for any questions regarding nmake.


2. New Features and Significant Enhancements

2.1 New :MAKE: feature to make directories

The :MAKE: assertion operator now creates directories in the top viewpath node as needed. If a valid, corresponding right-hand-side (RHS) directory and makefile exists in the viewpath but not the top node then the directory will be created in the top node when recursing. RHS shell patterns are supported (including no RHS prerequisites which acts as an implicit *). The recurse_mkdir variable is used to adjust the behavior. Set recurse_mkdir=silent or recurse_mkdir=-1 to disable the mkdir command shell trace in the output while keeping the feature enabled. Set recurse_mkdir=0 to disable the feature and restore the previous behavior where directories are not created. By default recurse_mkdir=1 is set to enable the feature. The recurse_mkdir variable can be set on a per-makefile basis.

2.2 New edit operator for PATH search

The new :P=Y edit operator does a whence-style PATH search. For each token in the variable the PATH is searched for a corresponding executable file and the absolute path of the file is returned. Null is returned for tokens not found.

$ cat Makefile
CMDS = cc ls abc pq
targ : .MAKE
        print $(CMDS:P=Y)

$ nmake
/usr/bin/cc /bin/ls /opt/exp/bin/pq

2.3 New edit operator for probe attributes

The :P=P edit operator (probe path) now accepts the new option "A" to return the current probe attributes instead of the probe information file path. The probe attributes, along with the compiler path, are used in generating the probe information file hash name so different attribute values use different information files. Use the :P=PA form for a list of the probe attributes for the specified compiler. The attribute list is similar to the output from probe -a option.

$ cat Makefile
CC = /usr/bin/cc
targ : .MAKE
        print $(CC:P=PA=C)

$ nmake
PREROOT='' UNIVERSE='ucb' VERSION_OS='Linux2.6 Red Hat Enterprise Linux Server r
elease 5.7 (Tikanga)' VERSION_NMAKE='16' ATTRIBUTES='PREROOT UNIVERSE VERSIO N_O
S VERSION_NMAKE'

2.4 Probe attribute handling and support for probe file collisions

The probe command and nmake base rules now verify correct probe information files are being created and loaded by reading the compiler and probe attribute values, used to determine the file name hash, from the information file and verifying they are correct. The probe information files contain a new probe_attributes section listing the compiler and attribute names and values used for this verification.

The probe information file is lib/probe/language/tool/XXXXXXXXPPPPPP, where XXXXXXXX is a hex hash value of the compiler path and probe attributes, and PPPPPP is derived from the last six characters of the compiler path. Information file name collisions are now handled. In case of a collision a sub-directory named XXXXXXXXPPPPPP.d is created in the same directory with additional probe information files named a string of three decimal digits ranging from 001 to 999. When a collision occurs on look-up the files in the .d directory are scanned until matching compiler and attributes are found.

2.5 nmakelog and xmakelog optimizations

Several optimizations and enhancements have been made to the nmakelog and xmakelog operations making them generally simpler, more efficient and more robust. These enhancements are especially important when using xmakelog to capture build timing data for use in build performance optimization.

2.6 New --help command line option

The new --help option prints the version information, contact information and a list of the command line options and then exits without processing any makefiles.


3. Detailed List of Changes

3.1 Baserules

  1. 070011 - unnecessary warning during a clobber
    The clobber common action no longer issues a warning when Java source files have been removed.
  2. 070012 - "warning" duplicated in warning messages
    Fixed an issue that could cause a warning messages from :JAVA: builds to contain two consecutive "Warning:" strings.

3.2 cpp

  1. 110029 - cpp error with strings.h on RH5
    Macro function calls with no arguments where one or more arguments are expected are now supported. Such macro calls expand a null string for the arguments which is consistent with other popular tool chains. Warnings are issued in strict mode or when warnings are enabled.

3.3 Engine

  1. 080034 - no error from missing .SOURCE.h dirs in incremental build
    An update is now triggered when a prerequisite file with the .DONTCARE attribute goes missing in an incremental build. In general it is treated as though the file has been touched. Many C/C++ include files get .DONTCARE from being included inside #ifdef blocks. If a change is made in an incremental build that causes a .DONTCARE file to not be found when it was found in the previous build, the change will now trigger an update and reveal any possible errors caused by the change or update the target as needed. Previously such a change was ignored and any errors introduced would not be found until some later build triggered an update related to the files.
  2. 130007 - New :P=Y and :P=PA edit operators
    The :P=Y variable edit operator has been added to do a whence-style PATH search. The :P=P (probe path) variable edit operator has been extended with an option to list probe attributes by specifying :P=PA.
    See also New Features and Enhancements: New edit operator for PATH search.
    See also New Features and Enhancements: New edit operator for probe attributes.
  3. 130014 - add --help command line option
    The --help command line option has been added to display version information, contact information and a list of command line options.
    See also New Features and Enhancements: New --help command line option.
  4. 140002 - dir/file bind fails when dir bound down vpath
    Fixed an issue where nmake would fail to bind directory/file if directory did not exist in the top viewpath node and had already been bound down the viewpath. For example, if dir is not in the top node then expanding $("dir":T=F) would correctly return the directory down the viewpath but a subsequent expansion of $("dir/file":T=F) would return null. This has been fixed so the file is now correctly bound.
  5. 140004 - empty :L expansion down viewpath
    Fixed an issue where the :L (list directory) variable edit operator would incorrectly expand an empty list under the following conditions: 1) :L is applied to a relative directory that does not exist in the top viewpath node (expands an empty list as expected). 2) The directory is bound down the viewpath (for example using :T=F). 3) :L is applied to the bound path but it expands an empty list instead of the contents of the directory down the viewpath. This has been fixed so the :L operation in step 3 now lists the contents of the bound directory.

3.4 Operators

  1. 030004 - :MAKE: recurse mkdir
    The :MAKE: operator will now create directories in the top viewpath node as needed if a corresponding directory and makefile exists in the viewpath. Set recurse_mkdir=silent to disable the mkdir command shell trace in the output. Set recurse_mkdir=0 to disable the feature. By default recurse_mkdir=1 is set to enable the feature.
    See also New Features and Enhancements: New :MAKE: feature to make directories.
  2. 140001 - add warning for :MAKE: dir with no makefile
    The :MAKE: operator will now issue a warning message when a directory specified on the right-hand-side (RHS) exists with no valid makefile in the viewpath. Previously such directories were silently ignored. RHS shell patterns do not trigger a warning since they are designed to only recurse into directories with valid makefiles. A new warning is also issued when the viewpath is set and the RHS directory is specified in the top node with an absolute path but without a makefile in the directory since absolute paths are not searched through the viewpath. Other existing :MAKE: warnings have been re-worded for clarification.
  3. 130013 - different results with .SOURCE.class vs CLASSPATH
    When using the :JAVA: operator with .SOURCE.class the actual classpath search order could be specified inconsistently on the generated jdeps and javac command lines. Also, the classpath specified to the jdeps command could be different depending on whether the classpath was defined using .SOURCE.class or the CLASSPATH variable. This has been changed so the search paths are consistent between jdeps and javac and the use of .SOURCE.class or CLASSPATH does not affect the order. The classpath command line options are derived as follows -- "." : JAVACLASSDEST : .SOURCE.class : CLASSPATH. The new java_override_path variable allows the default search to be overridden which was previously not possible. Set java_override_path=1 to use .SOURCE.class as-is for the classpath without adding JAVACLASSDEST nor CLASSPATH.
  4. 130017 - .dummy.class error with localjavadeps down vpath
    When using the :JAVA: operator with the current directory specifed on the right-hand-side, if the localjavadeps file existed down the viewpath then an incremental build would fail to load localjavadeps resulting in a confusing error message about .dummy.class. This has been fixed so localjavadeps is correctly loaded and the build runs as normal. Two new error messages have also been added to indicate when localjavadeps is not successfully loaded or when localjavadeps is loaded but does not define the expected information indicating an invalid file.

3.5 Probe

  1. 130007 - probe attributes
    The probe information files are now verified after being loaded by checking that the compiler and probe attributes in the file are correct. Probe information file collisions are now handled in case different compilers and attribute sets generate the same file name hash. The information files now contain a comment showing the user and machine which ran probe and the time and date the file was originally generated. A new probe attribute called NMAKEVER has been added for the nmake version so different nmake releases will now use different probe files. NMAKEVER can be overridden by setting environment variable VERSION_NMAKE.
    See also New Features and Enhancements: Probe attribute handling and support for probe file collisions.
  2. 130009 - support probe_hints without /bin/ksh
    The probe_hints feature no longer requires /bin/ksh to be installed. Previously using probe_hints without /bin/ksh present caused many of the probe variables to be set to null. Now if a probe_hints script is present it is run with the same shell used to execute the main probe script. New error messages are issued when probe_hints exits with an error code or when the probe_hints values cannot be loaded.
  3. 130018 - probe script exits without error code
    When running under older versions of ksh the probe scripts would not exit with an error code when an error occurred. This has been fixed so the scripts now exit with the appropriate error code.
  4. 130022 - probe failure with concurrent executions
    The probe command now uses mkstemp(3) to generate its temporary output file name instead of using the process ID. This eliminates possible file clashes when probe is run simultaneously on different machines using a shared nmake installation.

3.6 Build Log

  1. 130021 - optimize and simplify nmakelog/xmakelog action expansion
    The .ACTIONWRAP rule used for nmakelog and xmakelog builds has been optimized and simplified bringing improved performance and other enhancements.
    See also New Features and Enhancements: nmakelog and xmakelog optimizations.
  2. 140003 - Document optimizations for build data collection
    Several new optimization techniques have been added to the buildlog documentation for more efficient build data collection using xmakelog.
  3. 140005 - support recursive coshell builds for nmakelog/xmakelog
    An issue causing incomplete build log markup from xmakelog when using coshell has been fixed. Recursive coshell builds are now properly supported by nmakelog and xmakelog.

3.7 Miscellaneous

  1. 130019 - fix cut and paste from man pages
    The man pages have been updated to support pasting example text into unicode aware terminals. Previously copying example text from the HTML man pages would paste a unicode minus sign instead of a dash for argument flags leading to errors.

3.8 Variables

  1. New baserules variable recurse_mkdir - see 030004
  2. New environment variable VERSION_MAKE - see 130007
  3. New baserules variable java_override_path - see 130013

3.9 Options

  1. New --help command line option - see 130014

4. Changes Impacting nmake 15 Makefiles

The changes in nmake 16 are largely backward compatible with nmake 15. Every effort has been made to insure code changes do not unexpectedly change the documented behavior of nmake features.

The following may impact makefiles or local tools.

4.1 Probe overflow files

Description
Probe information files my be created in a new overflow sub-directory in case of a file name collision. Typically probe information files are created in nmake_install_root/lib/probe/C/make/ for nmake information and nmake_install_root/lib/probe/C/pp/ for cpp information with a file name of XXXXXXXXPPPPPP where XXXXXXXX is a hex hash value of the compiler path and probe attributes, and PPPPPP is derived from the last six characters of the compiler path. If a different compiler and/or set of attributes generates the same hash file name as an existing file then an overflow directory named XXXXXXXXPPPPPP.d is created to store additional probe information files named 001 to 999. nmake and probe handle the overflow directory and files transparently to the user and under normal conditions no special action is needed. However any custom tools or makefile rules that directly access the probe information files may need to be updated to account for the new overflow directories and files.
Who is affected
Projects with tools or makefile rules that directly access the probe information files and assume the files only have paths of lib/probe/C/make/XXXXXXXXPPPPPP and lib/probe/C/pp/XXXXXXXXPPPPPP.
What to do
If the probe information files are being copied then be sure to do a recursive copy on the directories to include any overflow sub-directories and files, and preserve the permissions and time stamps since duplicate entries are resolved based on their permissions and time stamps (eg. cp -pr). Use the probe -k option to get the proper absolute pathname of a probe file. The -k option will correctly give the path to an overflow file when appropriate and can be used in conjunction with -ppath to get files in alternate probe repositories. Similarly, the nmake :P=P variable edit operator will also return the path to overflow files when needed within nmake rules.

4.2 Date format change in the version string

Description
The format of the date in the version string changed to YYYY-MM-DD to be compliant with the ISO standard date format. Previous releases used a format of MM/DD/YYYY or MM/DD/YY.
Who is affected
Projects parsing the date in the version string from one of the following sources:
What to do
Update your tools or makefile rules to recognise the new date format. The following example shows a makefile able to parse the date from MAKEVERSION in any of the above formats.
$ cat Makefile
DATE  = $(MAKEVERSION:M=[0-9][0-9][0-9/][0-9][0-9/\-]+)
YMD   = $(DATE:M=/:Y?$(DATE:C|\(..\)/\(..\)/\(.*\)|\3 \1 \2|)?$(DATE:C|-| |G)?)
YEAR  = $(YMD:O=1)
MONTH = $(YMD:O=2)
DAY   = $(YMD:O=3)

targ : .MAKE
        print MAKEVERSION = $(MAKEVERSION)
        print DATE  = $(DATE)
        print YEAR  = $(YEAR)
        print MONTH = $(MONTH)
        print DAY   = $(DAY)

$ nmake
MAKEVERSION = Alcatel-Lucent (Bell Labs) nmake 16 2014-06-11
DATE  = 2014-06-11
YEAR  = 2014
MONTH = 06
DAY   = 11

$ nmake
MAKEVERSION = Alcatel-Lucent (Bell Labs) nmake 15 06/11/2013
DATE  = 06/11/2013
YEAR  = 2013
MONTH = 06
DAY   = 11

$ nmake
MAKEVERSION = nmake (Lucent Technologies Bell Laboratories) lu3.4 06/15/01
DATE  = 06/15/01
YEAR  = 01
MONTH = 06
DAY   = 15

5. Known Problems and Remarks

5.1 Known Problems

The following is a list of known problems:

  1. 980009 - problem in link=*
    The problem about using linking (specified using the link variable) to install targets in any adjacent nodes in viewpathing still exists. You may still use linking to install, but only in alternate nodes -- installation using copying must alternate with installation using linking. The problem occurs with both symbolic and hard linking.

5.2 Remarks

  1. :LINK: does not handle archive files that are generated by :: or :LIBRARY: assertion operators. Users should avoid use of :LINK: on archive files.
  2. When the cpp -I-S flag is on, -D-M is disabled. Users should not use these two flags together.
  3. Users should avoid including the same header file with both <...>-style and "..."-style #include statements in source files managed by a single makefile. nmake will assign .STD.INCLUDE and .LCL.INCLUDE attributes to the header file, and this can result in incorrect -I lists in the compiler command lines generated.
  4. When the SHELL environment variable points to ksh93 you may get the following warnings when building your shell scripts: warning: line 1: `...` obsolete, use $(...). The warning is generated running your script with ksh -n. To eliminate the warning either update your shell scripts as indicated by the message or set SHELL to another version of ksh that does not generate the warning, such as ksh88.
  5. nmake does not have large file support and cannot reference files over 2 GiB in size on 32-bit systems. For work-arounds and more information see newsletter issue 26 and issue 42.

Last Update: Friday,12-Aug-2016 13:41:57 EDT