Nokia

Nokia nmake Product Builder

 

Nokia nmake Product Builder
alu3.9 Release Notes

Table of Contents

Released: July 2007

1. Introduction
1.1 Supported Hardware
1.2 Hardware Requirements
1.3 Software Requirements
1.4 Customer Support

2. New Features and Significant Enhancements
2.1 Introduction of Sun Grid support
2.2 Introduction of Eclipse CDT support
2.3 Certification of JDK 6
2.4 Robust license key handling - no longer sensitive to white space
2.5 Java and Jar performance improvements with very large packages
2.6 New variable edit operator :P=VL
2.7 Execution on default configuration of Windows XP SP2
2.8 Improved -I compiler flag generation

3. Bug Fixes and Enhancements
3.1 Baserules
3.2 cpp
3.3 Engine
3.4 Operators
3.5 Probe
3.6 Miscellaneous
3.7 Variables
3.8 Options

4. Changes Impacting lu3.8 Makefiles
4.1 Defining a .MAKEDONE rule
4.2 The MAKEVERSION variable
4.3 The maxjavawarn variable
4.4 The recurse_begin_message variable

5. Known Problems and Remarks
5.1 Known Problems
5.2 Remarks


1. Introduction

This document announces the new release of nmake version alu3.9. nmake is fully supported and is in wide production use throughout Alcatel-Lucent, AT&T, and elsewhere.

These alu3.9 Release Notes discuss in detail the new features, and highlight bug fixes, additional enhancements, and known problems.

1.1 Supported Hardware

The alu3.9 release has been ported to many UNIX-based and UNIX-like systems. For a current list see the Availability Chart on the nmake web site, http://www.bell-labs.com/project/nmake/. Or contact the Customer Support helpdesk below.

1.2 Hardware Requirements

1.3 Software Requirements

This release of nmake is available for HP-UX, Linux, Solaris, and Windows (under SFU/Interix). See the release alu3.9 download page for a listing of available distribution packages. nmake is generally upward compatible with later OS releases in a series (for example, the series Solaris 2.5 through 2.10); contact Technical Support with any compatibility questions or with any requests for porting to other systems.

The Windows-based version of nmake is based on the Windows Services for UNIX (SFU/Interix) porting/development environment from Microsoft and requires installation of the SFU package in order to run. The SFU package must be obtained from Microsoft and installed following their installation procedure. For more information see the Support for Windows page.

nmake alu3.9 provides dependency-based Java build support. This feature currently requires an external Java source scanner called JavaDeps to extract inter-modules dependencies from Java source. nmake alu3.9 requires JavaDeps release lu2.2.2; this version is downloadable from the nmake JavaDeps page. Installation instructions are also on that page. THIS PACKAGE IS ONLY REQUIRED BY PROJECTS PERFORMING JAVA BUILDS USING THE :JAVA: ASSERTION OPERATOR. JavaDeps release lu2.2.2 is itself written in Java; it requires jdk1.2.1 or higher version on its PATH in order to run. (Note that this does not restrict the project being built to jdk1.2.1 or higher.)

1.4 Customer Support

We provide patch support, where code changes are required, for the latest 2 point releases of nmake (currently releases alu3.9 and lu3.8). Customers using older releases can still acquire help in using the product, but genuine problems found in older releases will not be fixed. Such problems may require an upgrade to a current release or, when possible, a makefile work-around will be provided.

Fee-based services are also available. These include makefile rewrites, conversion of non-nmake makefiles to nmake, integration with vendor tools, build assessments, and porting nmake to new machines.

Contact Customer Support for any questions regarding nmake.


2. New Features and Significant Enhancements

2.1 Introduction of Sun Grid support

This feature allows job action blocks to be piped to a job distribution utility such as the Sun Grid qrsh. Each triggered action block is piped to a separate invocation of qrsh or other distribution utility. No makefile changes are needed, the feature works for arbitrary existing makefiles. The build log serialization feature has also been extended to support concurrently executing jobs arising from this feature. Currently, all regular shell jobs are spun off without regard to the .LOCAL attribute.

The following variables are used to control the feature:

sge_qrsh
To enable the feature set sge_qrsh=1. Default: sge_qrsh = 0
QRSH
The job distribution utility. Default: QRSH = qrsh
QRSHFLAGS
Command line flags for the job distribution utility. Default: QRSHFLAGS += -cwd -noshell -V
SGESHELL
The remote shell to invoke. Default: SGESHELL := $(SGESHELL|COSHELL|SHELL|"ksh")

2.2 Introduction of Eclipse CDT support

A new recurse message customization feature allows nmake to be integrated with Eclipse CDT. Eclipse CDT is an Eclipse-based IDE for C and C++ development. The new feature is capable of emulating GNU make recurse messages. The approach has the additional advantage that it also allows interoperability with other compilation error tracking tools such as GNU Emacs compilation mode. The feature design has the advantages that it works for any recursive nmake invocation, not just :MAKE:/:CC:, supports reporting of make numeric recursion levels, and is completely automatic, no special nmake options need be added by the user to get it to work. The approach is backward compatible and should work with arbitrary existing Makefiles.

For help setting up Eclipse CDT to use nmake see the Eclipse CDT Support page on the nmake web site at http://www.bell-labs.com/project/nmake/manual/eclipsecdt.html. The provided eclipserules.mk file is not needed when using nmake alu3.9. Instead just set recurse_begin_message=gnu to configure the proper recurse messages.

The details of the new recurse message feature are as follows. recurse_begin_message and recurse_end_message allow customization of the format and content of messages output at entry and exit of recursive nmake invocations; in addition recurse_begin_message may now take 2 special keyword values:

    recurse_begin_message = default
    recurse_begin_message = gnu

Setting recurse_begin_message to "default" provides backward compatible nmake recurse messages. This prints the target directory/file before recursion and nothing after recursion. The default messages are output only for baserules provided recursion facilities such as :MAKE:. This is the default setting for recurse_begin_message.

Setting recurse_begin_message to "gnu" provides GNU make compatible recurse messages. These messages emulate gnu make recurse messages and contain the current recursion level and target directory. This format is especially useful when using nmake with development tools that already understand the GNU recurse message formats. These development tools typically use recurse messages to track the current directory in recursive builds, allowing the tool to perform automatic mapping of compilation errors to the source file and line containing the error. Examples of such tools are the Eclipse CDT C/C++ IDE and GNU Emacs compilation mode.

If recurse_begin_message is not one of the reserved keywords, recurse_begin_message and recurse_end_message specify customized messages. In this case nmake prints the expanded value of recurse_begin_message when it starts up and the expanded value of recurse_begin_message when it exits.

gnu style and customized messages are automatically output for all recursive nmake invocations, not just invocations initiated using baserules assertions such as :MAKE:. This output is automatic and does not require any special setup in user Makefiles. If desired, this behavior may be suppressed by resetting recurse_begin_message on the recursive nmake command line. Setting recurse_begin_message and recurse_end_message to null completely suppresses all recurse messages.

For current information on using Eclipse CDT, JDT and more, check the Eclipse Support page on the nmake web site at http://www.bell-labs.com/project/nmake/manual/eclipsesupport.html.

2.3 Certification of JDK 6

Sun JDK 6 (1.6.0) has been certified with nmake alu3.9.

2.4 Robust license key handling

The license key parser is no longer sensitive to white space making maintenance and installation of key files easier. The parser does the following to avoid common sources of invalid license errors that occur when installing keys.

2.5 Java and Jar performance improvements with very large packages

A performance bottleneck was identified in the :JAVA: and :JAR: support and a rule has been rewritten to improve the performance. It will be most noticeable for large Java packages. The following benchmarks measure the time the affected rule spent processing the list of files in the java package, not the entire compilation or archiving time. The old rule took about 1 hour and 28 minutes to process a package of 45000 files. The new rule processes the same set of files in about 18 seconds. The improvement is also noticed using a more reasonably sized package of about 3000 files. The old rule took about 22 seconds to process the 3000 file package where the new rule takes about 1 second.

The maxjavawarn baserule variable introduced in lu3.8 is no longer needed and has been removed.

2.6 New variable edit operator :P=VL

The new variable edit operator :P=VL has been added to the engine. :P=VL returns the view local path name for each token that is an atom bound to a file. Paths in the viewpath are canonicalized and converted to their corresponding relative local path. Paths outside the viewpath are returned unmodified. Unbound paths return null.

For the following example assume these values:

VPATH = /v1:/v2:/v3
PWD = /v1/src/abc

Variable Expanded Value
$("/v2/src/file1":P=VL) ../file1
$("../../src/file1":P=VL) ../file1
$("/v3/include/hdr.h":P=VL) ../../include/hdr.1

2.7 Execution on default configuration of Windows XP SP2

nmake can now run on Windows XP SP2 as configured by default. Previously it could not run if Data Execution Prevention (DEP) was enabled which is the default in SP2.

2.8 Improved -I compiler flag generation

This release fixes -I compiler flag generation for several corner cases. The problems caused -I flags to be omitted from compile lines leading to build errors. Most of the problems came when using the native preprocessor which is the default for popular compilers such as Sun Studio Compilers, HP aCC and gcc/g++. These fixes provide for more robust builds.


3. Bug Fixes and Enhancements

3.1 Baserules

  1. 010071 - replace coshell with 3rd party tools
    Job action blocks can now be piped to external job distribution utilities such as the Sun Grid Engine.
    For details see - New Features and Enhancements: Introduction of Sun Grid support
  2. 040071 - dropped -I flag when using nativepp=1
    Fixed a problem where -I compile flags were dropped when the compiler does not support -I- and the compiler's native preprocessor was used. For such configurations prefixinclude=0 was not being set which could lead to -I flags for prefixed include files to be omitted. The default value of variable nativepp is now -1 (changed from 1) which allows prefixinclude to be set appropriately for all compilers.
  3. 050058 - prefixinclude for nativepp not getting -I flag
    When a file was included using its absolute path, the -I flags for the files it included in its directory could be omitted. This has been fixed.
  4. 060000 - prefixinclude -I flag dropped in new vpath node
    Fixed a problem where -I compile flags could be dropped for an include file that is down the viewpath and no other files are included from its corresponding directory in the local node and the native preprocessor is being used.
    This solution also adds the new :P=VL variable edit operator. For details see - New Features and Enhancements: New variable edit operator :P=VL
  5. 060054 - dropped -I flag from prefixed/non-prefixed include
    Fixed a problem where a -I compile flag could be omitted for an include file that is included both with and without a directory prefix from different source files when using the native preprocessor.
  6. 060063 - no exit with compile error when using nmakelog
    A trap defined in the default .ACTIONWRAP rule used for output serialization failed to catch errors from the shell jobs. The .ACTIONWRAP rule has been fixed so errors are trapped so nmake can exit when appropriate.
  7. 070000 - suspicious test in .SOURCE.LCL.INCLUDE
    Fixed a problem that could cause unnecessary -I compile flags when nmake cpp was being used.
  8. 070003 - integrate eclipserules.mk
    Integration with Eclipse is supported by setting recurse_begin_message=gnu which provides gnu make style recurse messages in build output. Eclipse and other tools use these messages to determine where source files are located, these messages are essential in tracking files with non-unique names. By default recurse_begin_message=default which provides the traditional nmake style recurse messages. Other values of recurse_begin_message define the actual recurse message. When using anything other than the default, recurse messages are now printed for every recursive make, not just those using :MAKE:, including user defined actions. The variable MAKELEVEL has been added. MAKELEVEL indicates the recursion level of the current nmake instance. It is set to 0 in the top level nmake and incremented by 1 in each recursive nmake invocation.
    For details see - New Features and Enhancements: Introduction of Eclipse CDT support

3.2 cpp

  1. No changes to cpp in this release.

3.3 Engine

  1. 020088 - allow to suppress MAKEEXPORT output
    The exporting of MAKEEXPORT variables is no longer shown in the shell trace output. MAKEEXPORT lists nmake variables to be exported to the shell for shell jobs. Previously the definition and exportation of each variable was included in the shell trace output for every shell job resulting in a lot of redundant output. Now MAKEEXPORT is not included in the shell trace providing cleaner output but the functionality is the same. This change is used in implementation of 070003 to pass down recursion levels. Export messages are included in debug level 6 output.
  2. 050021 - remove whitespace sensitivity in license file
    The license key parser now ignores extra white space to avoid common sources of invalid license errors.
    For details see - New Features and Enhancements: Robust license key handling
  3. 060000 - :P=VL edit operator
    The new variable edit operator :P=VL has been added to return the view local path name for each token that is an atom bound to a file.
    For details see - New Features and Enhancements: New variable edit operator :P=VL
  4. 060028 - $(^) is null
    Fixed a problem where the automatic variable $(^) would expand to null for a file down the viewpath when the current makefile was in the same directory and referenced using the nmake -f flag with the absolute path to the makefile. The makefile's directory would get recorded as having view index 0 instead of the correct index, then other files in the same directory would inherit view index 0. nmake now records the proper view index.
  5. 060046 - prerequisite list is null
    Fixed a problem where automatic prerequisite variables (such as $(*) and $(>)) failed to expand a prerequisite when the prerequisite was specified with a path, its basename matched the target basename, and the prerequisite's path was in the default search path. This lead to the target being bound with the same path as the prerequisite leading to the prerequisite being dropped from the list. The prerequisite is no longer dropped.
  6. 060070 - :C edit operator core dump
    Expanding the variable $(...:C? ?|?) no longer causes a core dump nor corrupt output.
  7. 070001 - keepgoing changes errors to warnings
    Some error messages were being output as warnings when the keepgoing option was enabled. Errors are no longer changed to warnings.
  8. 070005 - $(...|VAR) expansion wrong
    Using "|" to expand the first non-null variable in a list would skip the automatic variable $(...), even though it is not null, and expand the next non-null variable instead. For example, $(...|VAR) would expand VAR if VAR was not null and expand null otherwise even though ... was not null. This has been fixed so $(...) now expands correctly when | is used.
  9. 070025 - port to Interix /bin/cc
    The purpose of the port to /bin/cc is to workaround the issue of gcc generated executables dumping core on Windows XP SP2 that was traced to the Data Execution Prevention (DEP) feature that is enabled by default on XP SP2. The build was moved to /bin/cc in order to maintain the security advantages of DEP as opposed to requiring DEP be disabled.
    See also - New Features and Enhancements: Windows version runs on Windows XP SP2

3.4 Operators

  1. 060010 - JAR.mk does not respect keepgoing option
    When a :JAR: assertion generated a JARROOT error and the keepgoing (-k) option was enabled nmake would stop the build instead of continuing to process the rest of the targets in the makefile. This has been fixed so the keepgoing option is now respected.
  2. 060014 - full Java rebuilds until clean build
    If a Java package had a compile error during its initial build the entire package would be rebuilt on subsequent builds until it was successful. Now if the initial build fails only the necessary files are rebuilt instead of the entire package.
  3. 060017 - :JAR: problem using * pattern without file extension
    Previously there was no way to tell :JAR: to pick up files without a dot (.) in the filename. Using *, as in $(JARROOT)/*, would appear to work for the first build but didn't actually track all the files in the jar. Touching a file would not cause the jar to be updated. The rules were written with the assumption of at least one dot in the filename so $(JARROOT)/*.* needed to be used to correctly maintain the jar, but this would not pick up files without a dot. This has been fixed so * can now be used to match all files.
  4. 060022 - keepgoing jar targs do not rebuild after failure
    When the keepgoing (-k) option was enabled and a :JAR: target failed, the target was recorded as up to date in the state file so the next build would not try to build the jar again. Failed jar targets are now rebuilt as expected when using the keepgoing option.
  5. 060041 - :JAR: rebuild from multiple patterns with matching dir
    A jar target would unnecessarily rebuild in a new viewpath node when the RHS contained two or more patterns with the same directory. For example, specifying patterns com/*.class and com/*.conf would trigger an unnecessary build. This has been fixed.
  6. 060049 - JARPREVIOUS bottleneck
    A performance enhancement was made to the :JAVA: and :JAR: operators.
    For details see - New Features and Enhancements: Java and Jar performance improvements with very large packages

3.5 Probe

  1. 060051 - probe scripts using ksh syntax
    Some ksh specific integer operations in the probe shell scripts caused some probe tests to fail when bash was used as the login shell. The scripts are now portable between ksh and bash with some additional syntax changes to support bash. The scripts also attempt to relaunch themselves with ksh or bash if they are being executed by a different shell.
  2. 070015 - probe misses SUNWCCh headers with Studio 11 compiler
    Fixed the probe script to correctly set CC.SUFFIX.LINKHEADER for the SUN Studio 11 compiler. Also fixed a problem where SUNWCCh files could be picked up from wrong directories for all SUN compilers.

3.6 Miscellaneous

  1. 070006 - certify jdk 6
    Sun JDK 6 (1.6.0) has been certified with nmake alu3.9.

3.7 Variables

  1. New variable QRSH - see 010071.
  2. New variable QRSHFLAGS - see 010071.
  3. New variable sge_qrsh - see 010071.
  4. New variable SGESHELL - see 010071.
  5. Change to variable recurse_begin_message - see 070003.
  6. Change to variable recurse_end_message - see 070003.
  7. New variable MAKELEVEL - see 070003.

3.8 Options

  1. Change to option -k / keepgoing - see 070001.

4. Changes Impacting lu3.8 Makefiles

The changes in release alu3.9 are largely backward compatible with lu3.8. Every effort has been made to insure code changes do not unexpectedly change the documented behavior of nmake features.

The following may impact builds and require changes.

4.1 Defining a .MAKEDONE rule

Description
The default baserules now define a .MAKEDONE rule. Locally defined .MAKEDONE rules should still work as expected. Local rules defined as .MAKEDONE prerequisites should also work as expected unless they are defined as .USE rules. Those defined as .USE rules will not be triggered.
Who is affected
Projects that define a local .MAKEDONE prerequisite rule with .USE. For example:
.MAKEDONE : .localdone
.localdone : .USE .VIRTUAL .FORCE
        silent echo "Build Complete"
What to do
In this case the .USE special atom is not needed. Remove .USE and the rule will work again.
.MAKEDONE : .localdone
.localdone : .VIRTUAL .FORCE
        silent echo "Build Complete"

4.2 The MAKEVERSION variable

Description
The string "Lucent(R)" was changed to "Alcatel-Lucent" in the MAKEVERSION variable.
Who is affected
Projects checking the MAKEVERSION variable for "Lucent(R)".
What to do
Adjust your makefiles to match the new string. MAKEVERSION expands as follows:
Alcatel-Lucent nmake (Bell Labs) alu3.9 06/15/2007

4.3 The maxjavawarn variable

Description
The maxjavawarn variable is obsolete and is no longer used. The variable was introduced in release lu3.8.
Who is affected
Projects that define maxjavawarn.
What to do
The definition can be removed but keeping it defined will cause no harm.
See also
New Features and Enhancements - Java and Jar performance improvements

4.4 The recurse_begin_message variable

Description
Usage of the recurse_begin_message and recurse_end_message variables has changed slightly. The $PWD shell variable in recurse_begin_message and recurse_end_message no longer works because the messages are output using the nmake print command rather than sent to the shell. The nmake $(PWD) variable works.
recurse_begin_message and recurse_end_message strings beginning with a + or other nmake "print" option flags fail. Use "--" to terminate print command option processing.
In the new implementation the custom recurse messages are output by the recursive nmake invocation, in the old implementation the messages are output before recursing. recurse_begin_message and recurse_end_message need to be defined in the recursively called Makefile. May want to consider putting the message definitions in the project's global makefile.
There are aliases available for gnu make style and traditional nmake style recurse messages (recurse_begin_message=gnu and recurse_begin_message=default respectively). Also, when recurse_begin_message is anything other than the default then recurse messages are printed for every recursive make, not just those using :MAKE:, including user defined actions. You may notice more recurse messages in the build output.
Who is affected
Projects that define recurse_begin_message.
What to do
If using $PWD or ${PWD} shell variables in recurse_begin_message or recurse_end_message change to $(PWD) nmake variable.
If recurse_begin_message or recurse_end_message begin with a + or other nmake print option flag add -- to the beginning. For example:
recurse_begin_message = -- + Entering directory $(PWD)
recurse_end_message = -- + Leaving directory $(PWD)
If gnu make style messages are hard coded in recurse_begin_message and recurse_end_message you can now optionally use the "gnu" alias to define both automatically:
recurse_begin_message = gnu
See also
Bug fixes and enhancements - integrate eclipserules.mk


5. Known Problems and Remarks

5.1 Known Problems

The following is a list of known problems:

  1. 980009 - problem in link=*
    The problem about using linking (specified using the link variable) to install targets in any adjacent nodes in viewpathing still exists. You may still use linking to install, but only in alternate nodes -- installation using copying must alternate with installation using linking. The problem occurs with both symbolic and hard linking.
  2. 030042 - :cc: generated header rebuild
    If a C file is built by using :cc: and this C file shares a generated header file with a C++ file specified in the same makefile, the generated header file may get regenerated during the build causing the source files to keep recompiling. This can be avoided by building the C++ files before the C files, or by generating the header file in a separate makefile at an earlier time during the build.
  3. 990099 - metarules not searched when CC.REPOSITORY is set
    The metarules may not be searched under the following conditions: there is a user defined metarule for a non-compiled file, variable CC.REPOSITORY is defined and CC.DIALECT contains either PTRIMPLICIT or PTRMKDIR, and :: is used to trigger the metarule. Note that CC.REPOSITORY and CC.DIALECT are defined as such in the probe file for some C++ compilers. The work-around is to add .IMPLICIT to the :: assertion, or to not use the :: assertion (such as add the target to :ALL: instead.)
  4. 020018 - adding shared lib building in same makefile won't re-link
    An executable target will not be re-linked when a shared library that is built in the same makefile is added as a prerequisite to the executable. The work-around is to set force_shared=1.
  5. 020051 - first metarule is used by mistake
    When multiple metarules generate the same source file target, such as %.c, the first metarule defined matching an existing file will be triggered instead of the metarule matching the explicit file prerequisite.
  6. 040043 - metarule $(>) expands non-matching prerequisites
    The $(>) and $(*) automatic variables used in meta rules to expand the prerequisite files may expand files that don't match the prerequisite pattern. In the following example the built in %.c->%.o rule expands the .xyz file which is included on the command line when compiling the .c.
    $ cat Makefile
    %.c : %.xyz
            cp $(>) $(<)
    
    abc.o :: abc.xyz
    
    $ nmake
    + cp abc.xyz abc.c
    + cc -O -I- -c abc.c abc.xyz
    
    Fix this by defining the %.c->%.o meta rule locally to filter out the unwanted files.
    cat Makefile
    %.c : %.xyz
            cp $(>) $(<)
    
    %.o : %.c (CC) (CCFLAGS)
            $(CC) $(CCFLAGS) -c $(>:N=*.c)
    
    abc.o :: abc.xyz
    
    $ nmake
    + cp abc.xyz abc.c
    + cc -O -I- -c abc.c
    
  7. 030057 - Sun Compilers
    A couple projects have reported problems using Sun 8 and above compilers where -I flags pointing to the system include files cause compiler errors. One work-around is to strip these -I flags from the command line. For example:
    %.o : %.c (CC) (CCFLAGS)
            $(CC) $(CCFLAGS:N!=-I/opt/sc8.0a/SUNWspro*) -c $(>)
    
  8. 060019 - .SHARED.o strips ../ files from link line
    Shared libraries cannot be created with .o files referenced with a path starting with ../.
  9. 070010 - fails to bind large files
    nmake does not have large file support and cannot reference files over 2 GiB in size.

5.2 Remarks

  1. :LINK: does not handle archive files that are generated by :: or :LIBRARY: assertion operators. Users should avoid use of :LINK: on archive files.
  2. When the cpp -I-S flag is on, -D-M is disabled. Users should not use these two flags together.
  3. Users should avoid including the same header file with both <...>-style and "..."-style #include statements in source files managed by a single makefile. nmake will assign .STD.INCLUDE and .LCL.INCLUDE attributes to the header file, and this can result in incorrect -I lists in the compiler command lines generated.

Last Update: Friday,12-Aug-2016 12:30:28 EDT