Nokia

Nokia nmake Product Builder

 

Nokia nmake Product Builder
nmake 11 Release Notes

Table of Contents

Released: July 2009

1. Introduction
1.1 Supported Hardware
1.2 Hardware Requirements
1.3 Software Requirements
1.4 Customer Support

2. New Features and Significant Enhancements
2.1 Refined structured build logs
2.2 New command - buildlog2html
2.3 Enhancements to versioned shared libraries
2.4 W3C conformant time zone designator
2.5 New command - astutil

3. Bug Fixes and Enhancements
3.1 Baserules
3.2 cpp
3.3 Engine
3.4 Operators
3.5 Probe
3.6 Miscellaneous
3.7 Variables
3.8 Options

4. Changes Impacting nmake 10 Makefiles
4.1 Versioned shared libraries
4.2 Shared library .no file
4.3 Date format field %z
4.4 Structured build logs / xmakelog

5. Known Problems and Remarks
5.1 Known Problems
5.2 Remarks


1. Introduction

This document announces the new release of nmake version 11. nmake is fully supported and is in wide production use throughout Alcatel-Lucent, AT&T, and elsewhere.

These nmake 11 Release Notes discuss in detail the new features, and highlight bug fixes, additional enhancements, and known problems.

1.1 Supported Hardware

The nmake 11 release has been ported to many UNIX-based and UNIX-like systems. For a current list see the Download Chart on the nmake web site, http://www.bell-labs.com/project/nmake/. Or contact the Customer Support helpdesk below.

1.2 Hardware Requirements

1.3 Software Requirements

The nmake 11 release is available for HP-UX, Linux, Solaris, and Windows (under SFU/Interix). See the nmake 11 download page for a listing of available distribution packages. nmake is generally upward compatible with later OS releases in a series (for example, the series Solaris 2.5 through 2.10); contact Technical Support with any compatibility questions or requests for porting to other systems.

The Windows-based version of nmake is based on the Windows Services for UNIX (SFU/Interix) porting/development environment from Microsoft and requires installation of the SFU package in order to run. The SFU package must be obtained from Microsoft and installed following their installation procedure. For more information see the Support for Windows page.

nmake 11 provides dependency-based Java build support. This feature currently requires an external Java source scanner called JavaDeps to extract inter-modules dependencies from Java source. nmake 11 requires JavaDeps release lu2.2.4 or greater. The latest version is downloadable from the nmake JavaDeps page. Installation instructions are also on that page. THIS PACKAGE IS ONLY REQUIRED BY PROJECTS PERFORMING JAVA BUILDS USING THE :JAVA: ASSERTION OPERATOR. JavaDeps release lu2.2.4 is itself written in Java; it requires jdk1.2.1 or higher version on its PATH in order to run. (Note that this does not restrict the project being built to jdk1.2.1 or higher.)

1.4 Customer Support

We provide patch support, where code changes are required, for the latest 2 point releases of nmake (currently releases nmake 11 and 10). Customers using older releases can still acquire help in using the product, but genuine problems found in older releases will not be fixed. Such problems may require an upgrade to a current release or, when possible, a makefile work-around will be provided.

Fee-based services are also available. These include makefile rewrites, conversion of non-nmake makefiles to nmake, integration with vendor tools, build assessments, and porting nmake to new machines.

Contact Customer Support for any questions regarding nmake.


2. New Features and Significant Enhancements

2.1 Refined structured build logs

Support for structured build logs has been refined and extended making the feature more robust. The enhancements include the following:

For more details see the xmakelog(1) man page.

2.2 New command - buildlog2html

A new experimental command called buildlog2html is included in the nmake package. The command takes a structured build log output by xmakelog and creates browsable HTML build log to provide enhanced visualization, navigation, summarization, and analysis. The HTML build log provides the following:

To get started first run a build using xmakelog instead of nmake or nmakelog. Use the xmakelog --info flag to set name:value pairs to record and display additional information in the logs. Any number of user-defined --info arguments may be provided. If 'build' is defined it is used as the build id in the HTML log, if it is not defined then a reasonable default is used.

$ xmakelog --info project:MYPROJ --info build:20090619 --info generic:proj1.0 \
> makelog.txt

$ ls -l makelog*
-rw-r--r-- 1 richb richb   60 Jun 19 13:41 makelog.txt
-rw-r--r-- 1 richb richb 1941 Jun 19 13:41 makelog.xml

The buildlog2html command can now be used to generate the HTML log. The command has two forms, the first sets up a build log directory and copies the XML and text logs to the directory. The second form runs the transformation and generates the HTML. This allows the transformation step to be run on a different machine, for example a web server instead of the build machine. The transformation requires Java 1.5 or greater in the PATH. If the dot and neato commands from the graphiz package are picked up in the PATH they will be used to create graphical views. Graphiz version 2.18 is known to work. For details on usage see the buildlog2html(1) man page.

To set up the new build log directory use the -x flag to suppress the transformation so it can be run later, possibly on another machine. Specify the log root directory with the -b flag. This directory will contain a subdirectory for each build log and should be accessible by your web server. If the -d flag is provided it sets the name of the build log directory created under the log root. If -d is not provided then a default is used, however including -d gives an optimization by skipping the need to run a Java application to parse the XML to determine the default and also allows this step to run on a machine that may not have Java 1.5 or greater available.

$ buildlog2html -x -b /www/bldlogs -d bld20090619
+ cp -p makelog.xml /www/bldlogs/bld20090619/buildlog.xml
+ cp -p makelog.txt /www/bldlogs/bld20090619/buildlog.txt
+ cp -p /tools/nmake/sparc5/11/lib/builddata/buildlog.css /www/bldlogs/buildlog.css
+ cp -p /tools/nmake/sparc5/11/lib/builddata/icon.gif /www/bldlogs/icon.gif

Now the HTML can be generated. If running on another machine then go to the build log root directory that was specified above with the -b flag. Then run buildlog2html with the build log directory to process on the command line. Multiple directories can be specified to bring them all up to date. A top-level index.html is created in the current directory that provides links to all the build logs under the directory. The index is updated when new build log directories are added and processed. After the transformation completes load the top index.html in your web browser.

$ cd /www/bldlogs

$ buildlog2html bld20090619 
+ java -cp /tools/nmake/sparc5/11/lib/builddata/saxon9.jar net.sf.saxon.Transfor
m -s:bld20090619/buildlog.xml -xsl:/tools/nmake/sparc5/11/lib/builddata/buildlog
.xsl -t logname=bld20090619
Saxon 9.1.0.6J from Saxonica
Java version 1.5.0
Stylesheet compilation time: 9235 milliseconds
Processing file:/home/richb/mrs/090016/bld20090619/buildlog.xml
... [ output deleted ] ...

$ ls -l *.html
-rw-r--r-- 1 richb richb 2101 Jun 19 14:37 index.html

$ ls -l bld20090619/
total 120
-rw-r--r-- 1 richb richb 8175 Jun 19 14:25 build_summary.html
-rw-r--r-- 1 richb richb 3440 Jun 19 14:25 build_summary.xml
-rw-r--r-- 1 richb richb   60 Jun 19 13:41 buildlog.txt
-rw-r--r-- 1 richb richb 1941 Jun 19 13:41 buildlog.xml
-rw-r--r-- 1 richb richb  334 Jun 19 14:25 graph1.dot
-rw-r--r-- 1 richb richb 1894 Jun 19 14:25 graph1.html
-rw-r--r-- 1 richb richb  214 Jun 19 14:25 graph1.map
-rw-r--r-- 1 richb richb 1952 Jun 19 14:25 graph1.png
-rw-r--r-- 1 richb richb  925 Jun 19 14:25 graph1_expanded.dot
-rw-r--r-- 1 richb richb 2190 Jun 19 14:25 graph1_expanded.html
-rw-r--r-- 1 richb richb  483 Jun 19 14:25 graph1_expanded.map
-rw-r--r-- 1 richb richb 3936 Jun 19 14:25 graph1_expanded.png
-rw-r--r-- 1 richb richb  334 Jun 19 14:25 graph2.dot
-rw-r--r-- 1 richb richb 1895 Jun 19 14:25 graph2.html
-rw-r--r-- 1 richb richb  215 Jun 19 14:25 graph2.map
-rw-r--r-- 1 richb richb 2223 Jun 19 14:25 graph2.png
-rw-r--r-- 1 richb richb  925 Jun 19 14:25 graph2_expanded.dot
-rw-r--r-- 1 richb richb 2193 Jun 19 14:25 graph2_expanded.html
-rw-r--r-- 1 richb richb  486 Jun 19 14:25 graph2_expanded.map
-rw-r--r-- 1 richb richb 4646 Jun 19 14:25 graph2_expanded.png
-rw-r--r-- 1 richb richb 1810 Jun 19 14:25 index.html
-rw-r--r-- 1 richb richb 1810 Jun 19 14:25 list.html
-rw-r--r-- 1 richb richb 2125 Jun 19 14:25 list_expanded.html
-rw-r--r-- 1 richb richb 4443 Jun 19 14:25 makefile_1.html
-rw-r--r-- 1 richb richb 6575 Jun 19 14:25 makefile_1_1.html
-rw-r--r-- 1 richb richb 2828 Jun 19 14:25 table.html
-rw-r--r-- 1 richb richb 3763 Jun 19 14:25 table_expanded.html

The results of example transformations are shown in the screenshots below. They were generated from typical recursive builds consisting of 15 Makefiles and 458 leaf (non-recursive) targets. Build errors, shown in red, were deliberately included to show how they are handled. Click any of the screenshots below to jump to the full example.

[ Note 2010-07-31: See the Build Log page for more recent examples and documentation. ]

[index screenshot] [list screenshot] [graph1 screenshot] [makefile screenshot] [summary screenshot] [table screenshot] [expanded graph1 screenshot] [graph2 screenshot]

2.3 Enhancements to versioned shared libraries

The following enhancements were made to the :LIBRARY: operator when making a shared library without disabling the library version number.

2.4 W3C conformant time zone designator

The time format output from the :F=%(format)T edit operator now supports a time zone format that compiles with conventions defined by the World Wide Web Consortium. This is useful for XML element attributes containing date and time information for processing with other tools that support the same conventions. The following format fields are affected:

Previously %z was documented as the time zone type but the field did not function properly and expanded null. Time zone type has been fixed and moved to the %q field. The new %z and %:z fields return the time zone offset in hours and minutes and are consistent with other date commands.

2.5 New command - astutil

A new command called astutil is included in the nmake package. astutil provides an interface to various libast routines to make features available outside of nmake. Currently it supports the date formatting features to provide a generic date command and a new xsdate command to format the date and time as an XML Schema datatype. Other features may be added later. The xsdate output is used to provide timing information for the new XML build log format.

The first argument to astutil is the internal command to execute. The currently supported commands are date and xsdate. Usage is as follows.

$ astutil --help
Usage: astutil --help|command [opts]
Interface to miscellaneous libast routines.
  --help                display this help text.
Valid commands:
  xsdate [-s|-S] [-z|-Z] [dateTime|date|time]
    dateTime            current date/time formatted as an XML Schema
                        dateTime (default).
    date                current date formatted as an XML Schema date.
    time                current time formatted as an XML Schema time.
    -z                  include time zone.
    -Z                  remove time zone.
    -s                  include fractions of a second.
    -S                  remove fractions of a second.
  date [+FORMAT]        current time in specified format.

The xsdate command outputs the current date and time formatted as an XML Schema datatype. To run the xsdate command run astutil xsdate. By default the date followed by the time (with fractional seconds) and time zone offset are output.

$ astutil xsdate
2009-06-18T15:08:05.243420056-04:00

To print only the date or time specify the date or time option.

$ astutil xsdate date
2009-06-18

$ astutil xsdate time
15:14:20

The -z option adds the time zone if it isn't output by default and -Z (upper case) removes the time zone. Likewise, use -s to add fractional seconds and and -S (upper case) to remove fractional seconds.

$ astutil xsdate     
2009-06-18T15:21:49.296742333-04:00

$ astutil xsdate -S
2009-06-18T15:21:53-04:00

$ astutil xsdate -S -Z
2009-06-18T15:22:07

$ astutil xsdate time
15:22:41

$ astutil xsdate -s time
15:22:45.455652203

$ astutil xsdate -s -z time
15:22:52.928145097-04:00

The date command acts as a generic date command and accepts optional date format fields.

$ astutil date                   
Thu Jun 18 15:36:33 EDT 2009

$ astutil date "+%Y-%m-%d %T %:z"
2009-06-18 15:36:36 -04:00

For more details on usage see the astutil(1) man page.


3. Bug Fixes and Enhancements

3.1 Baserules

  1. 050040 - shared library updated with wrong filename
    Under certain conditions when using :LIBRARY: to make a shared library and linking an application with the library in the same makefile, the library would be generated with the wrong filename when it was updated in a new viewpath node. This has been fixed.
  2. 050042 - should not link with shared lib .so filename
    Under certain conditions when using :LIBRARY: to make a shared library and linking an application with the library in the same makefile, the application would be linked using the library's filename rather than the -lname flag when the application was updated in a new viewpath node. This has been fixed.
  3. 050060 - double colon target ignored with only lib prereqs
    Double colon targets with only library prerequisites were not being built. Such targets are no longer ignored and are built as expected. An example of such a target is --  mycmd :: libabc.a
  4. 070042 - tries to link with non-existent shared library
    Three errors were fixed when an executable target name matches a prerequisite library name (eg. abc :: main.c -labc). In case the prerequisite library doesn't exist nmake now gives an error message instead of trying to link the executable with the non-existent library. In case the prerequisite library does exist and is a shared library nmake now correctly specifies the -lname flag to link with the library instead of specifying the library's file name. In case the library is an archive library made in the same makefile nmake now specifies the correct link command line to link with the library.
  5. 090004 - dot has changed message in explain output
    The unnecessary ". has changed" message produced by the -e explain option has been eliminated.

3.2 cpp

  1. No changes to cpp in this release.

3.3 Engine

  1. 060061 - core dump from empty scanrule
    A .SCAN assertion with no rule defined no longer causes nmake to core dump.
  2. 070009 - Makeargs core dump
    Fixed a problem that caused a corrupt rule name which sometimes lead to a core dump when a Makeargs file was used.
  3. 080013 - aix showing trap shell trace
    Unexpected shell trace output on AIX platforms when building archive libraries has been eliminated.
  4. 080017 - provide W3C conformant time zone designator
    The time format output from the :F=%(format)T edit operator now supports a time zone format in accordance with conventions defined by the World Wide Web Consortium.
    For details see - New Features and Enhancements: W3C conformant time zone designator.
    See also - Changes Impacting nmake 11 Makefiles: Date format field %z.
  5. 080019 - expandinclude core dump
    nmake no longer core dumps when "set expandinclude" is used in a makefile to enable the expandinclude option. When enabling this option in a makefile instead of using the -p command line option a warning is included in the generated .mi file to remind the user that statements prior to the option are omitted and the command line option should be used to capture everything. Both expandinclude and noexpandinclude are supported and may be set multiple times to turn the option on and off for capturing sections of the makefile.
  6. 090000 - :cc: causes canceled action when prereq is removed
    When a target had at least one prerequisite built by :cc: and a non-:cc: prerequisite was removed the target wasn't updated. An update of the target is now properly triggered.
  7. 090001 - extend buildlog XML format
    The engine was enhanced to facilitate the output of XML tags before and after all other messages output by the engine.

3.4 Operators

  1. 040011 - :cc: only supports .c files
    The :cc: operator now supports files with the following file name suffixes: .c, .C, .cc, .cpp, .cxx. These are the same C/C++ suffixes that are supported by the default metarules to generate .o files. Files with these suffixes specified on the RHS of :cc: are compiled with the $(cc) (lower case) compiler. Previously only files with a .c suffix were compiled with $(cc) and other files were ignored by :cc:.
  2. 040055 - :COPY: does not make destination directory
    The :COPY: operator now makes the destination directory if it does not already exist. Previously a non-existent directory would cause an error. The :LINK: operator was also enhanced to always make the destination directory if it doesn't exist. Previously :LINK: only made the directory if the .ALL atom was made which could lead to situations where the directory was not created. The new behavior improves consistency with both operators.
  3. 060048 - request more standard shared library naming
    Several enhancements were made to the :LIBRARY: operator for making a shared library without disabling the library version number.
    For details see - New Features and Enhancements: Enhancements to versioned shared libraries.
    See also - Changes Impacting nmake 10 Makefiles: Versioned shared libraries, Shared library .no file
  4. 070035 - JAVAPACKAGEROOT errors
    The cause of several false package root errors from Javadeps has been fixed. The :JAVA: operator now normalizes paths to avoid these errors when possible.
  5. 080020 - :JAR: clobber removes prereq class file
    After removing a prerequisite file (or a RHS pattern) from a :JAR: assertion in a makefile and updating the jar, a subsequent clobber common action would remove any files corresponding to the deleted prerequisite from the file system. This has been fixed, :JAR: should not remove files it didn't generate.

3.5 Probe

  1. 080016 - probe_hints missing CC.MEMBERS.UNDEF and CC.STDINCLUDE.OMIT
    The following probe variables were not available in the probe_hints script. They are now being properly exported and imported for probe_hints support: CC.MEMBERS.UNDEF, CC.PRELINK, CC.STDINCLUDE.OMIT

3.6 Miscellaneous

  1. 030045 - allow users to set $makelog without triggering taglines
    Setting the makelog variable and running nmake directly (without using nmakelog) is now supported and does not result in the extra output from taglines.
  2. 080014 - refine structured build log implementation
    Support for structured build logs has been refined and extended making the feature more robust.
    For details see - New Features and Enhancements: Refined structured build logs.
  3. 080022 - man pages should reference product manuals
    The man pages now reference and provide links to the user manuals.
  4. 080025 - expandinclude -p option missing from man page
    The following options were missing from the nmake man page and have been added: -B / believe, -p / expandinclude, -N / never, targetcontext. A description of the viewpath feature has also been added.
  5. 080028 - command to generate W3C conformant date strings
    A new command called astutil is included in the nmake package.
    For details see - New Features and Enhancements: New command - astutil.
  6. 090001 - extend buildlog XML format
    Added a new container element delimiting the output of each Makefile.
  7. 090007 - build log transformation
    A new experimental command called buildlog2html is included to generate HTML build logs from an XML structured build log.
    For details see - New Features and Enhancements: New command - buildlog2html.

3.7 Variables

  1. Change to variable makelog - see 030045.
  2. Change to variable preserve - see 060048.
  3. Change to variable CC.MEMBERS.UNDEF - see 080016.
  4. Change to variable CC.PRELINK - see 080016.
  5. Change to variable CC.STDINCLUDE.OMIT - see 080016.

3.8 Options

  1. Change to option -p / expandinclude - see 080019.

4. Changes Impacting nmake 10 Makefiles

The changes in nmake 11 are largely backward compatible with nmake 10. Every effort has been made to insure code changes do not unexpectedly change the documented behavior of nmake features.

The following may impact builds and require changes.

4.1 Versioned shared libraries

Description
The default version for a shared library made with the :LIBRARY: operator has been changed from "1.0" to "1" to be more consistent with system libraries. This generates a shared library name such as libname.so.1 instead of libname.so.1.0 (or .sl on HP-UX).
Who is affected
Projects using :LIBRARY: to build versioned shared libraries and expecting version 1.0.
What to do
If a 1.0 version string is still required it can be manually specified on the LHS of :LIBRARY: after the library name.
abc 1.0 :LIBRARY: a.c b.c c.c

4.2 Shared library .no file

Description
In previous release a versioned shared library created by the :LIBRARY: operator was installed with a .no file name with symbolic links to it for the plain and versioned .so/.sl files. For example:
-rwxr-xr-x 1 richb richb 4568 Jun 18 17:50 libabc.no.1.0
lrwxrwxrwx 1 richb richb   13 Jun 18 17:50 libabc.so -> libabc.so.1.0
lrwxrwxrwx 1 richb richb   13 Jun 18 17:50 libabc.so.1.0 -> libabc.no.1.0
The .no file is no longer used. The versioned .so/.sl file is installed with one symbolic link for the unversioned file name. For example:
lrwxrwxrwx 1 richb richb   11 Jun 18 17:56 libabc.so -> libabc.so.1
-rwxr-xr-x 1 richb richb 4564 Jun 18 17:56 libabc.so.1
Who is affected
Projects that have scripts or makefiles looking specifically for the .no file may have a problem.
What to do
Update the tools to disregard the .no file since it is no longer needed.

4.3 Date format field %z

Description
In previous releases the %z date format field used by the :F=%(format)T edit operator was documented as the time zone type. However the field did not work and always expanded null. The %z field has been changed to the time zone offset and the time zone type is now %q and has been fixed.
Who is affected
Projects using the %z date format field with the :F=%(format)T edit operator may get unexpected output.
What to do
To preserve the previous behavior remove the %z field since it expanded null. If the time zone type is really needed then change %z to %q. If you would like the time zone offset then leave %z.

4.4 Structured build logs / xmakelog

Description
The experimental structured build log support introduced in nmake 10 has had some changes. The xmakelog command is still used to generate XML output, however it now defaults to output appropriate for the new buildlog2html command in which case the makelog_format variable need not be set. Some changes have also been made to the XML output. In order to conserve disk space the <targ> tag and its attributes have been shortened to two characters since they will appear many times in a build. The following table shows these changes.
old name new name description
targ tg target XML tag
name nm target name attribute
host hn host machine name attribute
stime st action start time attribute
(new) et action end time attribute
(new) ec action exit code attribute
dir (deleted) directory name attribute
Who is affected
Projects generating structured build logs.
What to do
If you are setting the makelog_format variable to define the target elements change the value to use "tg" instead of "targ". For example, makelog_format=xml:tg/@. You can also unset makelog_format to use the default which now includes all the supported elements in the table above (same as "xml:tg/@"). If you have custom tools processing the XML then change them to use the new tag and attribute names as needed.


5. Known Problems and Remarks

5.1 Known Problems

The following is a list of known problems:

  1. 980009 - problem in link=*
    The problem about using linking (specified using the link variable) to install targets in any adjacent nodes in viewpathing still exists. You may still use linking to install, but only in alternate nodes -- installation using copying must alternate with installation using linking. The problem occurs with both symbolic and hard linking.
  2. 030042 - :cc: generated header rebuild
    If a C file is built by using :cc: and this C file shares a generated header file with a C++ file specified in the same makefile, the generated header file may get regenerated during the build causing the source files to keep recompiling. This can be avoided by building the C++ files before the C files, or by generating the header file in a separate makefile at an earlier time during the build.
  3. 990099 - metarules not searched when CC.REPOSITORY is set
    The metarules may not be searched under the following conditions: there is a user defined metarule for a non-compiled file, variable CC.REPOSITORY is defined and CC.DIALECT contains either PTRIMPLICIT or PTRMKDIR, and :: is used to trigger the metarule. Note that CC.REPOSITORY and CC.DIALECT are defined as such in the probe file for some C++ compilers. The work-around is to add .IMPLICIT to the :: assertion, or to not use the :: assertion (such as add the target to :ALL: instead.)
  4. 020018 - adding shared lib building in same makefile won't re-link
    An executable target will not be re-linked when a shared library that is built in the same makefile is added as a prerequisite to the executable. The work-around is to set force_shared=1.
  5. 020051 - first metarule is used by mistake
    When multiple metarules generate the same source file target, such as %.c, the first metarule defined matching an existing file will be triggered instead of the metarule matching the explicit file prerequisite.
  6. 070010 - fails to bind large files
    nmake does not have large file support and cannot reference files over 2 GiB in size.

5.2 Remarks

  1. :LINK: does not handle archive files that are generated by :: or :LIBRARY: assertion operators. Users should avoid use of :LINK: on archive files.
  2. When the cpp -I-S flag is on, -D-M is disabled. Users should not use these two flags together.
  3. Users should avoid including the same header file with both <...>-style and "..."-style #include statements in source files managed by a single makefile. nmake will assign .STD.INCLUDE and .LCL.INCLUDE attributes to the header file, and this can result in incorrect -I lists in the compiler command lines generated.
  4. When the SHELL environment variable points to ksh93 you may get the following warnings when building your shell scripts: warning: line 1: `...` obsolete, use $(...). The warning is generated running your script with ksh -n. To eliminate the warning either update your shell scripts as indicated by the message or set SHELL to another version of ksh that does not generate the warning, such as ksh88.

Last Update: Friday,12-Aug-2016 12:24:24 EDT