Nokia nmake Product Builder
Customer Support Newsletter

http://www.bell-labs.com/project/nmake/newsletters/

Issue No. 30 - November 2008
Contacts
Articles
  1. Structured Build Logs
  2. Probe VERSION_OS
  3. Viewpathing
  4. Suggestions

Structured Build Logs

nmake 10 introduced an experimental new feature allowing generation of structured build logs. The feature captures per-target information in a structured format and also captures the hierarchical structure of recursive builds. Information recorded per target includes exit code, start time, end time, execution host, and output of triggered action. The XML based log format allows build log querying and processing using widely available tools and libraries.

Applications of this feature include enhanced presentation, navigation, and analysis of build information. For example, could generate multiple interlinked web-based views of a build at different levels of detail, with links from high level summaries to more detailed views. Elements could be color coded based upon target attributes such as exit code. Analysis could identify bottlenecks and opportunities for build optimization. Experimentation with some of these possibilities is in progress; initial results look promising.

See Generating Structured Build Logs Using Alcatel-Lucent nmake for more information.

index

Probe VERSION_OS

In issue 15 we discussed the use of the VERSION_ENVIRONMENT variable to use different probe files for different OS versions. This is useful when the nmake installation is shared across multiple machines running different versions of the operating system. nmake 10 automatically keeps different probe files for different OS versions. VERSION_ENVIRONMENT is no longer needed for this and can be unset when using nmake 10.

The following demonstrates the new feature. The new probe variable CC.VERSION_OS contains the OS version information. The variable is used as a probe attribute to determine the probe filename hash. The probe -a flag lists the probe attributes, -k shows the probe configuration file name, and -l lists the contents of the configuration file. As you can see this is on a Solaris 10 machine.

$ type nmake
nmake is /tools/nmake/sparc5/10/bin/nmake

$ probe -a C make cc 
PREROOT='' UNIVERSE='ucb' VERSION_OS='SunOS5.10' ATTRIBUTES='PREROOT UNIVERSE VERSION_OS'

$ probe -k C make cc
/tools/nmake/sparc5/10/lib/probe/C/make/B5633F95obincc

$ probe -l C make cc | grep ^CC.VERSION_OS
CC.VERSION_OS = SunOS5.10

Below we show another Solaris machine using the same nmake installation. Notice the probe file name is different since VERSION_OS is different.

$ type nmake
nmake is /tools/nmake/sparc5/10/bin/nmake

$ probe -a C make cc
PREROOT='' UNIVERSE='ucb' VERSION_OS='SunOS5.6' ATTRIBUTES='PREROOT UNIVERSE VERSION_OS'

$ probe -k C make cc
/tools/nmake/sparc5/10/lib/probe/C/make/4D201C75obincc

$ probe -l C make cc | grep ^CC.VERSION_OS
CC.VERSION_OS = SunOS5.6

To override the default value set VERSION_OS in the environment.

$ export VERSION_OS="xyz"                 

$ probe -a C make cc            
PREROOT='' UNIVERSE='ucb' VERSION_OS='xyz' ATTRIBUTES='PREROOT UNIVERSE VERSION_OS'

$ probe -k C make cc            
/tools/nmake/sparc5/10/lib/probe/C/make/A9185FA6obincc

This new feature should be useful to many projects since it is common to share the nmake installation among many machines. Using different probe files for each OS version will help reduce probe related issues.

index

Viewpathing

One of the major features of nmake is the viewpath which defines a virtual directory structure allowing multiple, parallel trees to be viewed as a single tree. Each parallel tree is called a "node" of the viewpath. In typical usage a developer will have only the files they are changing in their local node with all other files picked up from other nodes. This allows developers to share a common base of code and built objects while minimizing their private areas. Another use of the viewpath is to separate source nodes and build nodes. By keeping the source in a separate node it can be referenced on multiple build machines so multi-platform builds all use the exact same code. The viewpath is also commonly used for patch or incremental builds, where updated code is placed in a new node and a new build performed by viewpathing through previous build and source nodes. New builds can be performed while keeping the previous revisions intact.

Most all nmake users are aware of the viewpath but how it works remains a mystery which can lead to build errors or makefiles that don't work with the viewpath properly. In this article we'll attempt to demystify the viewpath and give a better understanding of how to use it.

The Basics

Most of you probably know the basics already but lets go over it real quick for completeness. A viewpath is defined by setting the VPATH variable in the environment to a colon (:) separated list of directories. Each directory, or node, is the root of a parallel tree. nmake looks for files in the viewpath by searching the corresponding directory in each node for a named file. The first file of that name found is used, regardless of the time-stamp. The nodes are searched in the order they are listed in the VPATH variable from left to right, just like the PATH.

A Simple Makefile

Here is a simple makefile that builds a single executable. All the source code is down the viewpath with a .c and .h file in src/ex1/ and an extra .h file in src/include/.

$ pwd
/home/richb/node1/src/ex1

$ echo $VPATH
/home/richb/node1:/home/richb/node2

$ cat Makefile
CC = gcc
.SOURCE.h : $(VROOT)/src/include
hello :: hello.c

$ nmake
+ gcc -O -I/home/richb/node2/src/ex1 -I- -I/home/richb/node2/src/include -c /hom
e/richb/node2/src/ex1/hello.c
+ gcc -O -o hello hello.o

Notice when using the standard nmake conventions nothing special is needed in the makefile to use the viewpath. nmake automatically finds the prerequisites, both hello.c and its include files, in node2 and specifies the necessary paths on the compile line.

The .SOURCE.h special atom tells nmake where to look for include files in addition to the current directory. Relative directories in the .SOURCE[.pattern] atoms are viewpathed automatically. nmake sets $(VROOT) to the relative path back to the top of the viewpath node, in this case ../.. (to get from /home/richb/node1/src/ex1 to /home/richb/node1.) VROOT should never be defined in the makefiles or environment, nmake sets it automatically.

Custom Rules

Most viewpathing problems come from custom rules. Here is a makefile with a simple custom rule that greps some conf files to generate us.conf. Again the files are in node2 while we are building in node1.

$ pwd
/home/richb/node1/src/ex2

$ cat Makefile  
us.conf : global.conf en.conf
        grep "^us:" global.conf en.conf > us.conf

$ nmake
+ grep ^us: global.conf en.conf
+ 1> us.conf
grep: can't open global.conf
grep: can't open en.conf
make: *** exit code 2 making us.conf

The build failed! A common mistake is to think any files appearing in the shell action block will be viewpathed automatically. In fact this is not how viewpathing works. The shell action block is like a shell script, nmake only expands the nmake variables and then sends the action to the shell to be executed. Any files or paths hard coded in the action block are sent to the shell just as they appear and the shell will try to reference them as stated.

The key to using the viewpath is to use the automatic variables, particularly $(*) which expands all the file prerequisites where found in the viewpath. Here is the above makefile written correctly for nmake:

$ cat Makefile
us.conf : global.conf en.conf
        grep "^us:" $(*) > $(<)

$ nmake
+ grep ^us: /home/richb/node2/src/ex2/global.conf /home/richb/node2/src/ex2/en.conf
+ 1> us.conf

Notice $(<) is also being used now for the target. See Essential Automatic Variables in issue 27 for more information on the automatic variables.

Lets say you need to run a tool that is generated during the build. Since the tool may exist anywhere in the viewpath its command name cannot simply be specified in the shell action. Here is another version of the above rule that runs our custom tool, ppconf, from the viewpath. This shows how different prerequisites can be referenced in the shell action to create the right command line.

$ cat Makefile
us.conf : ppconf global.conf en.conf
        $(*:O=1) -o $(<) $(*:N=*.conf)

$ nmake
+ /home/richb/node2/src/ex2/ppconf -o us.conf /home/richb/node2/src/ex2/global.conf /home/richb/node2/src/ex2/en.conf

The :O=1 edit operator expands just the first token of the variable, which is ppconf. The :N=*.conf edit operator expands the tokens matching the shell pattern *.conf. Another option would be to use $(*:O>1) to expand the prerequisites past the first one.

Conclusion

The viewpath is a powerful feature. With some basic understanding it can be utilized well in your makefiles. We hope this article helps shed some light on this feature.

index

Suggestions

We're open to suggestions for future articles. If there is anything you would like to see in the newsletter please send us a note to nmake@alcatel-lucent.com. Thanks for your support!

index

<<home / newsletters