This document gives information on how to build and install DRAMA. It is still under developlement. If you have any problems, please feel free to contact me, Tony Farrell, at tjf@aao.gov.au  

Installation of DRAMA

Sources are available on our FTP site

The DRAMA directory structure contains the following files.
drama_source
The source tree
drama_make
The Unix/VxWorks build script
doc
The documentation files in tex format.
man
The man pages.
html
The html pages.

Normally you unpack these into the place you want drama to reside Or just soft link them.

Building Unix/VxWorks DRAMA

Run the script drama_make. By default, it builds DRAMA for the host machinešs architecture using default settings.

From DRAMA Version 1.0, these default settings are no longer AAO specific, but assume a standard configuration for the host machine. This should result in a clean build first time. Let me know if they donšt

There are various options to the drama_make command.

-g
Compiles with -g option
-O
Compiles with -O option
-t target
Specifies a target architecture. This is normally vw68k for VxWorks 68k or vwmips for VxWorks mips or vwppc for VxWorks PPC.
-co
Clean the source directories of built files.
-start sub-system
system Start at the specified system. (dits, sds etc.)
-forcegcc
Force the use of GCC instead of the default compiler. Normally, the default compiler is cc, although on some systems it may be gcc. Also see -nocompcheck
-nocompcheck
Don't check for compilers. By default, unless this flag or -forcegcc is supplied, the script checks for the existance of either an ANSI cc or gcc. If cc does not exist but gcc does, it then uses gcc. This flag stops this check forcing the use of the defautl compliler even if cc does not exist. You may want this option if the default compiler is named something other then cc.
-nocheck
By default, a check is made before overwitting existing DRAMA installations. This is done for each sub-system on a version based. So for example if you already have version r3_12 of dits and you try to build and release r3_12 of dits again - the check fails. When this happens, you are prompted for confirmation of wanting to overwrite the version. This often happens when rebuilding DRAMA. You can disable this check using this flag.
-noquiet
By default, DRAMA provides a simple indication of running the compiler etc. The compiler options etc. are not shown. Specify this option to show all details.
-envscript
This is normally combined with the -start option to say that we are generating a script which we can use to set up a shell environment so we can build the sub-system specified to -start. The output of this program is the script and no attempt is made to build the specified sub-system. The output is appropiate for the zsh and bash scripts (except that the dmkmf alias needs to be set up correctly. The output can be easily automatically converted to a csh style script using the awk script cvt_to_csh.awk. Having run such a script, you can then cd to the source directory concern and try building. This allows you to debug problems in one sub-system without having to rebuild all of DRAMA on each attempt.
-versions filename
This option causes the specified file to be written containing a version specification for each sub-system. This file can be placed in ~drama and used with the -v option to dramastart. Note that when this option is used, the various sub-system versions are not enabled - that is, they will not be selected automaticaly when you use the dramastart command.

Building for Unix and VxWorks

The normal sequence when building DRAMA for a UNIX machine and a VxWorks machine would be
./drama_make 
./drama_make -co
./drama_make -t vw68k
If you don't want VxWorks, you can drop the two later lines. Note that for the VxWorks Tornado product (VxWorks 5.3 and up), you must have previously sourced the appropriate VxWorks start up script. Otherwise, the assembler will not be correctly found and you will get strange errors during the build.

Generally follow this by ./drama_make -co to clean objects out of the source directories.

Using the -start option

If the build fails and you need to modify the config sub-system or local configuration file (described later) you probably need to do a clean (drama_make -co) followed by a complete rebuild.

Otherwise, you can restart the build at the system which failed, by specifying it to the -start option.

For example, if the build failed in the dits sub-system, you can restart it at that point using

drama_make -start dits

The DRAMA configuration (UNIX)

You may want to change the configuration, for example ­ Tcl/Tk is not part of a standard release. You may want its use by DRAMA enabled (so that Dtcl/Dtk is built). In addition Two possibilities
  1. Create a local configuration file. This avoids changing the DRAMA source and allow for machine specific options.
  2. Modify the configuration options. Was required before DRAMA 1.0 and may still be required in some cases as the system is not perfect yet.

Options to consider changing

The following options are those most commonly changed
HasGcc
Do we use gcc instead of cc. Defaults to YES on sunos machines and NO otherwise. This one must be modified by changing the DRAMA config source directly - i.e. you can not use the location configuration file. In more recent versions of DRAMA, from version 1.2, this is no longer required. Just use the -forcegcc option to drama_make.
HasFortran
Defaults to NO. Only really relevant to the DRAMA build if you also have starlink.
HasStarlink
Defaults to NO. If YES, then bits of DRAMA which use starlink are built
HasMotif
Does the machine have Motif. If YES, then xditscmd is built. Defaults to YES on Solaris and OSF.
HasTcl
Does the machine have Tcl. Defaults to NO.
HasTk
Does the machine have Tk. Defaults to NO.
FortranLibs
Define to the link options which enable a program built with cc/gcc to link against the fortran libraries. Only used if HasFortran is YES.
StarlinkDir
Location of starlink. Always defaults to /star
MotifLib, MotifIncl
Link options required to link Motif programs. For SunOS and Solaris prior to V2.4, you will need to set this appropirately
TclLib, TkLib, TclTkIncl
Link options to link against Tcl and Tk. TkLib should include TclLib. Include file locations.
HasJava
Does the machine have Java. If YES, then djava is built. By default, Mac Os X is presumed to have Jave (but this must be the developement SDK) whilst other machines are presumed not to. If you have Java, then you need the various tools to be in your path.

Local configuration files

Local configuration files were introduced at DRAMA V1.0. They allow you to implement changes to the standard DRAMA configuration which are maintained across DRAMA releases, without requiring DRAMA source changes.

They also allow you to make site/machine specific configuration changes i.e. Tcl/Tk may be in different places on different machines.

The file is local/drama_local.cf If it exists, you are told it is being used when you run dmkmf.

	Found local configuration file
Just create this file if needed. If it does not exist, DRAMA uses a standard configuration.

For example, the following is the contents of a drama_local.cf file that enables Tcl/Tk and defines its location. #undef HasTcl #undef HasTk #define HasTcl YES #define HasTk YES #undef TclLib #undef TkLib #define TclLib -L/usr/local/lib -ltcl7.4 #define TkLib -L/usr/local/lib -ltk4.0 -ltcl7.4 -L/usr/openwin/lib -lX11 -lm

You must undefine variables before defining the new value. The Language of this file is basically macros in the C pre-processor.

Host Specific Options

DRAMA for multiple machines/architectures can reside in the one directory structure. But some items used by DRAMA may be in different locations on different machines. In addition, not all machines will be upgraded to the same standard. e.g. Only from version 2.4 did solaris have the Motif libraries in a standard location.

We need some way of modifying our local configuration file based on details of the machine we are running on.

This is done with the location options, normally a shell script, kept in the file local/drama_local.opts. If it exists and has execute mode enabled, the local options file is executed as part of the dmkmf command.

Any text output by this program to stdout is passed to the imake program as extra options. The normal useage is for it to output macro definitions which can be used by drama_local.cf.

For example, at AAO, we have things in different places at the telescope from Epping. We also have Solaris 2.3 machines.

#!/bin/sh NODE=`uname -n` case "$NODE" in aaossz) echo -DAAO_EPPING -DAAOSSZ -DSOLARIS23 ;; aatssz) echo -DAAO_COONA -DSOLARIS23 ;; aao*) echo -DAAO_EPPING ;; aat*) echo -DAAO_COONA ;; *) ;; esac This allows the AAO drama_local.cf file to locate the 2dF VxWorks drivers directories and Solaris Motif libraries using #ifdef AAO_EPPING # define AAO2dFDir /instsoft/2dF #else # define AAO2dFDir /home/aatssb/2dF #endif #ifdef SOLARIS23 #undef MotifLib #undef MotifIncl #define MotifLib -L/opt/SUNWmotif/lib ... #define MotifIncl -I/opt/SUNWmotif/include ... #endif Ensure you do chmod a+x local/drama_local.opts The expected message from dmkmf Found local options file

VxWorks Configuration

Organisations keep VxWorks different places depending on there own requirements. Therefore, thre is no default location DRAMA can assume.

If you wish to build the VxWorks version, you must use drama_local.cf to define its location.

Two macros will be defined to help you

VxWorksTarget
Indicates the target is VxWorks
VxWorks_n_n_arch_Target
n_n indicates the version, e.g. 5_1, 5_2. arch Indicates a specific target (68k, mips). For example
VxWorks_5_2_68k_Target
You will need to define the following macros
Xdev
Used to define the Make macro XDEV, which locates the development tools
Vw
Used to define the macro VW, the VxWorks release directory
GccPrefix
Location of the GCC cross compiler for this release
DefaultCCOptions
C compiler options Options
VwVer
VxWorks version.
In addition, if using the Wind River Tornado product, you may need to redefine the StdIncludes macro in a predefined way. If this is not right, the compiler will not be able to find VxWorks include file and the compilation of DRAMA will fail in the "drama" library.

One additional item to set is the type of make you are running. If using GNU make, set marco HasGnuMake. If running a Solaris or a similar make, set HasSunStyleMake. If you fail to set one of these, the the build will fail in the DITS sub-system. The feature this macro selects is how we set a Macro macro to the value of a shell command. The GNU make approach is
MACRO = (shell cmd)
whist the sun approach is
MACRO:sh = cmd
Please let me know if you find other styles.

For example #define OSName VxWorks 5.2/2.2.3.1 #define OSMajorVersion 5 #define OSMinorVersion 2 /* * Make style is Gnu */ #define HasGnuMake /* * Base directory of Vxworks, just used in the following lines * * In this case, /opt/VxWorks is the base of our VxWorks tree. */ #define VxWorksBase /opt/VxWorks /* * The Xdev macro is used to define the XDEV make macro, which locates * the developement tools. * * In this case, we concat it with VxWorksBase above, such that the * result will be * /opt/VxWorks/gnu/solaris.68k * * But you could also use it to locate say a newer version of gcc then * is available in your VxWorks release. */ #define Xdev Concat(VxWorksBase,/gnu/solaris.68k) /* * Vw is used to define the macro VW, the VxWorks release directory * * A compile concatation with VxWorksBase above to get * /opt/VxWorks/vw */ #define Vw Concat(VxWorksBase,/vw) /* * Find the location of GCC for this release - make macro GCC_PREFIX * * The VWLIBPATH make macro is defined by vw68k.cf to be $(XDEV)/lib so * we can use this to locate gcc-lib. * * In this example, the result after translation by make will be * /opt/VxWorks/gnu/solaris.68k/lib/gcc-lib */ #define GccPrefix -B$(VWLIBPATH)/gcc-lib/ /* * Default C compiler options. Probably partly a matter of choice or * instruction from your VxWorks or compiler supplier, except * that you should have GccMOpt() listed. * * These will be placed in the CCOPTIONS make macro. * * GccMOpt has already been correctly for target architecture defined in * vw68k.cf */ #define DefaultCCOptions -pipe -fvolatile -fstrength-reduce GccMOpt() -fno-builtin /* * Version number. Used to define the VWVER macro althrough now really used * anywhere by DRAMA itself. */ #define VwVer 5_2 /* Required For Wind River Tornado product only */ #undef StdIncludes #define StdIncludes -I$(VW)/host/include /* /* * One of these must be selected for for the definition of MacroToShellCmd * in drama_source/config/Project.def. If you have GNU Make, use the * later. If you are using a Sun style make, use the former. The * significant point is how make allows you to put the output of a * shell command into the value of a macro. In GNU style make * the following is used * * name = $(shell cmd) * * For Sun style make, this approach is used * * name:sh = cmd * */ /*#define HasSunStyleMake*/ #define HasGnuMake

VMS

I do not as yet have imake for VMS. As a result, you need a specific VMS release created by me. You must specify the following I can then generate the required save set.

Building VMS

Local configuration files are not supported due to lack of imake. The Extracted save set contains
drama_source.dir
The source tree
drama_make.com
The build procedure
To build, just to @drama_make After building, you will want define jobs/system wide local name dramadisk as a concealed logical name and Define dramastart symbol in login (system wide?) dramastart ==ŗ@dramadisk:[000000]dramastart dramastart˛

VMS Build options

The options to @drama_make.com are similar to the Unix script.
-start system
Restart a partial build at the specific sub-system
-co
To clean out build directories
-com
Needed if you donšt have MMS. Will rebuild everthing regardless of if it is already up to date.
Define your CC and LINK symbols as appropriate for debugging/optimization. But beware, some built programs will be executed in later parts of the build, so in general, don't have LINK defined with /DEBUG

Build Problems.

This section looks at various issues you may find when building DRAMA.
Include file incompatibilities.
In an attempt to ensure portability and code quality, we build with the highest reasonable warning modes available. Because system include files are not always up to scratch and compiler versions impact such checks, this sometimes triggers warnings and errors.
Libraries not in the correct locations.
You may need override library locations in the local configuration file.
Compiler specific problems/bugs
Should not be as common now as I am trying to build on all support architectures before each release. But I don't have all versions of all compilers and operating systems so there may still be some issues.

System configuration

Various UNIX facilities used by IMP are either optional or static in size. IMP has tried to keep to those facilities which are most common and on a standard Unix System Five machine, DRAMA will run without kernel changes, but you may still need to modify your kernel to adjust the allocations.

On a typical BSD system, you may have to enable certain SysV options. This is required under SunOS, but not OSF/1 for example.

These techniques have been tested on SunOs 4.1.x and Solaris 2.x systems but probably apply to most UNIX system with mild differences. Solaris and Linux use System V semaphores, whilst MacOsX uses Posix sempahores.

System V semaphores

On some UNIX systems, IMP uses system V semaphores for inter-process communications and to protect the global IMP shared memory area (the area were IMP keeps track of all registered tasks). (POSIX semaphores are used on Mac Os X and can be used on other architecutres if desired and possible by a changing one file - please mail us for details if ou want to do this).

System V semaphores are a statically allocated Kernel resource. As a result, it is possible to run out of semaphores and you may have to reconfigure your system Kernel to increase the number of semaphores. In addition, some systems, which are based on BSD Unix (such as SunOS 4.x systems) do not support semaphores by default.

System V semaphores are allocated in id groups with static restrictions on the number of groups in a system, the total number of semaphores in a system and the number of semaphores in a group.

IMP requires one semaphore for each task running on a machine and one extra semaphore per machine. Each semaphore is allocated in a unique id group. If you are running ten IMP tasks on a machine you must have at least eleven semaphores available plus extras for any other programs using semaphores. Note that these routines are system wide, not user specific. As a result, one user may prevent other users using IMP by allocating all the semaphores.

Enabling Semaphores

Semaphores are available by default on Linux and System V Unix machines (e.g. Solaris 2.x) and I believe on Digital Unix (OSF). On BSD machines which have system V compatibility (Such as SunOs 4.1) you may have to modify the kernel to enable semaphores. In SunOs 4.1.x you need to modify the file /sys/(arch)/conf/(host) Where arch is the result of the command "uname -m" and host is the host specific configuration file. Add the following line to this file options IPCSEMAPHORE # System V IPC semaphore facility Note that you may also need to add IPCSMESSAGE and IPCSHMEM. See below.

Details of other BSD based machines should be similar, but I don't have any available to check this. You now need to rebuild your kernel and reboot.

Running out of semaphores.

You may find that you run out of semaphores. Typically, this will result in a message like this when a task starts up. !! Failed to create message semaphore # 17430 !! semget returned errno value 28, No space left on device !! ImpMaster task failed to register properly In later versions of DRAMA, this message has expanded somewhat

This indicates that the system has run out of semaphores. There are a number of possible reasons for this -
A.
IMP programs have crashed without cleaning up their semaphores. This can happen sometimes as UNIX cannot ensure an exit handler is always run. To clean up after these programs use the IMP command "cleanup". By default, this program cleans up all IMP programs belonging to the user who invokes it. Options allow you to just list the programs, interactively kill them or to kill other user's programs (if running as root).

Please use the -h open to cleanup to get details on its various options.

B.
If a simple cleanup does not allow you run your programs, it is possible other users are using semaphores. Several facilities are available for working out where the semaphores are being used, the simplist probably being the "ipcs -s" command. This lists details of all the semaphores in the system, showing the user involved. "ipcs -s" produces something like this. IPC status from as of Fri Jan 17 12:03:37 1997 T ID KEY MODE OWNER GROUP Semaphores: s1245184 0x00001cf4 --ra-ra-ra- tjf users s1179649 0x000f2200 --ra-ra-ra- tjf users s1376258 0x00001cf5 --ra-ra-ra- tjf users s1048579 0x00001cf6 --ra-ra-ra- tjf users You can see the users involved quickly. The keys used by IMP tasks are based on the process id, so in the above example if the first semaphore is a IMP task semaphore the process id is 7412 (0x0001cf4 converted to decimal). IMP also uses one system wide semaphore used to protect the global list of all tasks. The id for this one is generated by the function (((gid << 8) | uid) << 8) where gid is the user's group id and uid is their user id. In the above example, gid is 15 and uid is 290 (both obtained from /etc/passwd). Running the function we get 0xf2200, the second semaphore being used.

The cleanup command's -al option can also be useful. It lists details of all running IMP programs on the machine.

Having worked out which users are using semaphores, check that they have not been left around by crashed programs (indicated if a process of the specified id does not exist). If so, the user concerned or superuser will need to do a cleanup to get rid of the semaphores.

C.
It is possible your application together with all the other applications being run on your machine use more semaphores then are available. You can use the tools mentioned above to help work this out. In addition, you need to work out what the limits are. Under Solaris 2.0 and similar systems, enter the command sysdef | grep "SEMMN[I|S]" (sysdef is often found in /usr/sbin.). This produces lines like these 30 semaphore identifiers (SEMMNI) 100 semaphores in system (SEMMNS) The minimum of these two numbers minus one is the maximum number of IMP tasks which can be run, assuming no other programs are using semaphores.

For SunOs 4.1 and similar systems, grep the file

/sys/(arch)/conf/(host) Where arch = result of the command "uname -," and host is the host specific configuration file. If you get lines like these, it is again the minimum of the two which is significant. options SEMMNI="30" # Change no of semaphore ids options SEMMNS="100" # Change no of semaphores in system It is possible the above command will produce nothing or only one or of the two lines. If so, try grep "SEMMN[I|S]" /sys/sys/sem.h | grep "define" Which produces something like #define SEMMNI 10 /* # of semaphore identifiers */ #define SEMMNS 60 /* # of semaphores in system */ #define SEMMSL SEMMNS /* max # of semaphores per id */ If either of the first two lines were missing in the first command, use the number from the corresponding line here.

On some systems (e.g. Linux), the -ls option to ipcs shows the current limit values. You can also use

/sbin/sysctl -a | grep kernel.sem In this case the values are SEMMSL, SEMMNS, SEMOPM, and SEMNI.
D
I have seen on one Solaris machine (running version 2.6) a case where it appeared the system lost track of the number of sempahores. In this case, the system should have had 100 semaphores, but, even with ipcs -s showing no semaphores allocated, I could not allocate more then a few semaphores. This fault occured a number of times on this machine and was resolved by rebooting the machine in each case. I don't as yet know the cause or any other way around the problem.

Increasing the number of semaphores.

There are three kernel parameters which may need to be modified or considered to increase the number of semaphores. These are
SEMMNI
The number of semaphore groups (id's) available
SEMMNS
The total number of semaphores available
SEMMSL
The maximum number of semaphores in a group.
IMP uses one semaphore for each task plus one extra to protect common data. Each semaphore is allocated in the its own group. As a result, the minimum of the values of SEMMNI and SEMMNS, minus one, determines the maximum number of IMP tasks which can be run on a machine, assuming no other applications are using semaphores. Note that other applications may use more then one semaphore in a group, up to the value of SEMMSL, so as a result you may need the value of SEMMNS to be considerable greater then SEMMNI. This is not the case for IMP.

How you modify these parameters depends on the particular version of Unix but I can give examples of common systems. In SunOs 4.1.x you need to modify the file

/sys/(arch)/conf/(host) Where arch = result of the command "uname -," and host is the host specific configuration file. Search for the lines like the following ones or add them if the don't already exist. options SEMMNI="10" # No. semaphore ids options SEMMNS="60" # No. Semaphores in system The values above are the default values in my SunOs system. You will probably want to increase SEMMNI to the number of IMP tasks plus one plus whatever other semaphores are required by other programs. Ensure SEMMNS is larger then SEMMNI (if only IMP programs are running, they can be the same). You now need to rebuild your kernel in the normal way and reboot. For Solaris 2.x machines add the following lines to /etc/system if they do not already exist, modify the values as required and then reboot. set semsys:seminfo_semmni=10 set semsys:seminfo_semmns=60 For Linux (Kernel 2.4 onwards) you can use the /etc/sysctl command to set the values. The following command /sbin/sysctl -w kernel.sem="250 32000 32 251" Sets SEMMSL=250 SEMMNS=32000 SEMOPM=32 SEMNI=251 Which can then be checked with the ipcs -sl command. Under linux the effect is immediate. To make the effect permanent you must modify /etc/sysctl.conf appropiately.

Message Queues

IMP uses System V message queues in some cases for interprocess communication. As per System V semaphores, message queues are a statically allocated Kernel resource. As a result, it is possible to run out of them and you may have to reconfigure your system Kernel to increase the number of message queues. In addition, some systems, which are based on BSD Unix (such as SunOS 4.x systems) do not support message queues by default.

Enabling Messages

System V message are available by default on System V Unix machines (e.g. Solaris 2.x) and I believe on Digital Unix (OSF). On BSD machines which have system V compatibility (Such as SunOs 4.1) you may have to modify the kernel to enable semaphores. In SunOs 4.1.x you need to modify the file /sys/(arch)/conf/(host) Where arch is the result of the command "uname -m" and host is the host specific configuration file. Add the following line to this file options IPCMESSAGE # System V IPC message facility Note that you may also need to add IPCSEMAPHORE (see above) and IPCSHMEM (see below). Details of other BSD based machines should be similar. You now need to rebuild your kernel and reboot.

Running out of message queues.

The default for the number of message queues is normally sufficient (about 50) and such a failure has never been reported in a IMP system.

Techniques for debugging this situation is the same as for semaphores. Use the command "ipcs -q" to find a list of all the queues on the machine. IMP uses as most one per task and sometimes less. IMP message queue keys are process id's of the task.

If you do run out of message queues, the system parameter you need to modify is MSGMNI. You can find out it's value and modify it such the same techniques show above for the semaphore parameters SEMMNI/SEMMNS.

Shared memory

Older versions needed this, in newer versions mapped files are used. But you can enable the use of it if you prefer. See the Imp manual for details.


Click here for the DRAMA home page and here for the AAO home page.

 For more information, contact tjf@aao.gov.au