| United States |
|
 |
 |
||||
|
|||||
A Cookbook Approach To Creating an OpenVMS Shareable Image
Please forward suggestions and corrections to the author:
Shareable Image
The primary purposes of shareable images are the reduction in memory when multiple copies of the code are active, to reduce the amount of time required to link images, to allow easy and selective replacement of libraries of code, to allow development of chunks of code within an entire system, and to ease the upgrade or addition of code to a system.
Shareable images (and the OpenVMS Alpha SYS$BASE_IMAGE image) are the only images supported by the LIB$FIND_IMAGE_SYMBOL routine -- this routine allows a process to perform a variant of `dynamic linking', this call allows an image to access code in a shareable image `on-the-fly'.Shareable images are created from ordinary objects and object libraries by following the steps outlined in this cookbook.
Executable image
See the comments on INSTALL, below.
User-written system service (UWSS)
Differences between OpenVMS VAX and OpenVMS Alpha are explicitly called out.
Symbols that should be externally visible typically include the symbolic names of the various entry points that you want callers of the shareable image to be able to see and use. These externally visible symbols do not necessarily have to be subroutines, they could also be constants or variables that are intended to be externally visible.
DIGITAL recommends that all externally-visible symbols should follow the OpenVMS symbol naming conventions as documented in the Guide to Creating OpenVMS Modular Procedures manual.
Also determine what external symbols (the addresses of data and/or subroutines) that the shareable image will itself reference.(This is code and/or data that the shareable itself is based on.)These are known as external references.
ALL external references must be resolved when an image -- be it a shareable or an executable image -- is linked.In the case of a link of a shareable image, all external symbols referenced by code in the shareable must be resolved when the shareable is linked.
Shareable images are sensitive to the external models used by the particular compiler(s) in use.This document assumes the VAX C external model, and will cover the use of OpenVMS VAX transfer vectors or OpenVMS Alpha SYMBOL_VECTOR linker options.
If using OpenVMS VAX transfer vectors or the OpenVMS Alpha SYMBOL_VECTOR linker option, /SHARE_GLOBAL need not be specified.
The /SHARE_GLOBAL qualifier is not required under VAX C.
Further information on transfer vectors and on SYMBOL_VECTOR linker options is included below.
Create a set of transfer vectors for the symbols (OpenVMS VAX):
The OpenVMS VAX shareable image transfer vectors are located at a known position within the shareable image -- typically at the front -- and the various offsets are used to dispatch the run-time calls into the actual code position(s) within the shareable image. Use of consistent offsets means callers do not need to relink the calling image(s) when the shareable image is modified.
An example transfer vector -- XFRVEC.MAR -- looks like this:
.title$$$XFRVEC facnam_SHR Transfer Vectors .ident/facnam_SHR V1.0-0/ ; create a program section to hold just the vectors ; keep all transfer vectors quadword aligned. .psect$$$XFRVEC,exe,shr,nowrt,rd,pic,quad ; For routines entered via a CALLS or CALLG instruction: .macroXFRVEC_CALL entry_point,entry_point_int .alignQUAD .transfer entry_point .External entry_point_int .mask entry_point_int JMP l^entry_point_int+2 entry_point:: ; allows .transfer to work. .endm ; define a macro to simplify creating a transfer vector ; to a routine located in the shareable image. .macroXFRVECentry_point .alignQUAD .transfer entry_point .mask entry_point JMP l^entry_point+2 .endm ; define a macro that places a longword constant directly ; into the transfer vector array. .macroXFRCONconstant .alignQUAD .long constant .endm ; and start defining the transfer vectors... XFRCON42 XFRVECentry_point_name1,entry__point_name1 XFRVECentry_point_name2,entry__point_name2 .End
Use the above template for building a set of transfer vectors. You will need to substitute the externally visible symbols -- those symbols that you determined should be visible outside the shareable image -- in place of the example entry point names (entry_point_name1 or entry_point_name2) used in the template.
You do not need to change the symbol names of any externally visible symbols.You can (and probably should) use the same name in the transfer vectors (the transfer vectors are the .TRANSFER-created stuff in the MACRO32 file) as you have used in the code or data in the object file(s).The symbol used in the transfer vector will be the one visible to the callers of the shareable image. The linker knows that a TRANSFER may duplicate another internal symbol.(The linker uses the value from the transfer directive when it builds the externally-visible shareable image symbol table -- shareable and executable images that link against the shareable image never realize there is a second (internal) symbol of the same name, since it is not included in the externally-visible shareable image symbol table.) (Note that this means all calls to an external entry point that are made from within a shareable image do not use the shareable image transfer vector, these calls directly call the internal entry point of the same name.)
Put in as many occurences of XFRVEC as you need.Always add new transfer vectors to the end of the list.Never reorder the list. Do not remove any existing vectors.(Jump to a dummy routine that returns a failure status, if you must.)(Reordering the list of vectors or removing individual vectors will tend to break existing applications that are linked against the shareable image.)
If you really must change the order or remove a vector -- you have decided you must make an incompatible change -- further information on the GSMATCH option is available.
Create a set of SYMBOL_VECTOR transfer vectors in the LINKER options file for the symbols (OpenVMS Alpha):
SYMBOL_VECTOR = ( entry_point_name1 = PROCEDURE,- entry_point_name2 = PROCEDURE )
The OpenVMS Alpha LINKER implements the transfer vectors using the LINKER options file and the image symbol table.(The design of the Alpha Alpha pipelining and branch prediction makes the OpenVMS VAX transfer vector scheme rather expensive.)The shareable image support is otherwise the same.Rephrased: a Macro (the language) file containing transfer vectors is not required on OpenVMS Alpha.It's all done in the LINKER options file...
It is possible that a large or complex symbol vector may exceed the capacity of the LINKER to process a single line in the options file.If this occurs, multiple SYMBOL_VECTOR declarations can be used.
Always add new SYMBOL_VECTOR entries to the end of the list.Never reorder the list.Do not remove any existing vectors.(Jump to a dummy routine that returns a failure status, if you must.)(Reordering the list of vectors or removing individual vectors will tend to break existing applications that are linked against the shareable image.)
For declaring a transfer vector for data, use the following lines in the OpenVMS Alpha LINKER options file:
SYMBOL_VECTOR = ( variable1 = DATA,- variable2 = DATA )
Procedure to create common entry points (OpenVMS VAX and Alpha):
It is quite possible to create a small command procedure that produces either a linker options file or a transfer vector array, based on the contents of a list of entry points -- this procedure would allow maintenance of one source file for both varieties (OpenVMS VAX and Alpha) of transfer vector.
Adapt the following example procedure for your environment and files (OpenVMS VAX):
The following command procedure assumes that the objects, object libraries, and/or shareable images specific to the application have already been created, and that the necessary Macro32 module containing the transfer vectors has already been created, and thus that this procedure need only specify the components and commands specifically necessary for the shareable image LINK operation.This particular example assumes that all application object modules are included in the object library named facnam_SHR.OLB.
$!LINK.COM $! $!This is an example of how to convert a user-created, $!application-specific object library, facnam_SHR.OLB, $!into an OpenVMS shareable image... $ $!Assemble the transfer vectors... $! $ Macro xfrvec.mar /Object=Sys$Scratch:$$$XFRVEC $ $!Now create the link options file on the fly... $ $!Force any existing LNKOUT output file closed... $! $ Close/NoLog LNKOPT $ $!Now open and populate the LNKOPT linker options $!output file... $! $ Open/Write LNKOPT Sys$Scratch:$$$xfrvec.opt $!The (CLUSTER and COLLECT directives are present $!to pull the transfer vectors to the very front $!of the shareable image.) $ Write LNKOPT"Cluster=$$$XFRVEC" $ Write LNKOPT"Collect=$$$XFRVEC,$$$XFRVEC" $ Write LNKOPT"GSMATCH=LEQUAL,19,88" $ Write LNKOPT"IDENTIFICATION=""facnam V1.0"" " $ Close LNKOPT $ $!Pull everything together... $! $ Link - /Shareable=facnam_SHR.EXE - /Symbol=facnam_SHR.Stb - /Map=facnam_SHR.Map - /NODEBUG /NOTRACEBACK - Sys$Scratch:$$$XFRVEC.Obj, - Sys$Scratch:$$$XFRVEC.Opt/Option, - facnam_SHR.OLB/Lib $ $!And clean up $! $ Delete Sys$Scratch:$$$XFRVEC.Obj.* $ Delete Sys$Scratch:$$$XFRVEC.Opt.* $ $ Exit $
OpenVMS Alpha shareable images Macro32 transfer vector modules:
OpenVMS Alpha shareable shareable images do not require a Macro32 transfer vector module, the linker options file replaces this module.
Evaluate the PSECTs for Shareability:
Look through the image /MAP produced by the image LINK.In particular, locate ANY variables that have both the PSECT (program section) attribute of "SHR" (shareable) and "WRT" (writeable).Make these variables NOSHR (not shareable) or NOWRT (not writeable) -- how to do this is further discussed later in this section.
If you fail to make all shareable PSECTS non-writable, you will have writeable variables shared between processes, and this typically leads to undesirable consequences. If both "WRT" and "SHR" are desired, you will have to INSTALL the shareable with the /WRITEABLE option before anyone can use it. Using this qualifier also restricts installation to a single node in a VMScluster.
Also make certain that all code in the shareable is position independent code (PIC). (All Digital VAX high level languages typically produce position independent code -- watch out for code written in MACRO32, however.It is quite possible to write position dependent code in MACRO32.)
DEC C supports various external model pragmas, and these provide better control of the external declarations.
Should you wind up with a PSECT marked SHR and you want it NOSHR, you can copy the other PSECT attributes and a NOSHR onto a PSECT_ATTR directive in the LINKER options file. (This is a `brute force' method -- fixing the problem(s) by adding a language keyword (eg: "noshare" in VAX C) in the source is a better long term solution.) The following command procedure can be used to scan the LINK map looking for the WRT, SHR attribute combination.
$!PSECT_SCAN.COM - This procedure uses $SEARCH to find any $!PSECTs marked both SHR and WRT.Any such lines lines $!lines must be redefined as NOSHR,WRT with a PSECT_ATTR $!line in the linker or by using a language-specific $!directive such as the VAX C "noshare" storage modifier. $! $!$ $ search facnam_SHR.map " SHR,"," WRT,"/MATCH=AND - /output=SYS$SCRATCH:facnam_SHR_SHR-WRT.tmp $ save_status = $status $ If save_status .eqs. "%X00000001" $ Then $ ! Successful status means a bad PSECT combination was detected. $ Write Sys$Output "Invalid PSECT attributes have been detected." $ Write Sys$Output "The PSECT(s) requiring adjustment(s) are:" $ Type SYS$SCRATCH:facnam_SHR_SHR-WRT.tmp $ Exit 9 $ Endif $ If save_status .eqs. "%X08D78053" $ Then $ ! Here, "no strings matched" means that you have no problem(s). $ Write Sys$Output "Successful PSECT_SCAN completion." $ Write Sys$Output "No invalid PSECT attributes have been detected." $ Delete SYS$SCRATCH:facnam_SHR_SHR-WRT.tmp;* $ Exit 1 $ Endif $ Exit 1
Debugging:
Adapt the LINK.COM command procedure to allow /TRACEBACK and/or /DEBUG during testing and development.(But by all means, do not ship the shareable image with those options....) To facilitate debugging, you will typically need to compile with qualifiers such as /DEBUG and /NOOPTIMIZE, and LINK with /DEBUG.
The debugger "SET IMAGE" command allows you to access and debug shareable image.
The INSTALL utility, to prevent security problems, is explicitly coded to refuse installation of any images linked with /TRACEBACK or with /DEBUG.See the comments on INSTALL, below.
Set the Shareable Image Version and Matching:
To update the minor version of the GSMATCH line (`88' in the above LINK.COM command procedure) whenever making upward-compatable changes. Update the major version (`19' in the example) when making incompatable changes.
Updating the minor version means programs linked against a new shareable will fail when an attempt is made to run the image against an older shareable.Updating the major version forces all images linked against the shareable to be relinked.
Programs linked against an older shareable image will work with a newer shareable image.(Assuming both have the same major version number.)
A few examples of incompatable changes:
A few examples of upward compatable changes:
Update the IDENTIFICATION option:
Update the image IDENTIFICATION string.This string is included in the LINKER options file.In the above example, this file is dynamically created by the example command procedure shown above.This string should be altered whenever making releases, so that it is obvious what product the shareable is part of.It is also often useful to encode a version string and a build baselevel number, to indicate which version and build baselevel the shareable image represents.
The DIGITAL Software Engineering Standards requires that all DIGITAL products place their registered product name and version in this field. (And even if one is not developing a "real" product, this is still considered polite programming practice.)
Some shareable images deliberately preallocate a series of dummy transfer vectors within the sharable image transfer vector array, and the programmers then reassign these vectors as newer routines are added.This technique avoids having to increment the GSMATCH version whenever newer routines are implemented or updated, easing the application compatibility across multiple versions of the shareable image.This technique can be useful when there are frequent changes during debugging, and in cases where both upward and downward compatibility can be maintained.However, this technique removes any chance of catching cases of newer applications running against older and potentially incompatible shareable images, and the shareable image programmers must use particular care to maintain both upward and downward compatibility of the interface across the entire series of shareable image versions sharing the same GSMATCH value.
When a transfer vector does not reference an actual routine, it must return an application status code that indicates the routine is not currently implemented in the library.Further, it is imperative that the shareable image identification be maintained, so that there is some indication of what is contained within a particular copy of the shareable image.
When using shareable images that use the same GSMATCH value and the dummy transfer vector technique, all applications that reference the shareable image should be coded to expect an application-defined NOTIMPL status, should the application be invoked against an older shareable image, a version that predates the necessary support.(This is one of the cases that most programmers will depend on GSMATCH to detect and prevent, and this is one of the reasons why most shareable images should increment the minor GSMATCH whenever changes are made to the shareable image.)
Errors and Resolutions:
%LINK-E-OUTSIMG
This error can also occur as a result of a bug in the image section processing in the LINKER utility -- if the address is above 64K pages (hex 02000000), this error can be generated.(Work with the options file to reduce the size of the image section below 64K pages.)This bug is present in the LINKER in OpenVMS V5.5-2, and possibly in later versions.
%SYSTEM-F-VECFULL
Because of these vector limits, one can create an executable image that incorporates references to at most 42 seperate privileged shareable images.
On OpenVMS V7.1 and later, duplicate message section entries are suppressed.
In most cases, a single PSECT that is marked with both the WRT (writeable) and the SHR (shareable) attributes is undesirable, and should be avoided -- the unintentional and/or uncoordinated sharing of data among multiple applications can cause various transient problems at run-time. Additional information on program sections is available.
How does one use and debug the shareable image?
$! LINK against the facnam_SHR.EXE image. $ Link/Executable=ImageName.EXE - /NODEBUG /NOTRACEBACK - UserObject, Sys$Input:/Option facnam_SHR.EXE/Share $
To debug a shareable image, one can use the /DEBUG and /NOOPTIMIZE qualifiers on the object modules used to create the shareable image, and one can also use the /DEBUG qualifier on the LINK command used to LINK the shareable image.Within the OpenVMS Debugger, one can use the SET IMAGE command to access the symbol table and the source code associated with the shareable image.
What Is An OpenVMS VAX Transfer Vector?
OpenVMS VAX transfer vectors are typically aligned at longword, quadword or octaword boundaries.Correct alignment facilitates easy replacement of an individual entry without affecting the other entries in the shareable image transfer vector table, and without requiring error-prone padding offset calculations. [For example, the ".mask"/"JMP nn+2" pair might be replaced with a "MOVL SS$_NORMAL,R0"/"RET" instruction pair, should a particular routine in the transfer vector no longer be required.]To maintain upward compatibility, the size of the replacement instruction(s) must not alter the alignment (the offset from the base of the image) of other vectors in the table.
In the typical case, a vector that references a CALLS/CALLG routine entry point, each entry in the transfer vector contains the contents of the "real" (word-length) CALLS/CALLG entry mask, and a (self-relative and position independent, also known as PC-relative) jump to the first instruction of the "real" routine.(The word-length entry mask [per the VAX Calling Standard] is the two bytes that are located at the entry point, thus the "JMP real_routine_offset_address+2" must skip over them.)
Logically, a constant value can be embedded in the transfer vector as is demonstrated in the XFRCON macro, or as implemented in the ".mask" of the XFRVEC macro.(The former value is a longword value, the latter case is obviously a word.)XFRCON does not attempt to provide a symbolic name for the constant, so any images calling the shareable will have to do some pointer arithmetic to locate the constant value.Your application might want to use a symbol -- use a MACRO32 external_symbol:: (the double colon notation is important), or you might be able to get the .TRANSFER directive to work for you.
OpenVMS Alpha uses a somewhat different construct.
Based Shareable Image Caveats
Based shareable images should be avoided ("like the plague") whenever possible.Based images result in address space conflicts when other shareable images increase in size; thus cause a requirement for frequent image relinks.
If the linker reports the shareable image will be based, review the code, locate the offending instruction(s), and rework the path.(The MACRO32 manual contains instructions on position independent design.)
If you are taking the time to write a shareable, and the result is a based shareable image -- you've wasted considerable effort for a rather small gain.
User-Written System Service Caveats (OpenVMS VAX)
The design and coding required for a user-written system service is documented in the appendix of the system services manual and in the examples in SYS$EXAMPLES:USS*.*.To grant the caller the enhanced privileges, the user-written system service must be installed with the /PROTECT qualifier, not with the /PRIVILEGED qualifier.(The INSTALL /PRIVILEGED qualifier should be specified only on an executable image, the qualifier does not have the desired effect of granting privileges on an oridinary shareable image. Specification of /PRIVILEGED does, however, have some potentially interesting, unintended and likely undesireable side effects; it should not be specified on an ordinary shareable image.The same prohibition on specification of the /PRIVILEGED qualifier holds for user-written system services.)User-written system services gain enhanced privileges by the implicit SETPRV privilege that is granted all routines running in executive or kernel mode.See the appendix of the system services reference manual, and the examples in SYS$EXAMPLES:USS*.*, for more information.
Failure to specify the INSTALL qualifiers correctly for the specific type of image involved (executable, shareable or privileged system service) leads to confusing error messages, failure to receive enhanced privileges and overzealous treatment as a privileged image.(in the case of an ordinary shareable image installed /PRIVILEGED, no enhanced privileges are granted, but image rundown treatment is that of of a privileged image -- no CTRL/Y DEBUG is allowed.
User-written system services should not be installed with /PRIVILEGED, only with /PROTECT.
An example of a Digital image that uses the user-written system service interface is the OpenVMS V5.3 (and later) image SYS$SHARE:SECURESHRP.EXE
User-written system services, as the routines execute in elevated processor modes and the routines are located in program sections placed on virtual memory pages owned by elevated processor modes, should be debugged with the XDELTA debugger. [One known bug in XDELTA: program sections (psects) in the process or control regions (P0/P1 address space) may need to be marked writeable ("WRT") for the duration of the debugging.Obviously, this problem directly affects user-written system services. XDELTA currently has occassional problems inserting breakpoints in P0/P1 space marked as NOWRT.This typically manifests itself on the XDELTA "B" and "O" commands.]
Do not use the OpenVMS Debugger (DEBUG) to debug executive or kernel mode code -- if you somehow manage to get the privileged-mode code going in DEBUG, the mechanism by which the debugger implements breakpoints will likely cause a system crash when the breakpoint is hit -- the system will likely crash soon.
For privileged-mode code, use the XDELTA or DELTA debugger.XDELTA is used for kernel-mode code that normally executes at elevated IPL and for code that executes without a process context, while DELTA is suited for inner-mode code running in the context of a process.
Alternatively, if symbolic debugging of privileged-mode code is desired, the OpenVMS Alpha System Code Debugger can be configured and used.The System Code Debugger uses the OpenVMS Debugger to debug inner-mode software that is running on a remote host.
User-Written System Service Caveats (OpenVMS Alpha)
OpenVMS Alpha UWSS examples are located in SYS$EXAMPLES:UWSS*.*.
UWSS System Service Alternative:
If the system is running OpenVMS V6.0 or later, consider using a subsystem access control list entry (ACE) to allow finely-grained control over access, and to avoid the need for executive or kernel mode code -- the subsystem ACE allows the system or application manager to "grant" one or more identifiers to the user for the duration of the execution of an image, via a subsystem ACE on the image(s), and via a standard identifer ACE on the target object.(An "object" can be a file, mailbox, global section, etc.)And unlike an OpenVMS privilege, the access granted (or denied) by a specific identifier can be directly and easily controlled.
For information on the subsystem ACE, see the Guide to OpenVMS System Security.
Note that there exists a patch for images that use the subsystem ACE and are also installed /SHARED -- this combination causes the image to erroneously exit with the "SUBTRACED" (SS$_SUBTRACED) error. This patch is needed for OpenVMS V6.0 and V6.1 systems -- this problem was corrected in OpenVMS V6.2.
UWSS code cannot make arbitrary "outbound" calls to shareable images, as this could result in security problems.UWSS images can be linked against specific shareable images via specification of /NOSYSSHR on the LINK command and the explicit specification of the target shareable image(s) via the options file.
UWSS SECURITY NOTE:
Avoid an attempt to implement security checks within a UWSS image if it is possible to perform the necessary security check(s) via a call to the SYS$CHKPRO or SYS$CHECK_ACCESS system service.
INSTALL, RUN, LIB$FIND_IMAGE_SYMBOL (and the Image Activator) NOTES:
Note that there is no way to "unmerge" a specific image, short of the execution of an image (or process) rundown.
DEC C++ NOTES:
DEC C++ versions V5.6 and later include instructions, located in sys$common:[syshlp.examples.cxx], that allow you to correctly create shareable images using C++.Included in the information is how to determine what C++ and OpenVMS features you require, how to correctly deal with the mangled names, and how to create C++ shareable images on OpenVMS VAX and OpenVMS Alpha.
Unintentional Inclusion of Shareable Images:
It is possible that shareable images are indirectly included into the link of a shareable or executable image by the OpenVMS VAX LINKER -- where linking against one shareable image can implicitly include one or more other shareable images, and these images are also included in the shareable image list incorporated in the image headers of the image being linked.While this is often benign, there are situations where this behaviour of the OpenVMS VAX LINKER can cause problems.
Should it be necessary to avoid this behaviour, duplicate the entry points for the explicitly included shareable image and use these to create a "stub" image.One can then link any image(s) against this "stub" image and not include any other shareable images.
The OpenVMS Alpha LINKER does not have this behaviour.
 |
|
