WASD Hypertext Services - Technical Overview

13 - Mapping Rules

13.1 - Rule Interpretation
13.2 - VMS File System Specifications
13.3 - Extended File Specifications (ODS-5)
    13.3.1 - Characters In Request Paths
    13.3.2 - Characters In Server-Generated Paths
13.4 - Rules
    13.4.1 - MAP, PASS, FAIL Rules
    13.4.2 - REDIRECT Rule
    13.4.3 - USER Rule
    13.4.4 - EXEC/UXEC and SCRIPT, Script Mapping Rules
    13.4.5 - SET Rule
13.5 - Mapping Examples
13.6 - Virtual Servers
13.7 - Conditional Mapping
13.8 - Mapping User Directories (tilde character ("~"))
    13.8.1 - Using The SYSUAF
    13.8.2 - Without Using The SYSUAF
[next] [previous] [contents] [full-page]

Mapping rules are used for a few different purposes.

  1. To map a request path onto the VMS file system.

  2. To process a request path according to specified criteria resulting in an effective path that is different to that supplied with the request.

  3. To identify requests requiring script activation and to parse the script from the path portion of that request. The path portion is then independently re-mapped.

  4. To conditionally map to different end-results based on one or more criteria of the request.

  5. To provide differing virtual sites depending on the actual service accessed by the client.

Mapping is basically for server-internal purposes only. The only time the path information of the request itself is modified is when a script component is removed. At all other times the path information remains unchanged. Path authorization is always applied to the path supplied with the request.

By default, the system-table logical name HTTPD$MAP locates a common mapping rule file, unless an individual rule file is specified using a job- table logical name. Simple editing of the mapping file changes the rules. Comment lines may be included by prefixing them with the hash "#" character. Although, there is no fixed limit on the number of rules there are the processing implications of scanning a large, linear database.

The [IncludeFile] is a directive common to all WASD configuration, allowing a separate file to be included as a part of the current configuration (6.12 - Include File Directive).

Rules are given a basic consistency check when loaded (i.e. server startup, map reload, etc.) If there is an obvious problem (unknown rule, missing component, etc., path not absolute) a warning message is generated and the rule is not loaded into the database. This will not cause the server startup to fail. These warning messages may be found in the server process log.

Checking Mapping Rules

A server's currently loaded mapping rules may be interrogated. See 18 - Server Administration for further information. Also the Server Administration facility allows realm and arbitrary paths to be checked against the rule database in real-time using the WATCH facility (19 - WATCH Facility). In this way the rule database may be checked against test or even live requests.

Any subsequent changes to the mapping file may be reloaded into the running HTTPd server using the following command on the server system:

  $ HTTPD /DO=MAP


13.1 - Rule Interpretation

The rules are scanned from first towards last, until a matching final rule is encountered (PASS, EXEC, SCRIPT, FAIL, REDIRECT, UXEC and USER) when the mapping pass concludes. Non-final rules (MAP and SET) perform the appropriate action and continue to the next rule. One, two or more passes through the rules may occur due to implicit processing (if the path contains a script component) or by explicit restart (SET map=restart).


String Matching

The basis of path mapping is string pattern matching, comparing the request specified path, and optionally other components of the request when using configuration conditionals (12 - Conditional Configuration), to a series of patterns, usually until one of the patterns matches, at which stage some processing is performed. Both wildcard and regular expression based pattern matching is available. All rules have a template (string pattern to match against the path). Some rules have a result (how to restructure the components matching from the template).


Virtual Servers

As described in 6.3 - Virtual Services virtual service syntax may be used with mapping rules to selectively apply rules to one specific service. If virtual services are configured rule interpretation sees only rules common to all services and those specific to it's own service (host address and port). In all other aspects rule interpretation applies as described above.


Processing Overhead

Naturally, each rule that needs to be processed adds a little to consumed CPU, introduces some latency, and ultimately reduces throughput. The test-bench has shown this to be acceptably small compared to the overall costs of responding to a request. Using the ApacheBench tool on a Digital AlphaStation 500/333 with 256MB, VMS V7.3-1, and WASD v8.3, with a simple access to /ht_root/exercise/0k.txt showed approximately 108 requests/second throughput using the following mapping file.

  pass /ht_root/exercise/* 

After adding various quantities of the same intervening rule

  pass /ht_root/example/* 
  pass /ht_root/example/* 
    .
    .
    .
  pass /ht_root/example/* 
  pass /ht_root/exercise/* 
the following results were derived.

Mapping Overhead
Intervening RulesRequests/SThroughput
0108baseline
100104-3.7%
200100-7.4%
50091-15.7%
100070-35.1%

Although this is a fairly contrived set-up and actual real-world rule-sets are more complex than this, even one hundred rules is a very large set, and it does indicate that for all intents and purposes mapping rules may be used to achieve desired objectives without undue concern about impact on server throughput.


13.2 - VMS File System Specifications

The VMS file system in mapping rules is always assumed to begin with a device or concealed device logical. Specifying a Master File Directory (MFD) component, the [000000] is completely optional, although always implied. The mapping functions will always insert one if required for correct file system syntax. That is, if the VMS file system mapping of a path results in a file in a top-level directory an MFD is inserted if not explicitly present in the mapping. For example, both of the following paths

  /dka100/example.txt
  /dka100/000000/example.txt
would result in a mapping to
  DKA100:[000000]EXAMPLE.TXT
The MFD is completely optional when both specifying paths in mapping rules and when supplying paths in a request. Similarly, when supplying a path that includes directory components, as in
  /dka100/dir1/dir2/example.txt
  /dka100/000000/dir1/dir2/example.txt
both mapping to
  DKA100:[DIR1.DIR2]EXAMPLE.TXT
Implication

When using logical names in file system mappings they must be able to be used as concealed devices and cannot be logical equivalents of directory specifications.

Concealed device logicals are created using the following syntax:

  $ DEFINE LOGICAL_NAME device:[dir1.dir2.]
  $ DEFINE LOGICAL_NAME /TRANSLATION=CONCEALED physical_device:[dir1.dir2.]

For ODS-2 volumes (13.3 - Extended File Specifications (ODS-5) immediately below), when during rule mapping of a path to a VMS file specification an RMS-invalid character (e.g. "+") or syntax (e.g. multiple periods) is encountered a dollar symbol is substituted in an attempt to make it acceptable. This functionality is often useful for document collections imported to the local web originating from, for instance, a Unix site that utilizes non-RMS file system syntax. The default substitution character may be changed on a per-path basis using the SET rule (13.4.5 - SET Rule).


13.3 - Extended File Specifications (ODS-5)

OpenVMS Alpha V7.2 introduced a new on-disk file system structure, ODS-5. This brings to VMS in general, and WASD and other Web servers in particular, a number of issues regarding the handling of characters previously not encountered during (ODS-2) file system activities. It is necessary to distinguish paths to ODS-5, extended specification volumes from the default ODS-2 ones (13.4.5 - SET Rule).


13.3.1 - Characters In Request Paths

There is a standard for characters used in HTTP requests paths and query strings (URLs). This includes conventions for the handling of reserved characters, for example "?", "+", "&", "=" that have specific meanings in a request, characters that are completely forbidden, for example white-space, control characters (0x00 to 0x1f), and others that have usages by convention, for example the "~", commonly used to indicate a username mapping. The request can otherwise contain these characters provided they are URL-encoded (i.e. a percentage symbol followed by two hexadecimal digits representing the hexadecimal-encoded character value).

There is also an RMS standard for handling characters in extended file specifications, some of which are forbidden in the ODS-2 file naming conventions, and others which have a reserved meaning to either the command-line interpreter (e.g. the space) or the file system structure (e.g. the ":", "[", "]" and "."). Generally the allowed but reserved characters can be used in ODS-5 file names if escaped using the "^" character. For example, the ODS-2 file name "THIS_AND_THAT.TXT" could be named "This^_^&^_That.txt" on an ODS-5 volume. More complex rules control the use of character combinations with significance to RMS, for instance multiple periods. The following file name is allowed on an ODS-5 volume, "A-GNU-zipped-TAR-archive^.tar.gz", where the non-significant period has been escaped making it acceptable to RMS.

The WASD server will accept request paths for file specifications in both formats, URL-encoded and RMS-escaped. Of course characters absolutely forbidden in request paths must still be URL-encoded, the most obvious example is the space. RMS will accept the file name "This^ and^ that.txt" (i.e. containing escaped spaces) but the request path would need to be specified as "This%20and%20that.txt", or possibly "This^%20and^%20that.txt" although the RMS escape character is basically redundant.

Unlike for ODS-2 volumes, ODS-5 volumes do not have "invalid" characters, so no processing is performed to ensure RMS compliance.


13.3.2 - Characters In Server-Generated Paths

When the server generates a path to be returned to the browser, either in a viewable page such as a directory listing or error message, or as a part of the HTTP transaction such as a redirection, the path will contain the URL-encoded equivalent of the canonical form of an extended file specification escaped character. For example, the file name "This^_and^_that.txt" will be represented by "This%20and%20that.txt".

When presenting a file name in a viewable page the general rule is to also provide this URL-equivalent of the unescaped file name, with a small number of exceptions. The first is a directory listing where VMS format has been requested by including a version component in the request file specification. The second is in similar fashion, but with the tree facility, displaying a directory tree. The third is in the navigation page of the UPDate menu. In all of the instances the canonical form of the extended file specification is presented (although any actual reference to the file is URL-encoded as described above).


13.4 - Rules

These are the categories of mapping rules.


13.4.1 - MAP, PASS, FAIL Rules

  1. map template result

    If the URL path matches the template, substitute the result string for the path and use that for further rule processing. Both template and result paths must be absolute (i.e. begin with "/").

  2. pass template
    pass template result
    pass template "999 message text"

    If the URL path matches the template, substitute the result if present (if not just use the original URL path), processing no further rules.

    The result should be a either a physical VMS file system specification in URL format or an HTTP status-code message (see below). If there is a direct correspondance between the template and result the result may be omitted.

    NOTE

    The PASS directive is also used to reverse-map VMS file specifications to the URL path format equivalent.

    An HTTP status-code message can be provided as a result. The server then generates a response corresponding to that status code containing the supplied message. Status-code results should be enclosed in one of single or double quotes, or curly braces. See examples. A 3nn status results in a redirection response with the message text comprising the location. Codes 4nn and 5nn result in an error message. Other code ranges (e.g. 0, 1nn, 2nn, etc.) simply cause the connection to be immediately dropped, and can be used for that purpose (i.e. no indication of why!)

  3. fail template

    If the URL path matches the template, prohibit access, processing no further rules. The template path must be absolute (i.e. begin with "/").


13.4.2 - REDIRECT Rule

  1. redirect template result

    If the URL path matches the template, substitute the result string for the path. Process no further rules. Redirection rules can provide result URLs in one of a number of formats, each with a slightly different behaviour.

    1. The result can be a full URL ("http://host.domain/path/to/whatever"). This is used to redirect requests to a specific service, usually on a another host. A result may or may not contain a fixed query string ("/path/to/whatever?one=two").

    2. If the scheme (e.g. "http:") is omitted the scheme of the current request is substituted. This allows HTTP requests to be transparently redirected via HTTP and HTTPS (SSL) requests via HTTPS (e.g. "//host.domain/path/to/whatever", note the leading double-slash).

    3. In a similar fashion both the scheme and the host name may be omitted (e.g. "///path/to/whatever", note the leading triple-slash). The server then substitutes the appropriate request scheme and host name before returning the redirection to the client.

    4. If the scheme is provided but no host component the current request's host information is substituted and the redirection made using that (e.g. "https:///secure/path/to/whatever". This effectively allows a request to be redirected from standard to SSL, or from SSL to standard HTTP on the same server.

    5. Alternatively, it may be just a path ("/path/to/whatever", a single leading slash), which will cause the server to internally generate an entire new request structure to process the new path (i.e. request redirection is not returned to the client).
      NOTE

      Internal redirection (as this is termed) is a fundamental mechanism available with WASD to completely change the request path and/or query string components for the request - transparently to the client. It is essentially a complete rewrite of the request.

    6. Only if the last character in the result is a question mark ("?") will any query string in the original be propagated into the redirection URL (that is the original request "/original/test.txt?plus=query" is mapped using "redirect /original/* /path/to/*?" does the resulting URL become "/path/to/test.txt?plus=query").


13.4.3 - USER Rule

The USER rule maps a VMS user account default device and directory (i.e. home directory) into a request path. That is, the base location for the request is obtained from the VMS systems SYSUAF file. This is usually invoked by a request path in the form "/~username/", see 13.8 - Mapping User Directories (tilde character ("~")) for more detailed information.

  1. user template result

    If the path matches the template then the result is substituted, with the following conditions. At least one wildcard must be present. The first wildcard in the result substitutes the username's home directory into the path (in place of the "~username"). Any subsequent wildcard(s) substitute corresponding part(s) of the original path.

    If the user DANIEL's default device and directory were

      USER$DISK:[DANIEL]
    
    the following rule
      user /~*/* /*/www/*
    
    would result in the following path being mapped and used
      /user$disk/daniel/www/
    
NOTE

Accounts that possess SYSPRV, are CAPTIVE, have been DISUSERED or that have expired passwords will not be mapped. A "directory not found" error report is returned.


13.4.4 - EXEC/UXEC and SCRIPT, Script Mapping Rules

Also see "Scripting Environment" document for further information.

The EXEC/UXEC and SCRIPT directives have the variants EXEC+/UXEC+ and SCRIPT+. These behave in exactly the same fashion and simply mark the rule as representing a CGIplus script environment.

The EXEC/UXEC rules maps script directories.

The SCRIPT rules maps script file names. It behaves a little differently to the EXEC rule, essentially supplying in a single rule the effect of a MAP then an EXEC rule.

Both rules must have a template and result, and both must end in a wildcard asterisk. The placement of the wildcards and the subsequent functionality is slightly different however. Both template and result paths must be absolute (i.e. begin with "/").

  1. exec template result

    The EXEC rule requires the template's asterisk to immediately follow the slash terminating the directory specification containing the scripts. The script name follows immediately as part of the wildcard-matched string. For example:

      exec /htbin/* /ht_root/script/*
    

    If the URL path matches the template, the result, including the first slash-terminated part of the wildcard-matched section, becomes the URL format physical VMS file specification the script to be executed. What remains of the original URL path is used to create the path information. Process no further rules.

    Hence, the EXEC rule will match multiple script specifications without further rules, the script name being supplied with the URL path. Hence any script (i.e. procedure, executable) in the specified directory is accessable, a possible security concern if script management is distributed.

  2. exec template (run-time-environment)result

    A variation on the "exec" rules allows a Run-Time Environment (RTE) to be mapped. An RTE is a persistant scripting environment not unlike CGIplus. The essential difference is an RTE provides an environment in which a variety of scripts can be run. It is often an interpreter, such as Perl, where the advantages of persistance (reduced response latency and system impact) are available. For more information on RTEs and how they operate see the WASD Scripting Environment document.

    The RTE executable is specified in parentheses prefixed to the mapping result, as show in this example:

      exec /pl-bin/* (cgi-bin:[0000000]perlrte.exe)/ht_root/src/perl/*
    

  3. script template result

    The SCRIPT rule requires the template's asterisk to immediately follow the unique string identifying the script in the URL path. The wildcard-matched string is the following path, and supplied to the script. For example:

      script /conan* /ht_root/script/conan*
    

    If the URL path matches the template, the result becomes the URL format physical VMS file specification for the DCL procedure of the script to be executed (the default file extension of ".COM" is not required). What remains of the original URL path is used to create the path information. Process no further rules.

    NOTE

    The wildcard asterisk is best located immediately after the unique script identifier. In this way there does not need to be any path supplied with the script. If even a slash follows the script identifier it may be mapped into a file specification that may or may not be meaningful to the script.

    Hence, the SCRIPT rule will match only the script specified in the result, making for finely-granular scripting at the expense of a rule for each script thus specified. It also implies that only the script name need precede any other path information.

    It may be thought of as a more efficient implementation of the equivalent functionlity using two CERN rules, as illustrated in the following example:

      map /conan* /script/conan*
      exec /cgi-bin/* /cgi-bin/*
    

  4. uxec template result

    The UXEC rule is an analog to the EXEC rule, except it is used to map user scripts. It requires two mapping asterisks, the first for the username, the second for the script name. It must be used in conjunction with a SET script=as=~ rule. For example:

      SET   /~*/www/cgi-bin/*  script=as=~
      UXEC  /~*/cgi-bin/*  /*/www/cgi-bin/*
    

    For further information see User Account Scripting and the "Scripting Overview, Introduction".


Script Location

It is conventional to locate script images in HT_ROOT:[AXP-BIN] or HT_ROOT:[VAX-BIN] (depending on the platform), and procedures, etc. in HT_ROOT:[CGI-BIN]. These multiple directories are accessable via the single search list logical CGI-BIN.

Script files can be located in area completely outside of the HT_ROOT tree. Two approaches are available.

  1. Modify the search list CGI-BIN to include the additional directories. Only should be done with extreme care.

  2. Use mapping rules to make the script accessable. This can be done by using the EXEC or SCRIPT rule to specify the directory directly as in these examples
      exec /mycgi-bin/* /site_local_scripts/bin/*
      script /myscript* /web/myscripts/bin/myscript.exe*
    
    or by using the MAP rules to make a hierarchy of script locations obvious and accessable, as in this example
      map /cgi-bin/myscripts/* /cgi-bin_myscripts/* 
      exec /cgi-bin_myscripts/* /web/myscripts/bin/*
    


EXEC Directories and EXEC Files

Generally directories are specified as locations for script files. This is the more common application, with the EXEC rules used as in this example

  exec /cgi-bin/* /cgi-bin/*

Mapping a file type into an EXEC behaviour is also supported. This allows all files within the specified path and with the matching file suffix (extension) to be activated as scripts. Of course a script runtime must be available for the server to be able activate it. The following example demonstrates mapping all files ending in .CGI in the /web/ tree as executable scripts.

  exec /web/*.cgi* /web/*.cgi*
WARNING

Remember scripts are executables. Enabling scripting in a general user area allows any user to write and execute any script, by default under the server account. Deploy with discretion.


13.4.5 - SET Rule

The SET rule does not change the mapping of a path, it just sets one or more characteristics against that path that affect the subsequent processing in some way. It is a general purpose rule that conveniently allows the administrator to tell the server to process requests with particular paths in some ad hoc and generally useful fashion. Most SET parameters are single keywords that act as boolean switches on the request, some require parameter strings. Multiple space-separated parameters may be set against against the one path in a single SET statement.

Of course, as with all mapping rules, paths containing file types (extensions) may be specified so it is quite easy to apply settings to particular groups of files. Multiple settings may be made against the one path, merely separate set directives from each other with white-space. If a setting string is required to contain white-space enclose the string with single or double quotes, or curly brackets. The following example gives a small selection of potential uses.

  # examples of SET rule usage
  # --------------------------
  # disable caching for selected paths
  set /ht_root/src/* NOcache
  set /sys$common/* NOcache
  # enable stream-LF conversion in selected directory trees
  set /web/* stmlf
  set /ht_root/* stmlf
  # respond with Cyrillic character set(s) from relevant directories
  set /*/8859-5/* charset=ISO-8859-5
  set /*/koi8-r/* charset=KOI8-R
  # the Sun Java tutorial when UNZIPped contains underscores for invalid characters
  set /vms/java/tutorial/* RMSchar=_
  # if a request has "/plain-text/" in it's path then ALWAYS return as plain-text!
  set /*/plain-text/* content=text/plain
  map /*/plain-text/* /*/*
  # same for "/binary/"
  set /*/binary/* content=text/plain
  map /*/binary/* /*/*
  # indicate extended file specifications on this path
  set /Documents/* ODS=5
  pass /Documents/* /ods5_device/Documents/*
  # throttle this script's execution, 5 executing, unlimited waiting
  set /cgi-bin/big_script* throttle=5
  # disable server script search for this RTE
  set /onerte/*  script=nofind
  exec /onerte/* (CGI-BIN:[000000]ONERTE.EXE)/ht_root/src/one/*


Postfix SET Rule

Path SETings may appended to any rule that contains both a template and result. This makes it possible to apply path SETings using matching final rules. For example a matching PASS rule does not require a separate, preceding SET rule containing the same path to also apply required SETings. This is more efficient (requiring less pattern matching) and tends to make the rule set less cluttered.

  # examples of postfix SET rule usage
  # ----------------------------------
  # if a request has "/plain-text/" in it's path then ALWAYS return as plain-text!
  map /*/plain-text/* /*/* content=text/plain
  # same for "/binary/"
  map /*/binary/* /*/* content=text/plain
  # indicate extended file specifications on this path
  pass /Documents/* /ods5_device/Documents/* ODS=5
  # throttle this script's execution, 5 executing, unlimited waiting
  script /cgi-bin/big_script* throttle=5


13.5 - Mapping Examples

The example mapping rule file for the WASD HTTP server can be viewed.


Example of Map Rule

The result string of these rules may or may not correspond to to a VMS physical file system path. Either way the resulting rule is further processed before passing or failing.

  1. The following example shows a path "/web/unix/shells/c" being mapped to "/web/software/unix/scripts/c", with this being used to process further rules.
      map /web/unix/* /web/software/unix/*
    


Examples of Pass Rule

  1. This example shows a path "/web/rts/home.html" being mapped to "/user$rts/web/home.html", and this returned as the mapped path.
      pass /web/rts/* /user$rts/web/*
    

  2. This maps a path "/icon/bhts/dir.gif" to "/web/icon/bhts/dir.gif", and this returned as the mapped path.
      pass /icon/bhts/* /web/icon/bhts/*
    

  3. This example illustrates HTTP status code mapping. Each of these does basically the same thing, just using one of the three possible delimiters according to the characters required in the message. The server generates a 403 response with has as it's text the following message. (Also see the conditional mapping examples.)
      pass /private/* "403 Can't go in there!"
      pass /private/* '403 "/private/" is off-limits!'
      pass /private/* {403 Can't go into "/private/"}
    


Examples of Fail Rule

  1. If a URL path "/web/private/home.html" is being mapped the path would immediately be failed.
      fail /web/private/*
    

  2. To ensure all access fails, other than that explicitly passed, this entry should be included the the rules.
      fail /*
    


Examples of Exec and Script Rules

  1. If a URL path "/htbin/ismap/web/example.conf" is being mapped the "/ht_root/script/" must be the URL format equivalent of the physical VMS specification for the directory locating the script DCL procedure. The "/web/example.conf" that followed the "/htbin/ismap" in the original URL becomes the translated path for the script.
      exec /cgi-bin/* /cgi-bin/*
    

  2. If a URL path "/pl-bin/example/this/directory/and-file.txt" is being mapped the script name and filename become "/pl-bin/example" and "HT_ROOT:[SRC.PERL]EXAMPLE.PL" respectively, the path information and translated become "/this/directory/and-file.txt" and "THIS:[DIRECTORY]AND-FILE.TXT", and the interpreter (run-time environment) activated to interpret the script is CGI-BIN:[000000]PERLRTE.EXE.
      exec /pl-bin/* (cgi-bin:[000000]perlrte.exe)/ht_root/src/perl/*
    

  3. If a URL path "/conan/web/example.hlb" is being mapped the "/ht_root/script/conan" must be the URL format equivalent of the physical VMS specification for the DCL procedure. The "/web/example.hlb" that followed the "/conan/" in the original URL becomes the translated path for the script.
      script /conan* /ht_root/script/conan*
    


Example of Redirect Rule

  1. If a URL path "/AnotherGroup/this/that/other.html" is being mapped the URL would be redirected to "http://host/this/that/other.html"
      redirect /AnotherGroup/* http://host/group/*
    


13.6 - Virtual Servers

As described in 6.3 - Virtual Services, virtual service syntax may be used with mapping rules to selectively apply rules to one specific service. This example provides the essentials of using this syntax. Note that service-specific and service-common rules may be mixed in any order allowing common mappings (e.g. for scripting) to be shared.

  # a mapping rule example of virtual servers
  [[alpha.domain.name:80]]
  # ALPHA is the only service allowing access to VMS help directory
  pass /sys$common/syshlp/*
  [[beta.domain.name:80]]
  # good stuff is only available from BETA
  pass /good-stuff/*
  # BETA has it's own error report format, the others share one
  pass /errorreport /httpd/-/errorreportalpha.shtml  
  [[gamma.domain.name:80]]
  # gamma responds with documents using the Cyrillic character set
  set /* charset=ISO-8859-5
  [[*]]
  # common file and script mappings
  exec /cgi-bin/* /cgi-bin/*
  exec+ /cgiplus-bin/* /cgi-bin/*
  script+ /help/* /cgiplus-bin/conan/*
  pass /errorreport /httpd/-/errorreport.shtml  
  # now the base directories for all documents
  [[alpha.domain.name:80]]
  /* /web/alpha/*
  [[beta.domain.name:80]]
  /* /web/beta/*
  [[gamma.domain.name:80]]
  /* /web/gamma/*
  [[*]]
  # catch-all rule (just in case :^)
  pass /* /web/*

The Server Administration page WATCH report (18.4 - HTTPd Server Reports) provides the capability to view the rule databse as well as rule mapping during actual request processing, using the WATCH facility.


13.7 - Conditional Mapping

(Somewhat) Deprecated and Discouraged

There is now a more versatile approach to achieving the same functionality described in this section, see 12 - Conditional Configuration. Conditional mapping will be retained for the foreseeable future with this documentation available for reference by older site configurations. The two approaches may be used concurrently as required.

The purpose of conditional mapping is to apply rules only after certain criteria other than the initial path match are met.

THIS OFFERS A POWERFUL TOOL TO THE SERVER ADMINISTRATOR!

Conditional mapping can be applied on the following criteria:

client internet address
browser-accepted languages
browser-accepted character sets
browser-accepted content-types
browser identification string
cookie data
host and port specified in request header
HTTP method (GET, POST, etc.)
proxy/gateway host(s) request forwarded by
refering page
request scheme (protocol ... "http:" or "https:")
query string
server name
server port

Conditionals must follow the rule and are delimited by "[" and "]". Multiple, space-separated conditions may be included within one "[...]". This behaves as a logical OR (i.e. the condition only needs one matched to be true). Multiple "[...]" conditionals may be included against a rule. These act as a logical AND (i.e. all must have at least one condition matched). If a condition begins with a "!" it acts as a negation operator (i.e. matched strings result in a false condition, unmatched strings in a true condition). The result of an entire conditional may also be negated by prefixing the "[" with a "!".

If a conditional, or set of conditionals, is not met the rule is completely ignored.

Both wildcard and regular expression pattern matching is available (11 - String Matching). Characters reserved for delimiting the conditional must be backslash-escaped (spaces, TABs, wildcards and the delimiting "[" and "]").

Mapping Conditionals
ConditionalDescription
ac:browser-accepted content types ("Accept:" request header field)
al:browser-accepted languages ("Accept-Language:" request header field)
as:browser-accepted character sets ("Accept-Charset:" request header field)
ck:cookie data ("Cookie:" request header field)
ex:extended file specification (boolean)
fo:request forwarded by proxy/gateway host(s) ("Forwarded:" request header field)
ho:browser host internet name or address
hm:browser host internet address compare to dotted-decimal and mask
me:request HTTP method
mp:derived map path (after SCRIPT or MAP rule)
no:"notepad" contents
pa:first or second pass (after script resolution), as '1' or '2'
pi:path information
qs:query string
rc:internally redirected count, as '0', '1', '2' ..
rf:refering page ("Referer:" request header field)
ru:request URI (non-URL-decoded path)
sc:request scheme (protocol), "http", and if SSL is in use "https" (17 - Secure Sockets Layer)
sn:server name
sp:server port
st:script name (after first pass script resolution)
ua:browser ("User-Agent:" request header field)
vs:virtual host and port request directed to ("Host:" request header field)
xf:proxied client ("X-Forwarded-For:" request header field)


Examples

NOTE

It is possible to spoof (impersonate) internet host addresses. Therefore any controls applied using host name/address information cannot be used for authorization purposes in the strictest sense of the term.

  1. The following example shows a rule being applied only if the client host is within a particular subnet. This is being used to provide a "private" home page to those in the subnet while others get a "public" page by the second rule.
      pass / /web/internal/ [ho:131.185.250.*]
      pass / /web/
    

  2. This is a similar example to the above, but showing multiple host specifications and specifically excluding one particular host using the negation operator "!". This could be read as pass if ((host OR host) AND (not host)).
      pass / /web/internal/ [ho:*.fred.com ho:*.george.com] [!ho:you.fred.com]
      pass / /web/
    

  3. The next example shows how to prevent browsing of a particular tree except from specified host addresses.
      pass /web/internal/* /web/SorryNoAccess.html [!ho:131.185.250.*]
      pass /web/internal/*
    

    This could be used to prevent browsing of the server configuration files (an alternative to this sort of approach is to use the authorization file, see 15 - Authentication and Authorization).

      pass /httpd/-/* /web/SorryNoAccess.html [!ho:131.185.250.201]
    

  4. This example performs much the same task as the previous one, but uses whole conditional negation to prevent browsing of a particular tree except from specified addresses (as well as using the continuation character to provide a more easily comprehended layout ... note the trailing spaces as required). This could be read as pass if not (host OR host OR host).
      pass /web/internal/* /web/SorryNoAccess.html \
      ![\
      ho:131.185.250.* \
      ho:131.185.251.* \
      ho:131.185.45.1 \
      ho:ws2.wasd.dsto.gov.au\
      ]
      pass /web/internal/*
    

  5. This example demonstrates mapping pages according to geography or language preference (it's a bit contrived, but ...)
      pass /doc/* /web/doc/french/* [ho:*.fr al:fr]
      pass /doc/* /web/doc/swedish/* [ho:*.se al:se]
      pass /doc/* /web/doc/english/*
    

  6. How to exclude specific browsers from your site (how many times have we seen this!)
      # I had to pick on a well-known acronym, no offence Bill!
      pass /* /web/NoThankYou.html [ua:*MSIE*]
    

  7. This example allows excluding certain requests from specific addresses. This could be read as pass if ((method is POST) AND (not host)).
      pass /* /web/NotAllowed.html [me:POST] [!ho:*.my.net]
    

  8. The following illustrates using the server name and/or server port to conditionally map servers executing on clustered nodes using the same configuration file, or for multi-homed/multi-ported hosts. Distinct home pages are maintained for each system, and on BETA two servers execute, one on port 8000 that may only be used by those within the specified network address range.
      pass / /web/welcome_to_Alpha.html [sn:alpha.*]
      pass / /web/welcome_to_Beta.html [sn:beta.*] [sp:80]
      pass /* /sorry_no_access.html [sn:beta.*] [sp:8000] [!ho:*.my.sub.net]
      pass / /web/welcome_to_Beta_private.html [sn:beta.*] [sp:8000]
    

  9. Each of these three do basically the same thing, just using the three possible delimiters according to the characters required in the message. The server generates a 403 response with has as it's text the following message.
      pass /private/* "403 Can't go in there!" [!ho:my.host.name]
      pass /private/* '403 "/private/" is off-limits!' [!ho:my.host.name]
      pass /private/* {403 Can't go into "/private/"} [!ho:my.host.name]
    

  10. This example illustrates the use of a host network mask, the "HM:" conditional.
      pass /private/* "403 Can't go in there!" [!hm:131.185.250.128/255.255.255.192]
    

    The mask is a dotted-decimal network address, a slash, then a dotted-decimal mask. This example shows a 6 bit subnet. Network mask conditionals operate by bitwise-ANDing the client host address with the mask, bitwise-ANDing the network address supplied with the mask, then comparing the two results for equality. Using the above example the host 131.185.250.250 would be accepted, but 131.185.250.50 would be rejected.

Note that rule processing for any particular path may be checked using the WATCH facility from the Server Administration page. See 19 - WATCH Facility for details.


13.8 - Mapping User Directories (tilde character ("~"))

The convention for specifying user web areas is "/~username/". The basic idea is that the user's web-available file-space is mapped into the request in place of the tilde and username.


13.8.1 - Using The SYSUAF

The USER rule maps a VMS user account default device and directory (i.e. home directory) into a request path (13.4.3 - USER Rule). That is, the base location for the request is obtained from the VMS systems SYSUAF file. A user's home directory information is cached, to reduce load on the authorization databases. As this information is usually quite static there is no timeout period on such information (although it may be flushed to make room for other user's). Cache contents is include in the Mapping Rules Report (18.4 - HTTPd Server Reports) and is implicitly flushed when the server's rules are reloaded (18.6 - HTTPd Server Action).

The following is a typical usage of the rule.

  USER  /~*/*  /*/www/*

Note the "/www" subdirectory component. It is stongly recommended that users never be mapped into their top-level, but into a web-specific subdirectory. This effectively "sandboxes" Web access to that subdirectory hierarchy, allowing the user privacy elsewhere in the home area.

To accomodate request user paths that do not incorporate a trailing delimiter after the username the following redirect may be used to cause the browser to re-request with a more appropriate path (make sure it follows the USER rule).

  REDIRECT  /~*  ///~*/

WASD also "reverse maps" VMS specifications into paths and so requires additional rules to provide these mappings. (Reverse mapping is required during directory listings and error reporting.) For the continuing example the following rules would be required (and in the stated order).

  USER  /~*/*  /*/www/*
  REDIRECT  /~*  ///~*/
  PASS  /~*/*  /user$disk/*/www/*

Where user home directories are spread over multiple devices (physical or concealed logical) a reverse-mapping rule would be required for each. Consider the following situation, where user directories are distributed across these devices (concealed logicals)

  USER$GROUP1:
  USER$GROUP2:
  USER$GROUP2:
  USER$OTHER:

This would require the following mapping rules (in the stated order).

  USER  /~*/*  /*/www/
  PASS  /~*/*  /user$group1/*/www/*
  PASS  /~*/*  /user$group2/*/www/*
  PASS  /~*/*  /user$group3/*/www/*
  PASS  /~*/*  /user$other/*/www/*

Accounts with a search list as a default device (e.g. SYS$SYSROOT) present particular complications in this schema and should be avoided.

NOTE

Accounts that possess SYSPRV, are CAPTIVE, have been DISUSERED or that have expired passwords will not be mapped. A "directory not found" error report is returned. This error was chosen to make it to make more difficult to probe the authorization environment, determining whether accounts exist or not.

Of course vanilla mapping rules may be used to provide for special cases. For instance, if there is requirement for a particular, privileged account to have a user mapping that could be provided as in the following (rather exagerated) example.

  PASS  /~system/*  /sys$common/sysmgr/www/*
  USER  /~*/*  /*/www/
  PASS  /~*/*  /user$disk/*/www/*


User Account Scripting

In some situations it may be desirable to allow the average Web user to experiment with or implement scripts. With WASD 7.1 and later, and VMS V6.2 and later, this is possible. Detached scripting must be enabled, the /PERSONA startup qualifier used, and appropriate mapping rules in place. If the SET "script=as=" mapping rule specifies a tilde character then for a user request the mapped SYSUAF username is substituted.

The following example shows the essentials of setting up a user environment where access to a subdirectory in the user's home directory, [.WWW] with script's located in a subdirectory of that, [.WWW.CGI-BIN].

  SET   /~*/www/cgi-bin/*  script=as=~
  UXEC  /~*/cgi-bin/*  /*/www/cgi-bin/*
  USER  /~*/*  /*/www/*
  REDIRECT  /~*  /~*/
  PASS  /~*/*  /dka0/users/*/*

For more detailed information see the "Scripting Overview, Introduction".


13.8.2 - Without Using The SYSUAF

Deprecated and Discouraged

There are now "better" approaches to achieving the same functionality as described in this section. This documentation is retained only for reference by older site configurations.

The server is also able to map user directories using the same mechanisms as for any other. No reference needs to be made to the SYSUAF, user support can be accomplished via a combination of mapping rule and logical name. This approach relies on a correspondance between the username and the home directory name. Hence users are made known by the HTTPd using the name of their top-level directory. User scripts can also be supported using WASD's DECnet scripting environment.

The "PASS" rule provides a wildcard representation of users' directory paths. As part of this mapping a subdirectory specifically for the hypertext data should always be included. Never map users' top-level directories. For instance if a user's account home directory was located in the area USER$DISK:[DANIEL] the following rule would potentially allow the user DANIEL to provide web documents from the home subdirectory [.WWW] (if the user has created it) using the accompanying URL:

  pass /~*/* /user$disk/*/www/*
 
  http://host/~daniel/

It is recommended that a separate logical name be created for locating user directories. This helps hide the internal organisation of the file system. The following logical name definition and mapping rule illustrate this point.

  $ DEFINE /SYSTEM /EXEC /TRANSLATION=CONCEALED WWW_USER device:[USER.]
 
  pass /~*/* /www_user/*/www/*

Where users are grouped into different areas of the file system a logical search list may be defined.

  $ DEFINE /SYSTEM /EXEC /TRANSLATION=CONCEALED -
           WWW_USER -
           DISK1:[GROUP1.], -
           DISK1:[GROUP2.], -
           DISK2:[GROUP3.], -
           DISK2:[GROUP4.]
    
  pass /~*/* /www_user/*/www/*

As logical search lists have specific uses and some complications (e.g. when creating files) this is the only use for them recommended with this server, although it is specifically coded to allow for search lists in document specifications.

If only a subset of all users are to be provided with WWW publishing access either their account directories can be individually mapped (best used only with a small number) or a separate area of the file system be provided for this purpose and specifically mapped as user space.

Of course, user mapping is amenable to all other rule processing so it is a simple matter to redirect or otherwise process user paths. For instance, the published username does not need to, or need to continue to, correspond to any real user area, or the user's actual name or home area:

  redirect /~doej/* http://a.nother.host/~doej/*
  pass /~doej/* /www/messages/deceased.html
  pass /~danielm/* /special$www$area/danielm/*
  pass /~Mark.Daniel/* /user$disk/danielm/www/*
  pass /~*/* /www_user/*/www/*

A user directory is always presented as a top-level directory (i.e. no parent directory is shown), although any subdirectory tree is accesssable by default.


[next] [previous] [contents] [full-page]