From owner-hpff-doc Mon Jun 3 11:58:25 1996 Received: (from daemon@localhost) by cs.rice.edu (8.7.1/8.7.1) id LAA02117 for hpff-doc-out; Mon, 3 Jun 1996 11:58:25 -0500 (CDT) Date: Mon, 3 Jun 1996 11:58:25 -0500 (CDT) Message-Id: <199606031658.LAA02117@cs.rice.edu> From: offner@hpc.pko.dec.com (Carl Offner) Subject: hpff-doc: Chapter 3 again; fixed typo Sender: owner-hpff-doc Precedence: bulk --------------------------------------------------------------------------- hpff-doc@cs.rice.edu is a mailing list for HPF 2.0 language specification authors and editors. Instructions for adding or deleting yourself from this list appear at the bottom of this message. --------------------------------------------------------------------------- Oops. There was a small typo in the version of Chapter 3 I just sent out. I've corrected it here. --Carl --------------------------------------------------------------- % File: mapping-subr.tex % Contents: % Mapping constructs for dummy arguments for HPF 2.0 document, % including % interface rules % INHERIT % the rest of pointer mappings % Revision history: % Jun-03-96 First edit by Carl Offner, Digital. % (Rearrangements, many edits, and two new sections; see % the comments at the top of the chapter.) % May-10-96 Created by Charles Koelbel, Rice University % (from HPF 1.2 document and HPF 2.0 proposals) % If you don't have LaTeX2e available, uncomment the next three lines: %\def\emph#1{{\em #1}} %\def\texttt#1{{\tt #1}} %\def\textit#1{{\it #1}} \chapter{Data Mapping in Subprogram Interfaces} \label{ch-mapping-subr} {\em Comments on this chpater should be directed to Piyush Mehrotra ({\tt pm@icase.edu}), Carl Offner ({\tt offner@hpc.pko.dec.com}), Guy Steele ({\tt Guy.Steele@east.sun.com}), and \\ {\tt hpff-doc@cs.rice.edu}. Please use ``{\tt Comments on Mapping in Subprogram Calls}'' as the {\tt Subject:} line. \par } \begin{new} \emph{This chapter has been rearranged. I have added comments on actual text changes (other than the most trivial ones). I also replaced ``Fortran 90'' by ``Fortran'' consistently except in the last paragraph, where I replaced it by ``Fortran 95'' for clarity. I deleted the section on explicit dynamic remapping of dummy arguments (i.e., the section dealing with \texttt{DYNAMIC}, \texttt{REALIGN}, and \texttt{REDISTRIBUTE}); this section belongs with the approved extensions. Sections~\ref{mapsub:introduction} and \ref{mapsub:ExplicitInterfaces} (including \ref{mapsub:SameMap}) are new.---C.O.} \end{new} \section{Introduction} \label{mapsub:introduction} \emph{This introduction gives an overview of the ways in which mapping directives interact with argument passing to subroutines. The language used here, however, is not definitive; the subsequent sections of this chapter contain the authoritative rules.} In addition to the data mapping features described in Chapter~\ref{ch-mapping-base}, HPF allows a number of options for describing the mapping of dummy arguments to subprograms. The mapping of each such dummy argument may be related to the mapping of its associated actual argument in the calling routine (the ``caller'') in several different ways. To allow for this, mapping directives applied to dummy arguments can have three different syntactic forms: \emph{prescriptive}, \emph{descriptive}, and \emph{transcriptive}. HPF provides these three forms to allow the programmer either to leave the data in place, or to force data into a new (presumably more efficient) mapping for the duration of the subprogram's execution. The meaning of these forms is as follows: \begin{description} \item[prescriptive] The directive describes the mapping of the dummy. However, the actual as passed may not have this mapping. \emph{If it does not}, it is the responsibility of the subprogram to remap the argument as specified, and to restore the original mapping on exit. (However, see the ``Advice to implementors'' below.) Prescriptive directives are syntactically identical to directives occurring elsewhere in the program. For instance, if \texttt{A} is a dummy argument, \CODE !HPF$ DISTRIBUTE A (BLOCK, CYCLIC) \EDOC is a prescriptive directive. \item[descriptive] The directive describes the mapping of the dummy. It is the responsibility of the calling routine to insure that the actual as passed has this mapping. (Similarly, remapping to restore the original mapping on exit is also done by the caller.) Descriptive directives look like prescriptive directives, except that an asterisk precedes the description. For instance, \CODE !HPF$ DISTRIBUTE A *(BLOCK, CYCLIC) \EDOC is a descriptive directive. \item[transcriptive] The mapping is unspecified. The called subroutine must accept the mapping of the argument as it is passed. (Of course this means that the caller must pass this mapping information at run-time.) Transcriptive directives are written with a single asterisk for distributions and processor arrays; for instance \CODE !HPF$ DISTRIBUTE A * \EDOC is a transcriptive directive. The \texttt{INHERIT} directive (see Section~\ref{INHERIT-SECTION}) is used to specify a transcriptive alignment. \end{description} Both distributions and processor arrangements can be specified to be prescriptive, descriptive, or transcriptive. Alignment is more complicated, because of the need to specify the template to which the dummy is aligned. This template may be unspecified (in this case of course there is no \texttt{ALIGN} directive), in which case it is the \emph{natural template} of the dummy. (``Natural template'' is defined in Section~\ref{mapsub:templates} below.) Otherwise, one of the following disjoint possibilities must be true: \begin{itemize} \item The template is explicitly specified by a prescriptive ALIGN directive. \item The template is explicitly specified by a descriptive ALIGN directive. \item The template is \emph{inherited}. This is specified by giving the dummy the \texttt{INHERIT} attribute (described in Section~\ref{INHERIT-SECTION} below). This implicitly specifies the template to be the template with which the corresponding actual argument is ultimately aligned; further, the alignment of the dummy with that template is the same as that of the corresponding actual. This is in effect a transcriptive form of alignment. \end{itemize} This is restated more precisely in Section~\ref{mapsub:templates} below. If remapping is necessary at a procedure interface, this fact must be visible in the caller. This is a consequence of the requirements for explicit interfaces in Section~\ref{mapsub:ExplicitInterfaces}. \begin{implementors} As a result of this, an implementation may choose to have all necessary remapping done by the caller. This in effect eliminates the distinction between prescriptive and descriptive mappings. \end{implementors} \begin{users} Although it is possible to write combinations of mapping directives that are partially prescriptive and partially transcriptive, for instance, there is probably no virtue to so doing. The point of these directives is to enable the compiler to handle any necessary remapping correctly and efficiently. Now remapping can happen for one or more of the following reasons: \begin{itemize} \item to make the alignment of the actual and the dummy agree; \item to make the distribution of the actual and the dummy agree; \item to make the processor array of the actual and the dummy agree. \end{itemize} For most machines, there is no real difference in the cost of remapping for any of these reasons. It is therefore a better practice in general to make a mapping either purely transcriptive, purely prescriptive, or purely descriptive. While transcriptive mappings can be useful in writing libraries, they impose a run-time cost on the subprogram. They should therefore be avoided in normal user code. \end{users} \section{What Remapping is Required, and Who Does It} \begin{new} \emph{This section has been rewritten. I added Chuck's suggested wording, even though my reading of CCI 50 indicates that no wording change was necessary. But it certainly doesn't do any harm.---C.O.} \end{new} If there is an explicit interface for the called subprogram and that interface contains mapping directives (whether prescriptive or descriptive) for the dummy argument in question, and if a remapping of the actual argument is necessary, the call should proceed as if the data was copied to a temporary variable to match the mapping of the dummy argument as expressed by the directives in the explicit interface. The template of the dummy will then be as declared in the interface. If in such a case, the mapping directives are descriptive, then it is the responsibility of the caller to perform the necessary remapping. That is, the caller must provide an actual argument matching such directives, so that the descriptive directives correctly describe the dummy arguments to the subprogram. \begin{implementors} In fact, an implementation may choose to have \emph{all} necessary remapping done by the caller. \end{implementors} If there is no explicit interface, then no remapping will be necessary; this is a consequence of the requirements in Section~\ref{mapsub:ExplicitInterfaces}. \begin{new} \emph{These next sentences were taken from the section on \texttt{DYNAMIC} remappings, which is otherwise moved to the part of the document containing the approved extensions.---C.O.} \end{new} An overriding principle is that \emph{any mapping or remapping of arguments is not visible to the caller}. That is, when the subprogram returns and the caller resumes execution, all objects accessible to the caller after the call are mapped exactly as they were before the call. It is not possible for a subprogram to change the mapping of any object in a manner visible to its caller. \section{Distributions and Processor Arrangements} \label{mapsub:DistProcArr} In a \texttt{DISTRIBUTE} directive where every \textit{distributee} is a dummy argument, either the \textit{dist-format-clause} or the \textit{dist-target}, or both, may begin with, or consist of, an asterisk. \begin{itemize} \item Without an asterisk, a \textit{dist-format-clause} or \textit{dist-target} is prescriptive; the clause describes a distribution and constitutes a request of the language processor to make it so. This might entail the subprogram remapping or copying the actual argument on entry at run time in order to satisfy the requested distribution for the dummy. \begin{new} \emph{I added a couple of references to the ``subprogram'' in the previous and next items for clarity.---C.O.} \end{new} \item Starting with an asterisk, a \textit{dist-format-clause} or \textit{dist-target} is descriptive; the clause describes a distribution and constitutes an assertion to the language processor that on entry to the subprogram it will already be so. The programmer claims that, for every call to the subprogram, the actual argument will be such that the stated distribution already describes the mapping of that data. (The intent is that if the argument is passed by reference, no movement of the data by the subprogram will be necessary at run time. All this is under the assumption that the language processor has observed all other directives. While a conforming HPF language processor is not required to obey mapping directives, it should handle descriptive directives with the understanding that their implied assertions are relative to this assumption.) \begin{new} \emph{I deleted ``whether explicit or implicit'' in the last sentence of the following item, since there is no more implicit transcriptive behavior.---C.O.} \end{new} \item Consisting of only an asterisk, a \textit{dist-format-clause} or \textit{dist-target} is transcriptive; the clause says nothing about the distribution but constitutes a request of the language processor to copy that aspect of the distribution from that of the actual argument. (The intent is that if the argument is passed by reference, no movement of the data will be necessary at run time.) Note that the transcriptive case is not included in Subset HPF. \end{itemize} It is possible that, in a single \texttt{DISTRIBUTE} directive, the \textit{dist-format-clause} might have an asterisk but not the \textit{dist-target}, or vice versa. \subsection{Examples} These examples of \texttt{DISTRIBUTE} directives for dummy arguments illustrate the various combinations: \CODE !HPF$ DISTRIBUTE URANIA (CYCLIC) ONTO GALILEO \EDOC The language processor should do whatever it takes to cause \texttt{URANIA} to have a \texttt{CYCLIC} distribution on the processor arrangement \texttt{GALILEO}. \CODE !HPF$ DISTRIBUTE POLYHYMNIA * ONTO ELVIS \EDOC The language processor should do whatever it takes to cause \texttt{POLYHYMNIA} to be distributed onto the processor arrangement \texttt{ELVIS}, using whatever distribution format it currently has (which might be on some other processor arrangement). (You can't say this in Subset HPF.) \CODE !HPF$ DISTRIBUTE THALIA *(CYCLIC) ONTO FLIP \EDOC The language processor should do whatever it takes to cause \texttt{THALIA} to have a \texttt{CYCLIC} distribution on the processor arrangement \texttt{FLIP}; \texttt{THALIA} already has a cyclic distribution, though it might be on some other processor arrangement. \CODE !HPF$ DISTRIBUTE CALLIOPE (CYCLIC) ONTO *HOMER \EDOC The language processor should do whatever it takes to cause \texttt{CALLIOPE} to have a \texttt{CYCLIC} distribution on the processor arrangement \texttt{HOMER}; \texttt{CALLIOPE} is already distributed onto \texttt{HOMER}, though it might be with some other distribution format. \CODE !HPF$ DISTRIBUTE MELPOMENE * ONTO *EURIPIDES \EDOC \texttt{MELPOMENE} is asserted to already be distributed onto \texttt{EURIPIDES}; use whatever distribution format the actual argument had so, if possible, no data movement should occur. (You can't say this in Subset HPF.) \CODE !HPF$ DISTRIBUTE CLIO *(CYCLIC) ONTO *HERODOTUS \EDOC \texttt{CLIO} is asserted to already be distributed \texttt{CYCLIC} onto \texttt{HERODOTUS} so, if possible, no data movement should occur. \CODE !HPF$ DISTRIBUTE EUTERPE (CYCLIC) ONTO * \EDOC The language processor should do whatever it takes to cause \texttt{EUTERPE} to have a \texttt{CYCLIC} distribution onto whatever processor arrangement the actual was distributed onto. (You can't say this in Subset HPF.) \CODE !HPF$ DISTRIBUTE ERATO * ONTO * \EDOC The mapping of \texttt{ERATO} should not be changed from that of the actual argument. (You can't say this in Subset HPF.) \CODE !HPF$ DISTRIBUTE ARTHUR_MURRAY *(CYCLIC) ONTO * \EDOC \texttt{ARTHUR_MURRAY} is asserted to already be distributed \texttt{CYCLIC} onto whatever processor arrangement the actual argument was distributed onto, and no data movement should occur. (You can't say this in Subset HPF.) Please note that \texttt{DISTRIBUTE ERATO * ONTO *} does not mean the same thing as \CODE !HPF$ DISTRIBUTE ERATO *(*) ONTO * \EDOC This latter means: \texttt{ERATO} is asserted to already be distributed \texttt{*} (that is, on-processor) onto whatever processor arrangement the actual was distributed onto. Note that the processor arrangement is necessarily scalar in this case. GUY - CCI 30 SAYS THERE THERE SHOULD BE A NOTE THAT `THIS' ALSO APPLIES TO LOCAL VARIABLES. IT ISN'T CLEAR WHAT `THIS' IS OR HOW IT ANSWERS THE ORIGINAL CCI QUESTION - SHOULD IT BE SOMETHING LIKE, `LOCAL VARIABLES ALIGNED WITH ARGUMENTS IN A SIMILAR WAY WILL ALSO HAVE A SCALAR PROCESSOR ARRANGEMENT'? \subsection{What Happens When a Clause is Omitted} One may omit either the \textit{dist-format-clause} or the \textit{dist-onto-clause} for a dummy argument. If such a clause is omitted and the dummy argument has the \texttt{INHERIT} attribute (see Section~\ref{INHERIT-SECTION}), then the compiler must handle the directive as if \texttt{*} or \texttt{ONTO~*} had been specified explicitly. If such a clause is omitted and the dummy does not have the \texttt{INHERIT} attribute, then the compiler may choose the distribution format or a target processor arrangement arbitrarily. Here are two examples: \CODE !HPF$ DISTRIBUTE WHEEL_OF_FORTUNE *(CYCLIC) \EDOC \texttt{WHEEL_OF_FORTUNE} is asserted to already be \texttt{CYCLIC}. As long as it is kept \texttt{CYCLIC}, it may be remapped onto some other processor arrangement, but there is no reason to. \CODE !HPF$ DISTRIBUTE ONTO *TV :: DAVID_LETTERMAN \EDOC \texttt{DAVID_LETTERMAN} is asserted to already be distributed on \texttt{TV} in some fashion. The distribution format may be changed as long as \texttt{DAVID_LETTERMAN} is kept on \texttt{TV}. (Note that this declaration must be made in attributed form; the statement form \CODE !HPF$ DISTRIBUTE DAVID_LETTERMAN ONTO *TV !Nonconforming \EDOC does not conform to the syntax for a \texttt{DISTRIBUTE} directive.) \section{Alignment} \subsection{The Template of the Dummy Argument} \label{mapsub:templates} Here we describe precisely how the template with which the dummy argument is ultimately aligned is arrived at. \begin{new} \emph{This next paragraph was lifted from the TEMPLATE section of the mapping-base.tex file, and should be deleted from it.} \end{new} First, templates are not passed through the subprogram argument interface. The template to which a dummy argument is aligned is always distinct from the template to which the actual argument is aligned, though it may be a copy (see Section \ref{INHERIT-SECTION}). On exit from a subprogram, an HPF implementation arranges that the actual argument is aligned with the same template with which it was aligned before the call. Thus, a dummy argument always has a fresh template to which it is ultimately aligned. This template is constructed in one of three ways: \begin{itemize} \item If the dummy argument appears explicitly as an \textit{alignee} in an \texttt{ALIGN} directive, its template is specified by the \textit{align-target}. \item If the dummy argument is not explicitly aligned and does not have the \texttt{INHERIT} attribute (described in Section~\ref{INHERIT-SECTION} below), then the template has the same shape and bounds as the dummy argument; this is called the \textit{natural template} for the dummy. (Thus, all the examples in Section~\ref{mapsub:DistProcArr} use the natural template.) \item If the dummy argument is not explicitly aligned and does have the \texttt{INHERIT} attribute, then the template is ``inherited'' from the actual argument according to the following rules: \begin{itemize} \item If the actual argument is a whole array, the template of the dummy is a copy of the template with which the actual argument is ultimately aligned. \item If the actual argument is an array section of array \(A\) where no subscript is a vector subscript, then the template of the dummy is a copy of the template with which \(A\) is ultimately aligned. \item If the actual argument is any other expression, the shape and distribution of the template may be chosen arbitrarily by the language processor (and therefore the programmer cannot know anything \textit{a priori} about its distribution). \end{itemize} In all of these cases, we say that the dummy has an \textit{inherited template} rather than a natural template. \end{itemize} \subsection{The INHERIT Directive} \label{INHERIT-SECTION} The \texttt{INHERIT} directive specifies that a dummy argument should be aligned to a copy of the template of the corresponding actual argument in the same way that the actual argument is aligned. \BNF inherit-directive \IS INHERIT dummy-argument-name-list \FNB The \texttt{INHERIT} directive causes the named subprogram dummy arguments to have the \texttt{INHERIT} attribute. Only dummy arguments may have the \texttt{INHERIT} attribute. An object must not have both the \texttt{INHERIT} attribute and the \texttt{ALIGN} attribute. The \texttt{INHERIT} directive may appear only in a \textit{specification-part} of a scoping unit. If a dummy argument has the \texttt{TARGET} attribute and no explicit mapping attributes, then the \texttt{INHERIT} attribute is implicitly assumed. (See section~\ref{mapsub:pointers}.) \begin{new} \emph{Some sentences from the old 3.10 were inserted in the next paragraph (which was also split into two paragraphs), and the corresponding section was deleted from the discussion of distributions.---C.O.} \end{new} The \texttt{INHERIT} attribute specifies that the template for a dummy argument should be inherited, by making a copy of the template of the actual argument. Moreover, the \texttt{INHERIT} attribute implies a default distribution of \texttt{DISTRIBUTE~*~ONTO~*}. In such a case, the net effect is to tell the compiler to leave the data exactly where it is---and not attempt to remap the actual argument. The dummy argument will be mapped in exactly the same manner as the actual argument; the subprogram must be compiled in such a way as to work correctly no matter how the actual argument may be mapped onto abstract processors. If on the other hand an explicit mapping directive appears for a dummy argument with the \texttt{INHERIT} attribute, thereby overriding the default distribution, then the actual argument must be a whole array or an array section with no vector subscripts; it may not be an expression of any other form. \begin{new} \emph{The next paragraph is not needed; it just repeats the definition of natural template given in the previous section.---C.O.} \end{new} \begin{obsolete} If none of the attributes \texttt{INHERIT}, \texttt{ALIGN}, and \texttt{DISTRIBUTE} is specified explicitly for a dummy argument, then the template of the dummy argument has the same shape as the dummy itself and the dummy argument is aligned to its template by the identity mapping. \end{obsolete} An \texttt{INHERIT} directive may be combined with other directives, with the attributes stated in any order, more or less consistent with Fortran attribute syntax. \subsection{Examples} Here is a straightforward example of the use of \texttt{INHERIT}: \CODE REAL DOUGH(100) !HPF$ DISTRIBUTE DOUGH(BLOCK(10)) CALL PROBATE( DOUGH(7:23:2) ) ... SUBROUTINE PROBATE(BREAD) REAL BREAD(9) !HPF$ INHERIT BREAD \EDOC \begin{new} \emph{I added ``or descriptive'' in the next sentence.---C.O.} \end{new} The inherited template of \texttt{BREAD} has shape [100]; element \texttt{BREAD(I)} is aligned with element 5 + 2*I of the inherited template and, since \texttt{BREAD} does not appear in a prescriptive or descriptive \texttt{DISTRIBUTE} directive, it has a \texttt{BLOCK(10)} distribution. Here is a more complex example: \CODE LOGICAL FRUG(128),TWIST(128) !HPF$ PROCESSORS DANCE_FLOOR(16) !HPF$ DISTRIBUTE (BLOCK) ONTO DANCE_FLOOR::FRUG,TWIST CALL TERPSICHORE(FRUG(1:40:3),TWIST(1:40:3)) \EDOC The two array sections \texttt{FRUG(1:40:3)} and \texttt{TWIST(1:40:3)} are mapped onto abstract processors in the same manner: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,225)(0,0) \put(0,200){\makebox(35,25){\small\rm 1}} \put(35,200){\makebox(35,25){\small\rm 2}} \put(70,200){\makebox(35,25){\small\rm 3}} \put(105,200){\makebox(35,25){\small\rm 4}} \put(140,200){\makebox(35,25){\small\rm 5}} \put(175,200){\makebox(35,25){\small\rm 6}} \put(210,200){\makebox(35,25){\small\rm 7}} \put(245,200){\makebox(35,25){\small\rm 8}} \put(280,200){\makebox(35,25){\small\rm 9}} \put(315,200){\makebox(35,25){\small\rm 10}} \put(350,200){\makebox(35,25){\small\rm 11}} \put(385,200){\makebox(35,25){\small\rm 12}} \put(420,200){\makebox(35,25){\small\rm 13}} \put(455,200){\makebox(35,25){\small\rm 14}} \put(490,200){\makebox(35,25){\small\rm 15}} \put(525,200){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){7}{\line(1,0){560}} \thicklines \multiput(0,0)(0,200){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){200}} \put(0,175){\makebox(35,25){\tt 1}} \put(0,100){\makebox(35,25){\tt 4}} \put(0,25){\makebox(35,25){\tt 7}} \put(35,150){\makebox(35,25){\tt 10}} \put(35,75){\makebox(35,25){\tt 13}} \put(35,0){\makebox(35,25){\tt 16}} \put(70,125){\makebox(35,25){\tt 19}} \put(70,50){\makebox(35,25){\tt 22}} \put(105,175){\makebox(35,25){\tt 25}} \put(105,100){\makebox(35,25){\tt 28}} \put(105,25){\makebox(35,25){\tt 31}} \put(140,150){\makebox(35,25){\tt 34}} \put(140,75){\makebox(35,25){\tt 37}} \put(140,0){\makebox(35,25){\tt 40}} \end{picture} \end{center} However, the subroutine \texttt{TERPSICHORE} will view them in different ways because it inherits the template for the second dummy but not the first: \CODE SUBROUTINE TERPSICHORE(FOXTROT,TANGO) LOGICAL FOXTROT(:),TANGO(:) !HPF$ INHERIT TANGO \EDOC Therefore the template of \texttt{TANGO} is a copy of the 128 element template of the whole array \texttt{TWIST}. The template is mapped like this: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,225)(0,0) \put(0,200){\makebox(35,25){\small\rm 1}} \put(35,200){\makebox(35,25){\small\rm 2}} \put(70,200){\makebox(35,25){\small\rm 3}} \put(105,200){\makebox(35,25){\small\rm 4}} \put(140,200){\makebox(35,25){\small\rm 5}} \put(175,200){\makebox(35,25){\small\rm 6}} \put(210,200){\makebox(35,25){\small\rm 7}} \put(245,200){\makebox(35,25){\small\rm 8}} \put(280,200){\makebox(35,25){\small\rm 9}} \put(315,200){\makebox(35,25){\small\rm 10}} \put(350,200){\makebox(35,25){\small\rm 11}} \put(385,200){\makebox(35,25){\small\rm 12}} \put(420,200){\makebox(35,25){\small\rm 13}} \put(455,200){\makebox(35,25){\small\rm 14}} \put(490,200){\makebox(35,25){\small\rm 15}} \put(525,200){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){7}{\line(1,0){560}} \thicklines \multiput(0,0)(0,200){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){200}} \put(0,175){\makebox(35,25){\tt 1}} \put(0,150){\makebox(35,25){\tt 2}} \put(0,125){\makebox(35,25){\tt 3}} \put(0,100){\makebox(35,25){\tt 4}} \put(0,75){\makebox(35,25){\tt 5}} \put(0,50){\makebox(35,25){\tt 6}} \put(0,25){\makebox(35,25){\tt 7}} \put(0,0){\makebox(35,25){\tt 8}} \put(35,175){\makebox(35,25){\tt 9}} \put(35,150){\makebox(35,25){\tt 10}} \put(35,125){\makebox(35,25){\tt 11}} \put(35,100){\makebox(35,25){\tt 12}} \put(35,75){\makebox(35,25){\tt 13}} \put(35,50){\makebox(35,25){\tt 14}} \put(35,25){\makebox(35,25){\tt 15}} \put(35,0){\makebox(35,25){\tt 16}} \put(70,175){\makebox(35,25){\tt 17}} \put(70,150){\makebox(35,25){\tt 18}} \put(70,125){\makebox(35,25){\tt 19}} \put(70,100){\makebox(35,25){\tt 20}} \put(70,75){\makebox(35,25){\tt 21}} \put(70,50){\makebox(35,25){\tt 22}} \put(70,25){\makebox(35,25){\tt 23}} \put(70,0){\makebox(35,25){\tt 24}} \put(105,175){\makebox(35,25){\tt 25}} \put(105,150){\makebox(35,25){\tt 26}} \put(105,125){\makebox(35,25){\tt 27}} \put(105,100){\makebox(35,25){\tt 28}} \put(105,75){\makebox(35,25){\tt 29}} \put(105,50){\makebox(35,25){\tt 30}} \put(105,25){\makebox(35,25){\tt 31}} \put(105,0){\makebox(35,25){\tt 32}} \put(140,175){\makebox(35,25){\tt 33}} \put(140,150){\makebox(35,25){\tt 34}} \put(140,125){\makebox(35,25){\tt 35}} \put(140,100){\makebox(35,25){\tt 36}} \put(140,75){\makebox(35,25){\tt 37}} \put(140,50){\makebox(35,25){\tt 38}} \put(140,25){\makebox(35,25){\tt 39}} \put(140,0){\makebox(35,25){\tt 40}} \put(175,175){\makebox(35,25){\tt 41}} \put(175,150){\makebox(35,25){\tt 42}} \put(175,125){\makebox(35,25){\tt 43}} \put(175,100){\makebox(35,25){\tt 44}} \put(175,75){\makebox(35,25){\tt 45}} \put(175,50){\makebox(35,25){\tt 46}} \put(175,25){\makebox(35,25){\tt 47}} \put(175,0){\makebox(35,25){\tt 48}} \put(210,175){\makebox(35,25){\tt 49}} \put(210,150){\makebox(35,25){\tt 50}} \put(210,125){\makebox(35,25){\tt 51}} \put(210,100){\makebox(35,25){\tt 52}} \put(210,75){\makebox(35,25){\tt 53}} \put(210,50){\makebox(35,25){\tt 54}} \put(210,25){\makebox(35,25){\tt 55}} \put(210,0){\makebox(35,25){\tt 56}} \put(245,175){\makebox(35,25){\tt 57}} \put(245,150){\makebox(35,25){\tt 58}} \put(245,125){\makebox(35,25){\tt 59}} \put(245,100){\makebox(35,25){\tt 60}} \put(245,75){\makebox(35,25){\tt 61}} \put(245,50){\makebox(35,25){\tt 62}} \put(245,25){\makebox(35,25){\tt 63}} \put(245,0){\makebox(35,25){\tt 64}} \put(280,175){\makebox(35,25){\tt 65}} \put(280,150){\makebox(35,25){\tt 66}} \put(280,125){\makebox(35,25){\tt 67}} \put(280,100){\makebox(35,25){\tt 68}} \put(280,75){\makebox(35,25){\tt 69}} \put(280,50){\makebox(35,25){\tt 70}} \put(280,25){\makebox(35,25){\tt 71}} \put(280,0){\makebox(35,25){\tt 72}} \put(315,175){\makebox(35,25){\tt 73}} \put(315,150){\makebox(35,25){\tt 74}} \put(315,125){\makebox(35,25){\tt 75}} \put(315,100){\makebox(35,25){\tt 76}} \put(315,75){\makebox(35,25){\tt 77}} \put(315,50){\makebox(35,25){\tt 78}} \put(315,25){\makebox(35,25){\tt 79}} \put(315,0){\makebox(35,25){\tt 80}} \put(350,175){\makebox(35,25){\tt 81}} \put(350,150){\makebox(35,25){\tt 82}} \put(350,125){\makebox(35,25){\tt 83}} \put(350,100){\makebox(35,25){\tt 84}} \put(350,75){\makebox(35,25){\tt 85}} \put(350,50){\makebox(35,25){\tt 86}} \put(350,25){\makebox(35,25){\tt 87}} \put(350,0){\makebox(35,25){\tt 88}} \put(385,175){\makebox(35,25){\tt 89}} \put(385,150){\makebox(35,25){\tt 90}} \put(385,125){\makebox(35,25){\tt 91}} \put(385,100){\makebox(35,25){\tt 92}} \put(385,75){\makebox(35,25){\tt 93}} \put(385,50){\makebox(35,25){\tt 94}} \put(385,25){\makebox(35,25){\tt 95}} \put(385,0){\makebox(35,25){\tt 96}} \put(420,175){\makebox(35,25){\tt 97}} \put(420,150){\makebox(35,25){\tt 98}} \put(420,125){\makebox(35,25){\tt 99}} \put(420,100){\makebox(35,25){\tt 100}} \put(420,75){\makebox(35,25){\tt 101}} \put(420,50){\makebox(35,25){\tt 102}} \put(420,25){\makebox(35,25){\tt 103}} \put(420,0){\makebox(35,25){\tt 104}} \put(455,175){\makebox(35,25){\tt 105}} \put(455,150){\makebox(35,25){\tt 106}} \put(455,125){\makebox(35,25){\tt 107}} \put(455,100){\makebox(35,25){\tt 108}} \put(455,75){\makebox(35,25){\tt 109}} \put(455,50){\makebox(35,25){\tt 110}} \put(455,25){\makebox(35,25){\tt 111}} \put(455,0){\makebox(35,25){\tt 112}} \put(490,175){\makebox(35,25){\tt 113}} \put(490,150){\makebox(35,25){\tt 114}} \put(490,125){\makebox(35,25){\tt 115}} \put(490,100){\makebox(35,25){\tt 116}} \put(490,75){\makebox(35,25){\tt 117}} \put(490,50){\makebox(35,25){\tt 118}} \put(490,25){\makebox(35,25){\tt 119}} \put(490,0){\makebox(35,25){\tt 120}} \put(525,175){\makebox(35,25){\tt 121}} \put(525,150){\makebox(35,25){\tt 122}} \put(525,125){\makebox(35,25){\tt 123}} \put(525,100){\makebox(35,25){\tt 124}} \put(525,75){\makebox(35,25){\tt 125}} \put(525,50){\makebox(35,25){\tt 126}} \put(525,25){\makebox(35,25){\tt 127}} \put(525,0){\makebox(35,25){\tt 128}} \end{picture} \end{center} \noindent \texttt{TANGO(I)} is aligned with element 3*I-2 of the template. But the template of \texttt{FOXTROT} has the same size 14 as \texttt{FOXTROT} itself. The actual argument, \texttt{FRUG(1:40:3)} is mapped to the 16 processors in this manner: \begin{center} \begin{tabular}{cc} Abstract & Elements \\ processor & of FRUG \\ 1 & 1, 2, 3 \\ 2 & 4, 5, 6 \\ 3 & 7, 8 \\ 4 & 9, 10, 11 \\ 5 & 12, 13, 14 \\ 6--16 & none \end{tabular} \end{center} It would be reasonable to understand the mapping of the template of \texttt{FOXTROT} to coincide with the layout of the array section: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,225)(0,0) \put(0,200){\makebox(35,25){\small\rm 1}} \put(35,200){\makebox(35,25){\small\rm 2}} \put(70,200){\makebox(35,25){\small\rm 3}} \put(105,200){\makebox(35,25){\small\rm 4}} \put(140,200){\makebox(35,25){\small\rm 5}} \put(175,200){\makebox(35,25){\small\rm 6}} \put(210,200){\makebox(35,25){\small\rm 7}} \put(245,200){\makebox(35,25){\small\rm 8}} \put(280,200){\makebox(35,25){\small\rm 9}} \put(315,200){\makebox(35,25){\small\rm 10}} \put(350,200){\makebox(35,25){\small\rm 11}} \put(385,200){\makebox(35,25){\small\rm 12}} \put(420,200){\makebox(35,25){\small\rm 13}} \put(455,200){\makebox(35,25){\small\rm 14}} \put(490,200){\makebox(35,25){\small\rm 15}} \put(525,200){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){7}{\line(1,0){560}} \thicklines \multiput(0,0)(0,200){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){200}} \put(0,175){\makebox(35,25){\tt 1}} \put(0,100){\makebox(35,25){\tt 2}} \put(0,25){\makebox(35,25){\tt 3}} \put(35,150){\makebox(35,25){\tt 4}} \put(35,75){\makebox(35,25){\tt 5}} \put(35,0){\makebox(35,25){\tt 6}} \put(70,125){\makebox(35,25){\tt 7}} \put(70,50){\makebox(35,25){\tt 8}} \put(105,175){\makebox(35,25){\tt 9}} \put(105,100){\makebox(35,25){\tt 10}} \put(105,25){\makebox(35,25){\tt 11}} \put(140,150){\makebox(35,25){\tt 12}} \put(140,75){\makebox(35,25){\tt 13}} \put(140,0){\makebox(35,25){\tt 14}} \end{picture} \end{center} \noindent but we shall see that this is not permitted in HPF. Within subroutine \texttt{TERPSICHORE} it would be correct to make the descriptive assertion \CODE !HPF$ DISTRIBUTE TANGO *(BLOCK) \EDOC but it would not be correct to declare \CODE !HPF$ DISTRIBUTE FOXTROT *(BLOCK) !Nonconforming \EDOC Each of these asserts that the template of the specified dummy argument is already distributed \texttt{BLOCK} on entry to the subroutine. The shape of the template for \texttt{TANGO} is \texttt{[128]}, inherited (copied) from the array \texttt{TWIST}, whose section was passed as the corresponding actual argument, and that template does indeed have a \texttt{BLOCK} distribution. But the shape of the template for \texttt{FOXTROT} is \texttt{[14]}; the layout of the elements of the actual argument \texttt{FRUG(1:40:3)} (3 on the first processor, 3 on the second processor, 2 on the third processor, 3 on the fourth processor, \dots) cannot properly be described as a \texttt{BLOCK} distribution of a length-14 template, so the \texttt{DISTRIBUTE} declaration for \texttt{FOXTROT} shown above would indeed be erroneous. On the other hand, the layout of \texttt{FRUG(1:40:3)} can be described in terms of an alignment to a length-128 template which can be described by an explicit \texttt{TEMPLATE} declaration (see Section \ref{TEMPLATE-SECTION}), so the directives \CODE !HPF$ PROCESSORS DANCE_FLOOR(16) !HPF$ TEMPLATE, DISTRIBUTE(BLOCK) ONTO DANCE_FLOOR::GURF(128) !HPF$ ALIGN FOXTROT(J) WITH *GURF(3*J-2) \EDOC could be correctly included in \texttt{TERPSICHORE} to describe the layout of \texttt{FOXTROT} on entry to the subroutine without using an inherited template. \subsection{ALIGN Directives} The presence or absence of an asterisk at the start of an \textit{align-spec} has the same meaning as in a \textit{dist-format-clause}: it specifies whether the \texttt{ALIGN} directive is descriptive or prescriptive, respectively. If an \textit{align-spec} that does not begin with \texttt{*} is applied to a dummy argument, the meaning is that the dummy argument will be forced to have the specified alignment on entry to the subprogram. This may require either the caller or the subprogram to temporarily remap the data of the actual argument or a copy thereof. Note that a dummy argument may also be used as an \textit{align-target}. \CODE SUBROUTINE NICHOLAS(TSAR,CZAR) REAL, DIMENSION(1918) :: TSAR,CZAR !HPF$ INHERIT :: TSAR !HPF$ ALIGN WITH TSAR :: CZAR \EDOC In this example the first dummy argument, \texttt{TSAR}, is allowed to remain aligned with the corresponding actual argument, while the second dummy argument, \texttt{CZAR}, is forced to be aligned with the first dummy argument. If the two actual arguments are already aligned, no remapping of the data will be required at run time. If they are not, some remapping will take place. \begin{new} \emph{I took out the phrase ``and not \texttt{REALIGN}'' at the end of the first sentence of the next paragraph, since \texttt{REALIGN} now belongs with the approved extensions.---C.O.} \end{new} If the \textit{align-spec} begins with ``\texttt{*}'', then the \textit{alignee} must be a dummy argument and the directive must be \texttt{ALIGN}. The ``\texttt{*}'' indicates that the \texttt{ALIGN} directive constitutes a guarantee on the part of the programmer that, on entry to the subprogram, the indicated alignment will already be satisfied by the dummy argument, without any action to remap it required (on the part of the subprogram) at run time. For example: \CODE SUBROUTINE GRUNGE(PLUNGE,SPONGE) REAL PLUNGE(1000),SPONGE(1000) !HPF$ INHERIT SPONGE !HPF$ ALIGN PLUNGE WITH *SPONGE \EDOC This asserts that, for every \texttt{J} in the range \texttt{1:1000}, on entry to subroutine \texttt{GRUNGE}, the directives in the program have specified that \texttt{PLUNGE(J)} is currently mapped to the same abstract processor as \texttt{SPONGE(J)}. (The intent is that if the language processor has in fact honored the directives, then no interprocessor communication on the part of the subprogram will be required to achieve the specified alignment.) The alignment of a general expression is up to the language processor and therefore unpredictable by the programmer; but the alignment of whole arrays and array sections is predictable. In the code fragment \CODE REAL FIJI(5000),SQUEEGEE(2000) !HPF$ ALIGN SQUEEGEE(K) WITH FIJI(2*K) CALL GRUNGE(FIJI(2002:4000:2),SQUEEGEE(1001:)) \EDOC it is true that every element of the array section \texttt{SQUEEGEE(1001:)} is aligned with the corresponding element of the array section \texttt{FIJI(2002:4000:2)}, so the claim made in subroutine \texttt{GRUNGE} is satisfied by this particular call. Under certain circumstances, it may be possible to specify that one dummy argument be remapped if necessary and then to specify that another dummy will then be aligned with it: \CODE SUBROUTINE MURKY(THINK, DENSE) !HPF$ PROCESSORS GUNK(32) !HPF$ DISTRIBUTE (BLOCK) ONTO GUNK :: DENSE !HPF$ ALIGN WITH *DENSE :: THICK \EDOC Note that the programmer cannot be justified in descriptively asserting that \texttt{THICK} will be aligned with \texttt{DENSE} after its remapping unless the remapping is fully specified (that is, no part of the remapping is left to the compiler to choose). Therefore an explicit processors arrangement necessarily appears in the example. The caller must ensure that the first actual argument is appropriately mapped onto an identical processors arrangement. It is not permitted to say simply ``\texttt{ALIGN WITH *}''; an \textit{align-target} must follow the asterisk. (The proper way to say ``accept any alignment'' is \texttt{INHERIT}.) If a dummy argument has no explicit \texttt{ALIGN} or \texttt{DISTRIBUTE} attribute, then the compiler provides an implicit alignment and distribution specification, one that could have been described explicitly without any ``assertion asterisks''. \section{Explicit Interfaces} \label{mapsub:ExplicitInterfaces} An explicit interface is required \emph{except} when all four of the following conditions hold: \begin{enumerate} \item Fortran does not require one, \emph{and} \item No dummy argument is passed transcriptively or with the \texttt{INHERIT} attribute, \emph{and} \item Either of the following cases applies to each pair of corresponding actual and dummy arguments: \begin{enumerate} \item They are both implicitly mapped. \item They are both explicitly mapped and the mappings are the same. Section~\ref{mapsub:SameMap} below defines what ``the mappings are the same'' means in this context. \end{enumerate} \emph{and} \item Either of the following cases applies to each pair of corresponding actual and dummy arguments: \begin{enumerate} \item Both are sequential. \item Both are nonsequential. \end{enumerate} \end{enumerate} \begin{rationale} This has the following consequences: \begin{itemize} \item A plain Fortran program (i.e., with no HPF directives) will continue to be legal without the need to add additional interfaces. This is insured by items 1, 2, 3a, and 4a. \item If remapping is necessary, this fact will be visible to the caller. Thus the implementation may choose to have all remapping performed by the caller. \end{itemize} \end{rationale} \begin{users} This requirement pushes the user strongly in the direction of always providing explicit interfaces. This is a good thing---explicit interfaces allow many errors to be caught at compile-time and greatly speed up the process of robust software development. Note, that an explicit interface can be provided in three ways: \begin{enumerate} \item by using a module. \item by a contained subprogram; \item by an interface block; \end{enumerate} In addition, an intrinsic procedure always has an explicit interface by definition. The idiomatic Fortran way of programming makes extensive use of modules; every subroutine, for instance, can be in a module. This provides explicit interfaces automatically, with no extra effort on the part of the programmer. It should very seldom be necessary to write an interface block. \end{users} \subsection{Explicit Mappings Without an Explicit Interface} \label{mapsub:SameMap} When there is no explicit interface, the actual and dummy arguments may still be explicitly mapped (but not transcriptively or with the \texttt{INHERIT} attribute), as long as the mappings are the same. This must be true in a very strict sense; in particular, the mappings must be the same whatever configuration of processors is used, and whatever system or user defaults may be in force. We give here the only circumstances under which this may be said to be true. First note that since only named objects can be mapped, the actual argument must be a whole array or array section. For the purposes of the conditions below, we shall regard a whole array as a particular kind of array section; i.e., the section each of whose subscripts is \texttt{:}. Denote the dummy argument by D, the actual argument by A, and the array of which the actual argument is a section by AA (AA might be the same as A). Denote the templates of the dummy and the actual by TD and TA, respectively. Denote the processor arrangements onto which TD and TA are distributed by PD and PA, respectively. PD and PA must have the same shape. Each axis of TD that is not distributed \texttt{*} must have the same extent and the same distribution format as the corresponding axis of TA, and conversely. In such a case, the axis of PD onto which the axis of TD is distributed must have the same extent as the axis of PA onto which the axis of TA is distributed. One of the two following cases must hold: \begin{enumerate} \item \label{mapsub:MapSameExpTmp} The dummy is described using an explicit named template (using a \texttt{TEMPLATE} directive); an example of this (\texttt{GURF}) appears in Section~\ref{INHERIT-SECTION}. (Actually, in that example, an explicit interface would be required, but only because of the other argument, which is inherited.) In such a case, for each \textit{section-subscript} of A, the following must hold: \begin{itemize} \item If the \textit{section-subscript} is scalar and the array axis is collapsed (as by an \texttt{ALIGN} directive) then no entry may appear in the distribution for TD. \item If the \textit{section-subscript} is scalar and the array axis corresponds to an axis of the template TA distributed \texttt{*}, then no entry may appear in the distribution for TD. \item If the \textit{section-subscript} is a scalar and the array axis corresponds to an axis of the template TA that is not distributed \texttt{*}, then the position on the axis of TA to which the scalar is aligned must correspond to the position on the corresponding axis of TD to which D is aligned. \item If the \textit{section-subscript} is a \textit{subscript-triplet} and the array axis is collapsed (as by an ALIGN directive), then this case can be treated \emph{as if} a corresponding axis, having the same extent as the axis of AA, were added to TA, distributed \texttt{*}; the next item in this list then applies. \item If the \textit{section-subscript} is a \textit{subscript-triplet} and the array axis corresponds to an axis of the template TA, then the distribution format of the axis of TA must be the same as the distribution format of the corresponding axis of TD, and corresponding elements of A along its axis and D along its axis must align to corresponding positions on the axes of TA and TD, respectively. \end{itemize} \item Otherwise, the dummy argument's template TD is its natural template. In this case the template TA of the actual must be coextensive with the array AA along any axes having a distribution format other than ``\texttt{*}.'' In such a case, for each \textit{section-subscript} of A, the following must hold: \begin{itemize} \item If the \textit{section-subscript} is scalar and the array axis is collapsed (as by an \texttt{ALIGN} directive) then no entry may appear in the distribution for TD. \item If the \textit{section-subscript} is scalar and the array axis corresponds to an axis of TA distributed \texttt{*}, then no entry may appear in the distribution for the TD. \item If the \textit{section-subscript} is a \textit{subscript-triplet} and the array axis is collapsed (as by an \texttt{ALIGN} directive), then \texttt{*} must appear in the distribution for TD. Further, the \textit{stride} of the \textit{subscript-triplet} must be 1. \item If the \textit{section-subscript} is a \textit{subscript-triplet} and the array axis corresponds to an axis of TA distributed \texttt{*}, then \texttt{*} must appear in the distribution for TD. Further, consecutive elements of the actual argument A along the array axis must be aligned with consecutive elements of TA along its corresponding axis. \item If the \textit{section-subscript} is a \textit{subscript-triplet} \texttt{\(l\):\(u\):\(s\)} and the array axis corresponds to an axis of TA distributed \texttt{BLOCK(\(n\))} (which might have been specified as simply \texttt{BLOCK}, but there will be some \(n\) that describes the resulting distribution), then the distribution of the axis of TD also must be \texttt{BLOCK(\(n\))} (or \texttt{BLOCK}, as above). Similarly for \texttt{CYCLIC(\(n\))} and \texttt{CYCLIC}. (This just repeats a constraint previously mentioned that the distribution formats of these axes of TA and TD have to be the same.) Note that the constraints that apply to this case require that \(l\) must be the declared lower bound, \(u\) must be the declared upper bound, and \(s\) must be 1; equivalently, the triplet \texttt{\(l\):\(u\):\(s\)} could simply have been written as \texttt{:}. \end{itemize} Note that when the dummy has a natural template, it would be non-conforming to have a scalar \textit{section-subscript} on an array axis corresponding to an axis of TA not distributed \texttt{*}. \end{enumerate} Here is an example: The main program has a two-dimensional array \texttt{TROGGS}, which is to be processed by a subroutine one column at a time. (Perhaps processing the entire array at once would require prohibitive amounts of temporary space.) Each column is to be distributed across many processors. \CODE REAL TROGGS(1024,473) !HPF$ PROCESSORS PMAIN(8) !HPF$ DISTRIBUTE TROGGS(BLOCK,*) ONTO PMAIN DO J=1,473 CALL WILD_THING(TROGGS(:,J)) END DO \EDOC Each column of {\tt TROGGS} has a {\tt BLOCK} distribution. The rules listed above mean that the lack of an explicit interface is legal in this case provided the programmer writes directives in the subprogram as follows (either a descriptive or a prescriptive form of the distribution directive is correct): \CODE SUBROUTINE WILD_THING(GROOVY) REAL GROOVY(1024) !HPF$ PROCESSORS PSUB(8) !HPF$ DISTRIBUTE GROOVY *(BLOCK) ONTO *PSUB \EDOC \begin{users} These conditions are complex. The important things to realize are these: \begin{itemize} \item You don't have to read any of this if you have an explicit interface. So if there is any doubt in your mind, just make sure you have an explicit interface. \item When there is an explicit interface, there are other kinds of pairs of mappings that can legitimately be said to be ``the same''. These restrictions apply only when there is no explicit interface. \end{itemize} \end{users} \section{Restrictions on Pointers and Targets} \label{mapsub:pointers} If, on invocation of a procedure P: (a)~a dummy argument has the \texttt{TARGET} attribute, and (b)~the corresponding actual argument has the \texttt{TARGET} attribute and is not an array section with a vector subscript (and therefore is an object A or a section of an array A), then the program is not HPF-conforming unless: \begin{enumerate} \item No remapping of the actual argument occurs during the call; or \item the remainder of program execution would be unaffected if \begin{enumerate} \item each pointer associated with any portion of the dummy argument or with any portion of A during execution of P were to acquire undefined pointer association status on exit from P; and \item each pointer associated with any portion of A before the call were to acquire undefined pointer association status on entry to P and, if not reassigned during execution of P, were to be restored on exit to the pointer association status it had before entry. \end{enumerate} \end{enumerate} Note that if a dummy argument has the \texttt{TARGET} attribute and no explicit mapping attributes, then the \texttt{INHERIT} attribute is implicitly assumed (see section~\ref{INHERIT-SECTION}); therefore no remapping occurs for such a dummy argument and there is no problem. \begin{rationale} These restrictions are made in order to support the following part of the Fortran standard (in Section 12.4.1.1 of that document) in the face of implicit remapping across the subprogram interface: \begin{quote} If the dummy argument does not have the \texttt{TARGET} or \texttt{POINTER} attribute, any pointers associated with the actual argument do not become associated with the corresponding dummy argument on invocation of the procedure. If the dummy argument has the \texttt{TARGET} attribute and the corresponding actual argument has the \texttt{TARGET} attribute but is not an array section with a vector subscript: \begin{enumerate} \item Any pointers associated with the actual argument become associated with the corresponding dummy argument on invocation of the procedure. \item When execution of the procedure completes, any pointers associated with the dummy argument remain associated with the actual argument. \end{enumerate} If the dummy argument has the \texttt{TARGET} attribute and the corresponding actual argument does not have the \texttt{TARGET} attribute or is an array section with a vector subscript, any pointers associated with the dummy argument become undefined when execution of the procedure completes. \end{quote} \end{rationale} \section{Argument Passing and Sequence Association} For actual arguments in a procedure call, Fortran allows an array element (scalar) to be associated with a dummy argument that is an array. It furthermore allows the shape of a dummy argument to differ from the shape of the corresponding actual array argument, in effect reshaping the actual argument via the subroutine call. Storage sequence properties of Fortran are used to identify the values of the dummy argument. This feature, carried over from FORTRAN 77, has been widely used to pass starting addresses of subarrays, rows or columns of a larger array, to procedures. For HPF arrays that are potentially mapped across processors, this feature is not fully supported. \subsection{Sequence Association Rules} \begin{enumerate} \item When an array element or the name of an assumed-size array is used as an actual argument, the associated dummy argument must be a scalar or specified to be a sequential array. An array-element designator of a nonsequential array must not be associated with a dummy array argument. \item When an actual argument is an array or array section and the corresponding dummy argument differs from the actual argument in shape, then the dummy argument must be declared sequential and the actual array argument must be sequential. \item An object of type character (scalar or array) is nonsequential if it conforms to the requirements of Definition~\ref{seq-var} of Section~\ref{sequence-defs}. If the length of an explicit-length character dummy argument differs from the length of the actual argument, then both the actual and dummy arguments must be sequential. \item Without an explicit interface, a sequential actual may not be associated with a nonsequential dummy and a nonsequential actual may not be associated with a sequential dummy. (This item merely repeats part of Section~\ref{mapsub:ExplicitInterfaces}.) \end{enumerate} \subsection{Discussion of Sequence Association} When the shape of the dummy array argument and its associated actual array argument differ, the actual argument must not be an expression. There is no HPF mechanism for declaring that the value of an array-valued expression is sequential. In order to associate such an expression as an actual argument with a dummy argument of different rank, the actual argument must first be assigned to a named array variable that is forced to be sequential according to Definition~\ref{seq-var} of Section~\ref{sequence-defs}. \subsection{Examples of Sequence Association} Given the following subroutine fragment: \CODE SUBROUTINE HOME (X) DIMENSION X (20,10) \EDOC By rule 1 \CODE CALL HOME (ET (2,1)) \EDOC is legal only if \texttt{X} is declared sequential in \texttt{HOME} and \texttt{ET} is sequential in the calling routine. Likewise, by rule 2 and 4 \CODE CALL HOME (ET) \EDOC requires either that \texttt{ET} and \texttt{X} are both sequential arrays or that \texttt{ET} and \texttt{X} have the same shape and have the same sequence attribute. Rule 3 addresses a special consideration for objects of type character. Change of the length of character objects across a call, as in \CODE CHARACTER (LEN=44) one_long_word one_long_word = 'Chargoggagoggmanchaugagoggchaubunagungamaugg' CALL webster(one_long_word) SUBROUTINE webster(short_dictionary) CHARACTER (LEN=4) short_dictionary (11) !Note that short_dictionary(3) is 'agog', for example \EDOC \noindent is conceptually legal in FORTRAN 77 and Fortran 95. In HPF, both the actual argument and dummy argument must be sequential. (By the way, ``Chargoggagoggmanchaugagoggchaubunagungamaugg'' is the original Nipmuc name for what is now called ``Lake Webster'' in Massachusetts.) --------------------------------------------------------------------------- To (un)subscribe to this list, send mail to hpff-doc-request@cs.rice.edu. Leave the subject line blank, and in the body put the line (un)subscribe --------------------------------------------------------------------------- From owner-hpff-doc Mon Jun 3 12:01:02 1996 Received: (from daemon@localhost) by cs.rice.edu (8.7.1/8.7.1) id LAA01777 for hpff-doc-out; Mon, 3 Jun 1996 11:54:12 -0500 (CDT) Date: Mon, 3 Jun 1996 11:54:12 -0500 (CDT) Message-Id: <199606031654.LAA01777@cs.rice.edu> From: offner@hpc.pko.dec.com (Carl Offner) Subject: hpff-doc: new Chapter 3 (mappings in subroutine interfaces) Sender: owner-hpff-doc Precedence: bulk --------------------------------------------------------------------------- hpff-doc@cs.rice.edu is a mailing list for HPF 2.0 language specification authors and editors. Instructions for adding or deleting yourself from this list appear at the bottom of this message. --------------------------------------------------------------------------- % File: mapping-subr.tex % Contents: % Mapping constructs for dummy arguments for HPF 2.0 document, % including % interface rules % INHERIT % the rest of pointer mappings % Revision history: % Jun-03-96 First edit by Carl Offner, Digital. % (Rearrangements, many edits, and two new sections; see % the comments at the top of the chapter.) % May-10-96 Created by Charles Koelbel, Rice University % (from HPF 1.2 document and HPF 2.0 proposals) % If you don't have LaTeX2e available, uncomment the next three lines: %\def\emph#1{{\em #1}} %\def\texttt#1{{\tt #1}} %\def\textit#1{{\it #1}} \chapter{Data Mapping in Subprogram Interfaces} \label{ch-mapping-subr} {\em Comments on this chpater should be directed to Piyush Mehrotra ({\tt pm@icase.edu}), Carl Offner ({\tt offner@hpc.pko.dec.com}), Guy Steele ({\tt Guy.Steele@east.sun.com}), and \\ {\tt hpff-doc@cs.rice.edu}. Please use ``{\tt Comments on Mapping in Subprogram Calls}'' as the {\tt Subject:} line. \par } \begin{new} \emph{This chapter has been rearranged. I have added comments on actual text changes (other than the most trivial ones). I also replaced ``Fortran 90'' by ``Fortran'' consistently except in the last paragraph, where I replaced it by ``Fortran 95'' for clarity. I deleted the section on explicit dynamic remapping of dummy arguments (i.e., the section dealing with \texttt{DYNAMIC}, \texttt{REALIGN}, and \texttt{REDISTRIBUTE}); this section belongs with the approved extensions. Sections~\ref{mapsub:introduction} and \ref{mapsub:ExplicitInterfaces} (including \ref{mapsub:SameMap}) are new.---C.O.} \end{new} \section{Introduction} \label{mapsub:introduction} \emph{This introduction gives an overview of the ways in which mapping directives interact with argument passing to subroutines. The language used here, however, is not definitive; the subsequent sections of this chapter contain the authoritative rules.} In addition to the data mapping features described in Chapter~\ref{ch-mapping-base}, HPF allows a number of options for describing the mapping of dummy arguments to subprograms. The mapping of each such dummy argument may be related to the mapping of its associated actual argument in the calling routine (the ``caller'') in several different ways. To allow for this, mapping directives applied to dummy arguments can have three different syntactic forms: \emph{prescriptive}, \emph{descriptive}, and \emph{transcriptive}. HPF provides these three forms to allow the programmer either to leave the data in place, or to force data into a new (presumably more efficient) mapping for the duration of the subprogram's execution. The meaning of these forms is as follows: \begin{description} \item[prescriptive] The directive describes the mapping of the dummy. However, the actual as passed may not have this mapping. \emph{If it does not}, it is the responsibility of the subprogram to remap the argument as specified, and to restore the original mapping on exit. (However, see the ``Advice to implementors'' below.) Prescriptive directives are syntactically identical to directives occurring elsewhere in the program. For instance, if \texttt{A} is a dummy argument, \CODE !HPF$ DISTRIBUTE A (BLOCK, CYCLIC) \EDOC is a prescriptive directive. \item[descriptive] The directive describes the mapping of the dummy. It is the responsibility of the calling routine to insure that the actual as passed has this mapping. (Similarly, remapping to restore the original mapping on exit is also done by the caller.) Descriptive directives look like prescriptive directives, except that an asterisk precedes the description. For instance, \CODE !HPF$ DISTRIBUTE A *(BLOCK, CYCLIC) \EDOC is a descriptive directive. \item[transcriptive] The mapping is unspecified. The called subroutine must accept the mapping of the argument as it is passed. (Of course this means that the caller must pass this mapping information at run-time.) Transcriptive directives are written with a single asterisk for distributions and processor arrays; for instance \CODE !HPF$ DISTRIBUTE A * \EDOC is a transcriptive directive. The \texttt{INHERIT} directive (see Section~\ref{INHERIT-SECTION}) is used to specify a transcriptive alignment. \end{description} Both distributions and processor arrangements can be specified to be prescriptive, descriptive, or transcriptive. Alignment is more complicated, because of the need to specify the template to which the dummy is aligned. This template may be unspecified (in this case of course there is no \texttt{ALIGN} directive), in which case it is the \emph{natural template} of the dummy. (``Natural template'' is defined in Section~\ref{mapsub:templates} below.) Otherwise, one of the following disjoint possibilities must be true: \begin{itemize} \item The template is explicitly specified by a prescriptive ALIGN directive. \item The template is explicitly specified by a descriptive ALIGN directive. \item The template is \emph{inherited}. This is specified by giving the dummy the \texttt{INHERIT} attribute (described in Section~\ref{INHERIT-SECTION} below). This implicitly specifies the template to be the template with which the corresponding actual argument is ultimately aligned; further, the alignment of the dummy with that template is the same as that of the corresponding actual. This is in effect a transcriptive form of alignment. \end{itemize} This is restated more precisely in Section~\ref{mapsub:templates} below. If remapping is necessary at a procedure interface, this fact must be visible in the caller. This is a consequence of the requirements for explicit interfaces in Section~\ref{mapsub:ExplicitInterfaces}. \begin{implementors} As a result of this, an implementation may choose to have all necessary remapping done by the caller. This in effect eliminates the distinction between prescriptive and descriptive mappings. \end{implementors} \begin{users} Although it is possible to write combinations of mapping directives that are partially prescriptive and partially transcriptive, for instance, there is probably no virtue to so doing. The point of these directives is to enable the compiler to handle any necessary remapping correctly and efficiently. Now remapping can happen for one or more of the following reasons: \begin{itemize} \item to make the alignment of the actual and the dummy agree; \item to make the distribution of the actual and the dummy agree; \item to make the processor array of the actual and the dummy agree. \end{itemize} For most machines, there is no real difference in the cost of remapping for any of these reasons. It is therefore a better practice in general to make a mapping either purely transcriptive, purely prescriptive, or purely descriptive. While transcriptive mappings can be useful in writing libraries, they impose a run-time cost on the subprogram. They should therefore be avoided in normal user code. \end{users} \section{What Remapping is Required, and Who Does It} \begin{new} \emph{This section has been rewritten. I added Chuck's suggested wording, even though my reading of CCI 50 indicates that no wording change was necessary. But it certainly doesn't do any harm.---C.O.} \end{new} If there is an explicit interface for the called subprogram and that interface contains mapping directives (whether prescriptive or descriptive) for the dummy argument in question, and if a remapping of the actual argument is necessary, the call should proceed as if the data was copied to a temporary variable to match the mapping of the dummy argument as expressed by the directives in the explicit interface. The template of the dummy will then be as declared in the interface. If in such a case, the mapping directives are descriptive, then it is the responsibility of the caller to perform the necessary remapping. That is, the caller must provide an actual argument matching such directives, so that the descriptive directives correctly describe the dummy arguments to the subprogram. \begin{implementors} In fact, an implementation may choose to have \emph{all} necessary remapping done by the caller. \end{implementors} If there is no explicit interface, then no remapping will be necessary; this is a consequence of the requirements in Section~\ref{mapsub:ExplicitInterfaces}. \begin{new} \emph{These next sentences were taken from the section on \texttt{DYNAMIC} remappings, which is otherwise moved to the part of the document containing the approved extensions.---C.O.} \end{new} An overriding principle is that \emph{any mapping or remapping of arguments is not visible to the caller}. That is, when the subprogram returns and the caller resumes execution, all objects accessible to the caller after the call are mapped exactly as they were before the call. It is not possible for a subprogram to change the mapping of any object in a manner visible to its caller. \section{Distributions and Processor Arrangements} \label{mapsub:DistProcArr} In a \texttt{DISTRIBUTE} directive where every \textit{distributee} is a dummy argument, either the \textit{dist-format-clause} or the \textit{dist-target}, or both, may begin with, or consist of, an asterisk. \begin{itemize} \item Without an asterisk, a \textit{dist-format-clause} or \textit{dist-target} is prescriptive; the clause describes a distribution and constitutes a request of the language processor to make it so. This might entail the subprogram remapping or copying the actual argument on entry at run time in order to satisfy the requested distribution for the dummy. \begin{new} \emph{I added a couple of references to the ``subprogram'' in the previous and next items for clarity.---C.O.} \end{new} \item Starting with an asterisk, a \textit{dist-format-clause} or \textit{dist-target} is descriptive; the clause describes a distribution and constitutes an assertion to the language processor that on entry to the subprogram it will already be so. The programmer claims that, for every call to the subprogram, the actual argument will be such that the stated distribution already describes the mapping of that data. (The intent is that if the argument is passed by reference, no movement of the data by the subprogram will be necessary at run time. All this is under the assumption that the language processor has observed all other directives. While a conforming HPF language processor is not required to obey mapping directives, it should handle descriptive directives with the understanding that their implied assertions are relative to this assumption.) \begin{new} \emph{I deleted ``whether explicit or implicit'' in the last sentence of the following item, since there is no more implicit transcriptive behavior.---C.O.} \end{new} \item Consisting of only an asterisk, a \textit{dist-format-clause} or \textit{dist-target} is transcriptive; the clause says nothing about the distribution but constitutes a request of the language processor to copy that aspect of the distribution from that of the actual argument. (The intent is that if the argument is passed by reference, no movement of the data will be necessary at run time.) Note that the transcriptive case is not included in Subset HPF. \end{itemize} It is possible that, in a single \texttt{DISTRIBUTE} directive, the \textit{dist-format-clause} might have an asterisk but not the \textit{dist-target}, or vice versa. \subsection{Examples} These examples of \texttt{DISTRIBUTE} directives for dummy arguments illustrate the various combinations: \CODE !HPF$ DISTRIBUTE URANIA (CYCLIC) ONTO GALILEO \EDOC The language processor should do whatever it takes to cause \texttt{URANIA} to have a \texttt{CYCLIC} distribution on the processor arrangement \texttt{GALILEO}. \CODE !HPF$ DISTRIBUTE POLYHYMNIA * ONTO ELVIS \EDOC The language processor should do whatever it takes to cause \texttt{POLYHYMNIA} to be distributed onto the processor arrangement \texttt{ELVIS}, using whatever distribution format it currently has (which might be on some other processor arrangement). (You can't say this in Subset HPF.) \CODE !HPF$ DISTRIBUTE THALIA *(CYCLIC) ONTO FLIP \EDOC The language processor should do whatever it takes to cause \texttt{THALIA} to have a \texttt{CYCLIC} distribution on the processor arrangement \texttt{FLIP}; \texttt{THALIA} already has a cyclic distribution, though it might be on some other processor arrangement. \CODE !HPF$ DISTRIBUTE CALLIOPE (CYCLIC) ONTO *HOMER \EDOC The language processor should do whatever it takes to cause \texttt{CALLIOPE} to have a \texttt{CYCLIC} distribution on the processor arrangement \texttt{HOMER}; \texttt{CALLIOPE} is already distributed onto \texttt{HOMER}, though it might be with some other distribution format. \CODE !HPF$ DISTRIBUTE MELPOMENE * ONTO *EURIPIDES \EDOC \texttt{MELPOMENE} is asserted to already be distributed onto \texttt{EURIPIDES}; use whatever distribution format the actual argument had so, if possible, no data movement should occur. (You can't say this in Subset HPF.) \CODE !HPF$ DISTRIBUTE CLIO *(CYCLIC) ONTO *HERODOTUS \EDOC \texttt{CLIO} is asserted to already be distributed \texttt{CYCLIC} onto \texttt{HERODOTUS} so, if possible, no data movement should occur. \CODE !HPF$ DISTRIBUTE EUTERPE (CYCLIC) ONTO * \EDOC The language processor should do whatever it takes to cause \texttt{EUTERPE} to have a \texttt{CYCLIC} distribution onto whatever processor arrangement the actual was distributed onto. (You can't say this in Subset HPF.) \CODE !HPF$ DISTRIBUTE ERATO * ONTO * \EDOC The mapping of \texttt{ERATO} should not be changed from that of the actual argument. (You can't say this in Subset HPF.) \CODE !HPF$ DISTRIBUTE ARTHUR_MURRAY *(CYCLIC) ONTO * \EDOC \texttt{ARTHUR_MURRAY} is asserted to already be distributed \texttt{CYCLIC} onto whatever processor arrangement the actual argument was distributed onto, and no data movement should occur. (You can't say this in Subset HPF.) Please note that \texttt{DISTRIBUTE ERATO * ONTO *} does not mean the same thing as \CODE !HPF$ DISTRIBUTE ERATO *(*) ONTO * \EDOC This latter means: \texttt{ERATO} is asserted to already be distributed \texttt{*} (that is, on-processor) onto whatever processor arrangement the actual was distributed onto. Note that the processor arrangement is necessarily scalar in this case. GUY - CCI 30 SAYS THERE THERE SHOULD BE A NOTE THAT `THIS' ALSO APPLIES TO LOCAL VARIABLES. IT ISN'T CLEAR WHAT `THIS' IS OR HOW IT ANSWERS THE ORIGINAL CCI QUESTION - SHOULD IT BE SOMETHING LIKE, `LOCAL VARIABLES ALIGNED WITH ARGUMENTS IN A SIMILAR WAY WILL ALSO HAVE A SCALAR PROCESSOR ARRANGEMENT'? \subsection{What Happens When a Clause is Omitted} One may omit either the \textit{dist-format-clause} or the \textit{dist-onto-clause} for a dummy argument. If such a clause is omitted and the dummy argument has the \texttt{INHERIT} attribute (see Section~\ref{INHERIT-SECTION}), then the compiler must handle the directive as if \texttt{*} or \texttt{ONTO~*} had been specified explicitly. If such a clause is omitted and the dummy does not have the \texttt{INHERIT} attribute, then the compiler may choose the distribution format or a target processor arrangement arbitrarily. Here are two examples: \CODE !HPF$ DISTRIBUTE WHEEL_OF_FORTUNE *(CYCLIC) \EDOC \texttt{WHEEL_OF_FORTUNE} is asserted to already be \texttt{CYCLIC}. As long as it is kept \texttt{CYCLIC}, it may be remapped onto some other processor arrangement, but there is no reason to. \CODE !HPF$ DISTRIBUTE ONTO *TV :: DAVID_LETTERMAN \EDOC \texttt{DAVID_LETTERMAN} is asserted to already be distributed on \texttt{TV} in some fashion. The distribution format may be changed as long as \texttt{DAVID_LETTERMAN} is kept on \texttt{TV}. (Note that this declaration must be made in attributed form; the statement form \CODE !HPF$ DISTRIBUTE DAVID_LETTERMAN ONTO *TV !Nonconforming \EDOC does not conform to the syntax for a \texttt{DISTRIBUTE} directive.) \section{Alignment} \subsection{The Template of the Dummy Argument} \label{mapsub:templates} Here we describe precisely how the template with which the dummy argument is ultimately aligned is arrived at. \begin{new} \emph{This next paragraph was lifted from the TEMPLATE section of the mapping-base.tex file, and should be deleted from it.} \end{new} First, templates are not passed through the subprogram argument interface. The template to which a dummy argument is aligned is always distinct from the template to which the actual argument is aligned, though it may be a copy (see Section \ref{INHERIT-SECTION}). On exit from a subprogram, an HPF implementation arranges that the actual argument is aligned with the same template with which it was aligned before the call. Thus, a dummy argument always has a fresh template to which it is ultimately aligned. This template is constructed in one of three ways: \begin{itemize} \item If the dummy argument appears explicitly as an \textit{alignee} in an \texttt{ALIGN} directive, its template is specified by the \textit{align-target}. \item If the dummy argument is not explicitly aligned and does not have the \texttt{INHERIT} attribute (described in Section~\ref{INHERIT-SECTION} below), then the template has the same shape and bounds as the dummy argument; this is called the \textit{natural template} for the dummy. (Thus, all the examples in Section~\ref{mapsub:DistProcArr} use the natural template.) \item If the dummy argument is not explicitly aligned and does have the \texttt{INHERIT} attribute, then the template is ``inherited'' from the actual argument according to the following rules: \begin{itemize} \item If the actual argument is a whole array, the template of the dummy is a copy of the template with which the actual argument is ultimately aligned. \item If the actual argument is an array section of array \(A\) where no subscript is a vector subscript, then the template of the dummy is a copy of the template with which \(A\) is ultimately aligned. \item If the actual argument is any other expression, the shape and distribution of the template may be chosen arbitrarily by the language processor (and therefore the programmer cannot know anything \textit{a priori} about its distribution). \end{itemize} In all of these cases, we say that the dummy has an \textit{inherited template} rather than a natural template. \end{itemize} \subsection{The INHERIT Directive} \label{INHERIT-SECTION} The \texttt{INHERIT} directive specifies that a dummy argument should be aligned to a copy of the template of the corresponding actual argument in the same way that the actual argument is aligned. \BNF inherit-directive \IS INHERIT dummy-argument-name-list \FNB The \texttt{INHERIT} directive causes the named subprogram dummy arguments to have the \texttt{INHERIT} attribute. Only dummy arguments may have the \texttt{INHERIT} attribute. An object must not have both the \texttt{INHERIT} attribute and the \texttt{ALIGN} attribute. The \texttt{INHERIT} directive may appear only in a \textit{specification-part} of a scoping unit. If a dummy argument has the \texttt{TARGET} attribute and no explicit mapping attributes, then the \texttt{INHERIT} attribute is implicitly assumed. (See section~\ref{mapsub:pointers}.) \begin{new} \emph{Some sentences from the old 3.10 were inserted in the next paragraph (which was also split into two paragraphs), and the corresponding section was deleted from the discussion of distributions.---C.O.} \end{new} The \texttt{INHERIT} attribute specifies that the template for a dummy argument should be inherited, by making a copy of the template of the actual argument. Moreover, the \texttt{INHERIT} attribute implies a default distribution of \texttt{DISTRIBUTE~*~ONTO~*}. In such a case, the net effect is to tell the compiler to leave the data exactly where it is---and not attempt to remap the actual argument. The dummy argument will be mapped in exactly the same manner as the actual argument; the subprogram must be compiled in such a way as to work correctly no matter how the actual argument may be mapped onto abstract processors. If on the other hand an explicit mapping directive appears for a dummy argument with the \texttt{INHERIT} attribute, thereby overriding the default distribution, then the actual argument must be a whole array or an array section with no vector subscripts; it may not be an expression of any other form. \begin{new} \emph{The next paragraph is not needed; it just repeats the definition of natural template given in the previous section.---C.O.} \end{new} \begin{obsolete} If none of the attributes \texttt{INHERIT}, \texttt{ALIGN}, and \texttt{DISTRIBUTE} is specified explicitly for a dummy argument, then the template of the dummy argument has the same shape as the dummy itself and the dummy argument is aligned to its template by the identity mapping. \end{obsolete} An \texttt{INHERIT} directive may be combined with other directives, with the attributes stated in any order, more or less consistent with Fortran attribute syntax. \subsection{Examples} Here is a straightforward example of the use of \texttt{INHERIT}: \CODE REAL DOUGH(100) !HPF$ DISTRIBUTE DOUGH(BLOCK(10)) CALL PROBATE( DOUGH(7:23:2) ) ... SUBROUTINE PROBATE(BREAD) REAL BREAD(9) !HPF$ INHERIT BREAD \EDOC \begin{new} \emph{I added ``or descriptive'' in the next sentence.---C.O.} \end{new} The inherited template of \texttt{BREAD} has shape [100]; element \texttt{BREAD(I)} is aligned with element 5 + 2*I of the inherited template and, since \texttt{BREAD} does not appear in a prescriptive or descriptive \texttt{DISTRIBUTE} directive, it has a \texttt{BLOCK(10)} distribution. Here is a more complex example: \CODE LOGICAL FRUG(128),TWIST(128) !HPF$ PROCESSORS DANCE_FLOOR(16) !HPF$ DISTRIBUTE (BLOCK) ONTO DANCE_FLOOR::FRUG,TWIST CALL TERPSICHORE(FRUG(1:40:3),TWIST(1:40:3)) \EDOC The two array sections \texttt{FRUG(1:40:3)} and \texttt{TWIST(1:40:3)} are mapped onto abstract processors in the same manner: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,225)(0,0) \put(0,200){\makebox(35,25){\small\rm 1}} \put(35,200){\makebox(35,25){\small\rm 2}} \put(70,200){\makebox(35,25){\small\rm 3}} \put(105,200){\makebox(35,25){\small\rm 4}} \put(140,200){\makebox(35,25){\small\rm 5}} \put(175,200){\makebox(35,25){\small\rm 6}} \put(210,200){\makebox(35,25){\small\rm 7}} \put(245,200){\makebox(35,25){\small\rm 8}} \put(280,200){\makebox(35,25){\small\rm 9}} \put(315,200){\makebox(35,25){\small\rm 10}} \put(350,200){\makebox(35,25){\small\rm 11}} \put(385,200){\makebox(35,25){\small\rm 12}} \put(420,200){\makebox(35,25){\small\rm 13}} \put(455,200){\makebox(35,25){\small\rm 14}} \put(490,200){\makebox(35,25){\small\rm 15}} \put(525,200){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){7}{\line(1,0){560}} \thicklines \multiput(0,0)(0,200){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){200}} \put(0,175){\makebox(35,25){\tt 1}} \put(0,100){\makebox(35,25){\tt 4}} \put(0,25){\makebox(35,25){\tt 7}} \put(35,150){\makebox(35,25){\tt 10}} \put(35,75){\makebox(35,25){\tt 13}} \put(35,0){\makebox(35,25){\tt 16}} \put(70,125){\makebox(35,25){\tt 19}} \put(70,50){\makebox(35,25){\tt 22}} \put(105,175){\makebox(35,25){\tt 25}} \put(105,100){\makebox(35,25){\tt 28}} \put(105,25){\makebox(35,25){\tt 31}} \put(140,150){\makebox(35,25){\tt 34}} \put(140,75){\makebox(35,25){\tt 37}} \put(140,0){\makebox(35,25){\tt 40}} \end{picture} \end{center} However, the subroutine \texttt{TERPSICHORE} will view them in different ways because it inherits the template for the second dummy but not the first: \CODE SUBROUTINE TERPSICHORE(FOXTROT,TANGO) LOGICAL FOXTROT(:),TANGO(:) !HPF$ INHERIT TANGO \EDOC Therefore the template of \texttt{TANGO} is a copy of the 128 element template of the whole array \texttt{TWIST}. The template is mapped like this: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,225)(0,0) \put(0,200){\makebox(35,25){\small\rm 1}} \put(35,200){\makebox(35,25){\small\rm 2}} \put(70,200){\makebox(35,25){\small\rm 3}} \put(105,200){\makebox(35,25){\small\rm 4}} \put(140,200){\makebox(35,25){\small\rm 5}} \put(175,200){\makebox(35,25){\small\rm 6}} \put(210,200){\makebox(35,25){\small\rm 7}} \put(245,200){\makebox(35,25){\small\rm 8}} \put(280,200){\makebox(35,25){\small\rm 9}} \put(315,200){\makebox(35,25){\small\rm 10}} \put(350,200){\makebox(35,25){\small\rm 11}} \put(385,200){\makebox(35,25){\small\rm 12}} \put(420,200){\makebox(35,25){\small\rm 13}} \put(455,200){\makebox(35,25){\small\rm 14}} \put(490,200){\makebox(35,25){\small\rm 15}} \put(525,200){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){7}{\line(1,0){560}} \thicklines \multiput(0,0)(0,200){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){200}} \put(0,175){\makebox(35,25){\tt 1}} \put(0,150){\makebox(35,25){\tt 2}} \put(0,125){\makebox(35,25){\tt 3}} \put(0,100){\makebox(35,25){\tt 4}} \put(0,75){\makebox(35,25){\tt 5}} \put(0,50){\makebox(35,25){\tt 6}} \put(0,25){\makebox(35,25){\tt 7}} \put(0,0){\makebox(35,25){\tt 8}} \put(35,175){\makebox(35,25){\tt 9}} \put(35,150){\makebox(35,25){\tt 10}} \put(35,125){\makebox(35,25){\tt 11}} \put(35,100){\makebox(35,25){\tt 12}} \put(35,75){\makebox(35,25){\tt 13}} \put(35,50){\makebox(35,25){\tt 14}} \put(35,25){\makebox(35,25){\tt 15}} \put(35,0){\makebox(35,25){\tt 16}} \put(70,175){\makebox(35,25){\tt 17}} \put(70,150){\makebox(35,25){\tt 18}} \put(70,125){\makebox(35,25){\tt 19}} \put(70,100){\makebox(35,25){\tt 20}} \put(70,75){\makebox(35,25){\tt 21}} \put(70,50){\makebox(35,25){\tt 22}} \put(70,25){\makebox(35,25){\tt 23}} \put(70,0){\makebox(35,25){\tt 24}} \put(105,175){\makebox(35,25){\tt 25}} \put(105,150){\makebox(35,25){\tt 26}} \put(105,125){\makebox(35,25){\tt 27}} \put(105,100){\makebox(35,25){\tt 28}} \put(105,75){\makebox(35,25){\tt 29}} \put(105,50){\makebox(35,25){\tt 30}} \put(105,25){\makebox(35,25){\tt 31}} \put(105,0){\makebox(35,25){\tt 32}} \put(140,175){\makebox(35,25){\tt 33}} \put(140,150){\makebox(35,25){\tt 34}} \put(140,125){\makebox(35,25){\tt 35}} \put(140,100){\makebox(35,25){\tt 36}} \put(140,75){\makebox(35,25){\tt 37}} \put(140,50){\makebox(35,25){\tt 38}} \put(140,25){\makebox(35,25){\tt 39}} \put(140,0){\makebox(35,25){\tt 40}} \put(175,175){\makebox(35,25){\tt 41}} \put(175,150){\makebox(35,25){\tt 42}} \put(175,125){\makebox(35,25){\tt 43}} \put(175,100){\makebox(35,25){\tt 44}} \put(175,75){\makebox(35,25){\tt 45}} \put(175,50){\makebox(35,25){\tt 46}} \put(175,25){\makebox(35,25){\tt 47}} \put(175,0){\makebox(35,25){\tt 48}} \put(210,175){\makebox(35,25){\tt 49}} \put(210,150){\makebox(35,25){\tt 50}} \put(210,125){\makebox(35,25){\tt 51}} \put(210,100){\makebox(35,25){\tt 52}} \put(210,75){\makebox(35,25){\tt 53}} \put(210,50){\makebox(35,25){\tt 54}} \put(210,25){\makebox(35,25){\tt 55}} \put(210,0){\makebox(35,25){\tt 56}} \put(245,175){\makebox(35,25){\tt 57}} \put(245,150){\makebox(35,25){\tt 58}} \put(245,125){\makebox(35,25){\tt 59}} \put(245,100){\makebox(35,25){\tt 60}} \put(245,75){\makebox(35,25){\tt 61}} \put(245,50){\makebox(35,25){\tt 62}} \put(245,25){\makebox(35,25){\tt 63}} \put(245,0){\makebox(35,25){\tt 64}} \put(280,175){\makebox(35,25){\tt 65}} \put(280,150){\makebox(35,25){\tt 66}} \put(280,125){\makebox(35,25){\tt 67}} \put(280,100){\makebox(35,25){\tt 68}} \put(280,75){\makebox(35,25){\tt 69}} \put(280,50){\makebox(35,25){\tt 70}} \put(280,25){\makebox(35,25){\tt 71}} \put(280,0){\makebox(35,25){\tt 72}} \put(315,175){\makebox(35,25){\tt 73}} \put(315,150){\makebox(35,25){\tt 74}} \put(315,125){\makebox(35,25){\tt 75}} \put(315,100){\makebox(35,25){\tt 76}} \put(315,75){\makebox(35,25){\tt 77}} \put(315,50){\makebox(35,25){\tt 78}} \put(315,25){\makebox(35,25){\tt 79}} \put(315,0){\makebox(35,25){\tt 80}} \put(350,175){\makebox(35,25){\tt 81}} \put(350,150){\makebox(35,25){\tt 82}} \put(350,125){\makebox(35,25){\tt 83}} \put(350,100){\makebox(35,25){\tt 84}} \put(350,75){\makebox(35,25){\tt 85}} \put(350,50){\makebox(35,25){\tt 86}} \put(350,25){\makebox(35,25){\tt 87}} \put(350,0){\makebox(35,25){\tt 88}} \put(385,175){\makebox(35,25){\tt 89}} \put(385,150){\makebox(35,25){\tt 90}} \put(385,125){\makebox(35,25){\tt 91}} \put(385,100){\makebox(35,25){\tt 92}} \put(385,75){\makebox(35,25){\tt 93}} \put(385,50){\makebox(35,25){\tt 94}} \put(385,25){\makebox(35,25){\tt 95}} \put(385,0){\makebox(35,25){\tt 96}} \put(420,175){\makebox(35,25){\tt 97}} \put(420,150){\makebox(35,25){\tt 98}} \put(420,125){\makebox(35,25){\tt 99}} \put(420,100){\makebox(35,25){\tt 100}} \put(420,75){\makebox(35,25){\tt 101}} \put(420,50){\makebox(35,25){\tt 102}} \put(420,25){\makebox(35,25){\tt 103}} \put(420,0){\makebox(35,25){\tt 104}} \put(455,175){\makebox(35,25){\tt 105}} \put(455,150){\makebox(35,25){\tt 106}} \put(455,125){\makebox(35,25){\tt 107}} \put(455,100){\makebox(35,25){\tt 108}} \put(455,75){\makebox(35,25){\tt 109}} \put(455,50){\makebox(35,25){\tt 110}} \put(455,25){\makebox(35,25){\tt 111}} \put(455,0){\makebox(35,25){\tt 112}} \put(490,175){\makebox(35,25){\tt 113}} \put(490,150){\makebox(35,25){\tt 114}} \put(490,125){\makebox(35,25){\tt 115}} \put(490,100){\makebox(35,25){\tt 116}} \put(490,75){\makebox(35,25){\tt 117}} \put(490,50){\makebox(35,25){\tt 118}} \put(490,25){\makebox(35,25){\tt 119}} \put(490,0){\makebox(35,25){\tt 120}} \put(525,175){\makebox(35,25){\tt 121}} \put(525,150){\makebox(35,25){\tt 122}} \put(525,125){\makebox(35,25){\tt 123}} \put(525,100){\makebox(35,25){\tt 124}} \put(525,75){\makebox(35,25){\tt 125}} \put(525,50){\makebox(35,25){\tt 126}} \put(525,25){\makebox(35,25){\tt 127}} \put(525,0){\makebox(35,25){\tt 128}} \end{picture} \end{center} \noindent \texttt{TANGO(I)} is aligned with element 3*I-2 of the template. But the template of \texttt{FOXTROT} has the same size 14 as \texttt{FOXTROT} itself. The actual argument, \texttt{FRUG(1:40:3)} is mapped to the 16 processors in this manner: \begin{center} \begin{tabular}{cc} Abstract & Elements \\ processor & of FRUG \\ 1 & 1, 2, 3 \\ 2 & 4, 5, 6 \\ 3 & 7, 8 \\ 4 & 9, 10, 11 \\ 5 & 12, 13, 14 \\ 6--16 & none \end{tabular} \end{center} It would be reasonable to understand the mapping of the template of \texttt{FOXTROT} to coincide with the layout of the array section: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,225)(0,0) \put(0,200){\makebox(35,25){\small\rm 1}} \put(35,200){\makebox(35,25){\small\rm 2}} \put(70,200){\makebox(35,25){\small\rm 3}} \put(105,200){\makebox(35,25){\small\rm 4}} \put(140,200){\makebox(35,25){\small\rm 5}} \put(175,200){\makebox(35,25){\small\rm 6}} \put(210,200){\makebox(35,25){\small\rm 7}} \put(245,200){\makebox(35,25){\small\rm 8}} \put(280,200){\makebox(35,25){\small\rm 9}} \put(315,200){\makebox(35,25){\small\rm 10}} \put(350,200){\makebox(35,25){\small\rm 11}} \put(385,200){\makebox(35,25){\small\rm 12}} \put(420,200){\makebox(35,25){\small\rm 13}} \put(455,200){\makebox(35,25){\small\rm 14}} \put(490,200){\makebox(35,25){\small\rm 15}} \put(525,200){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){7}{\line(1,0){560}} \thicklines \multiput(0,0)(0,200){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){200}} \put(0,175){\makebox(35,25){\tt 1}} \put(0,100){\makebox(35,25){\tt 2}} \put(0,25){\makebox(35,25){\tt 3}} \put(35,150){\makebox(35,25){\tt 4}} \put(35,75){\makebox(35,25){\tt 5}} \put(35,0){\makebox(35,25){\tt 6}} \put(70,125){\makebox(35,25){\tt 7}} \put(70,50){\makebox(35,25){\tt 8}} \put(105,175){\makebox(35,25){\tt 9}} \put(105,100){\makebox(35,25){\tt 10}} \put(105,25){\makebox(35,25){\tt 11}} \put(140,150){\makebox(35,25){\tt 12}} \put(140,75){\makebox(35,25){\tt 13}} \put(140,0){\makebox(35,25){\tt 14}} \end{picture} \end{center} \noindent but we shall see that this is not permitted in HPF. Within subroutine \texttt{TERPSICHORE} it would be correct to make the descriptive assertion \CODE !HPF$ DISTRIBUTE TANGO *(BLOCK) \EDOC but it would not be correct to declare \CODE !HPF$ DISTRIBUTE FOXTROT *(BLOCK) !Nonconforming \EDOC Each of these asserts that the template of the specified dummy argument is already distributed \texttt{BLOCK} on entry to the subroutine. The shape of the template for \texttt{TANGO} is \texttt{[128]}, inherited (copied) from the array \texttt{TWIST}, whose section was passed as the corresponding actual argument, and that template does indeed have a \texttt{BLOCK} distribution. But the shape of the template for \texttt{FOXTROT} is \texttt{[14]}; the layout of the elements of the actual argument \texttt{FRUG(1:40:3)} (3 on the first processor, 3 on the second processor, 2 on the third processor, 3 on the fourth processor, \dots) cannot properly be described as a \texttt{BLOCK} distribution of a length-14 template, so the \texttt{DISTRIBUTE} declaration for \texttt{FOXTROT} shown above would indeed be erroneous. On the other hand, the layout of \texttt{FRUG(1:40:3)} can be described in terms of an alignment to a length-128 template which can be described by an explicit \texttt{TEMPLATE} declaration (see Section \ref{TEMPLATE-SECTION}), so the directives \CODE !HPF$ PROCESSORS DANCE_FLOOR(16) !HPF$ TEMPLATE, DISTRIBUTE(BLOCK) ONTO DANCE_FLOOR::GURF(128) !HPF$ ALIGN FOXTROT(J) WITH *GURF(3*J-2) \EDOC could be correctly included in \texttt{TERPSICHORE} to describe the layout of \texttt{FOXTROT} on entry to the subroutine without using an inherited template. \subsection{ALIGN Directives} The presence or absence of an asterisk at the start of an \textit{align-spec} has the same meaning as in a \textit{dist-format-clause}: it specifies whether the \texttt{ALIGN} directive is descriptive or prescriptive, respectively. If an \textit{align-spec} that does not begin with \texttt{*} is applied to a dummy argument, the meaning is that the dummy argument will be forced to have the specified alignment on entry to the subprogram. This may require either the caller or the subprogram to temporarily remap the data of the actual argument or a copy thereof. Note that a dummy argument may also be used as an \textit{align-target}. \CODE SUBROUTINE NICHOLAS(TSAR,CZAR) REAL, DIMENSION(1918) :: TSAR,CZAR !HPF$ INHERIT :: TSAR !HPF$ ALIGN WITH TSAR :: CZAR \EDOC In this example the first dummy argument, \texttt{TSAR}, is allowed to remain aligned with the corresponding actual argument, while the second dummy argument, \texttt{CZAR}, is forced to be aligned with the first dummy argument. If the two actual arguments are already aligned, no remapping of the data will be required at run time. If they are not, some remapping will take place. \begin{new} \emph{I took out the phrase ``and not \texttt{REALIGN}'' at the end of the first sentence of the next paragraph, since \texttt{REALIGN} now belongs with the approved extensions.---C.O.} \end{new} If the \textit{align-spec} begins with ``\texttt{*}'', then the \textit{alignee} must be a dummy argument and the directive must be \texttt{ALIGN}. The ``\texttt{*}'' indicates that the \texttt{ALIGN} directive constitutes a guarantee on the part of the programmer that, on entry to the subprogram, the indicated alignment will already be satisfied by the dummy argument, without any action to remap it required (on the part of the subprogram) at run time. For example: \CODE SUBROUTINE GRUNGE(PLUNGE,SPONGE) REAL PLUNGE(1000),SPONGE(1000) !HPF$ INHERIT SPONGE !HPF$ ALIGN PLUNGE WITH *SPONGE \EDOC This asserts that, for every \texttt{J} in the range \texttt{1:1000}, on entry to subroutine \texttt{GRUNGE}, the directives in the program have specified that \texttt{PLUNGE(J)} is currently mapped to the same abstract processor as \texttt{SPONGE(J)}. (The intent is that if the language processor has in fact honored the directives, then no interprocessor communication on the part of the subprogram will be required to achieve the specified alignment.) The alignment of a general expression is up to the language processor and therefore unpredictable by the programmer; but the alignment of whole arrays and array sections is predictable. In the code fragment \CODE REAL FIJI(5000),SQUEEGEE(2000) !HPF$ ALIGN SQUEEGEE(K) WITH FIJI(2*K) CALL GRUNGE(FIJI(2002:4000:2),SQUEEGEE(1001:)) \EDOC it is true that every element of the array section \texttt{SQUEEGEE(1001:)} is aligned with the corresponding element of the array section \texttt{FIJI(2002:4000:2)}, so the claim made in subroutine \texttt{GRUNGE} is satisfied by this particular call. Under certain circumstances, it may be possible to specify that one dummy argument be remapped if necessary and then to specify that another dummy will then be aligned with it: \CODE SUBROUTINE MURKY(THINK, DENSE) !HPF$ PROCESSORS GUNK(32) !HPF$ DISTRIBUTE (BLOCK) ONTO GUNK :: DENSE !HPF$ ALIGN WITH *DENSE :: THICK \EDOC Note that the programmer cannot be justified in descriptively asserting that \texttt{THICK} will be aligned with \texttt{DENSE} after its remapping unless the remapping is fully specified (that is, no part of the remapping is left to the compiler to choose). Therefore an explicit processors arrangement necessarily appears in the example. The caller must ensure that the first actual argument is appropriately mapped onto an identical processors arrangement. It is not permitted to say simply ``\texttt{ALIGN WITH *}''; an \textit{align-target} must follow the asterisk. (The proper way to say ``accept any alignment'' is \texttt{INHERIT}.) If a dummy argument has no explicit \texttt{ALIGN} or \texttt{DISTRIBUTE} attribute, then the compiler provides an implicit alignment and distribution specification, one that could have been described explicitly without any ``assertion asterisks''. \section{Explicit Interfaces} \label{mapsub:ExplicitInterfaces} An explicit interface is required \emph{except} when all four of the following conditions hold: \begin{enumerate} \item Fortran does not require one, \emph{and} \item No dummy argument is passed transcriptively or with the \texttt{INHERIT} attribute, \emph{and} \item Either of the following cases applies to each pair of corresponding actual and dummy arguments: \begin{enumerate} \item They are both implicitly mapped. \item They are both explicitly mapped and the mappings are the same. Section~\ref{mapsub:SameMap} below defines what ``the mappings are the same'' means in this context. \end{enumerate} \emph{and} \item Either of the following cases applies to each pair of corresponding actual and dummy arguments: \begin{enumerate} \item Both are sequential. \item Both are nonsequential. \end{enumerate} \end{enumerate} \begin{rationale} This has the following consequences: \begin{itemize} \item A plain Fortran program (i.e., with no HPF directives) will continue to be legal without the need to add additional interfaces. This is insured by items 1, 2, 3a, and 4a. \item If remapping is necessary, this fact will be visible to the caller. Thus the implementation may choose to have all remapping performed by the caller. \end{itemize} \end{rationale} \begin{users} This requirement pushes the user strongly in the direction of always providing explicit interfaces. This is a good thing---explicit interfaces allow many errors to be caught at compile-time and greatly speed up the process of robust software development. Note, that an explicit interface can be provided in three ways: \begin{enumerate} \item by using a module. \item by a contained subprogram; \item by an interface block; \end{enumerate} In addition, an intrinsic procedure always has an explicit interface by definition. The idiomatic Fortran way of programming makes extensive use of modules; every subroutine, for instance, can be in a module. This provides explicit interfaces automatically, with no extra effort on the part of the programmer. It should very seldom be necessary to write an interface block. \end{users} \subsection{Explicit Mappings Without an Explicit Interface} \label{mapsub:SameMap} When there is no explicit interface, the actual and dummy arguments may still be explicitly mapped (but not transcriptively or with the \texttt{INHERIT} attribute), as long as the mappings are the same. This must be true in a very strict sense; in particular, the mappings must be the same whatever configuration of processors is used, and whatever system or user defaults may be in force. We give here the only circumstances under which this may be said to be true. First note that since only named objects can be mapped, the actual argument must be a whole array or array section. For the purposes of the conditions below, we shall regard a whole array as a particular kind of array section; i.e., the section each of whose subscripts is \texttt{:}. Denote the dummy argument by D, the actual argument by A, and the array of which the actual argument is a section by AA (AA might be the same as A). Denote the templates of the dummy and the actual by TD and TA, respectively. Denote the processor arrangements onto which TD and TA are distributed by PD and PA, respectively. PD and PA must have the same shape. Each axis of TD that is not distributed \texttt{*} must have the same extent and the same distribution format as the corresponding axis of TA, and conversely. In such a case, the axis of PD onto which the axis of TD is distributed must have the same extent as the axis of PA onto which the axis of TA is distributed. One of the two following cases must hold: \begin{enumerate} \item \label{mapsub:MapSameExpTmp} The dummy is described using an explicit named template (using a \texttt{TEMPLATE} directive); an example of this (\texttt{GURF}) appears in Section~\ref{INHERIT-SECTION}. (Actually, in that example, an explicit interface would be required, but only because of the other argument, which is inherited.) In such a case, for each \textit{section-subscript} of A, the following must hold: \begin{itemize} \item If the \textit{section-subscript} is scalar and the array axis is collapsed (as by an \texttt{ALIGN} directive) then no entry may appear in the distribution for TD. \item If the \textit{section-subscript} is scalar and the array axis corresponds to an axis of the template TA distributed \texttt{*}, then no entry may appear in the distribution for TD. \item If the \textit{section-subscript} is a scalar and the array axis corresponds to an axis of the template TA that is not distributed \texttt{*}, then the position on the axis of TA to which the scalar is aligned must correspond to the position on the corresponding axis of TD to which D is aligned. \item If the \textit{section-subscript} is a \textit{subscript-triplet} and the array axis is collapsed (as by an ALIGN directive), then this case can be treated \emph{as if} a corresponding axis, having the same extent as the axis of AA, were added to TA, distributed \texttt{*}; the next item in this list then applies. \item If the \textit{section-subscript} is a \textit{subscript-triplet} and the array axis corresponds to an axis of the template TA, then the distribution format of the axis of TA must be the same as the distribution format of the corresponding axis of TD, and corresponding elements of A along its axis and D along its axis must align to corresponding positions on the axes of TA and TD, respectively. \end{itemize} \item Otherwise, the dummy argument's template TD is its natural template. In this case the template TA of the actual must be coextensive with the array AA along any axes having a distribution format other than ``\texttt{*}.'' In such a case, for each \textit{section-subscript} of A, the following must hold: \begin{itemize} \item If the \textit{section-subscript} is scalar and the array axis is collapsed (as by an \texttt{ALIGN} directive) then no entry may appear in the distribution for TD. \item If the \textit{section-subscript} is scalar and the array axis corresponds to an axis of TA distributed \texttt{*}, then no entry may appear in the distribution for the TD. \item If the \textit{section-subscript} is a \textit{subscript-triplet} and the array axis is collapsed (as by an \texttt{ALIGN} directive), then \texttt{*} must appear in the distribution for TD. Further, the \textit{stride} of the \textit{subscript-triplet} must be 1. \item If the \textit{section-subscript} is a \textit{subscript-triplet} and the array axis corresponds to an axis of TA distributed \texttt{*}, then \texttt{*} must appear in the distribution for TD. Further, consecutive elements of the actual argument A along the array axis must be aligned with consecutive elements of TA along its corresponding axis. \item If the \textit{section-subscript} is a \textit{subscript-triplet} \texttt{\(l\):\(u\):\(s\)} and the array axis corresponds to an axis of TA distributed \texttt{BLOCK(\(n\))} (which might have been specified as simply \texttt{BLOCK}, but there will be some \(n\) that describes the resulting distribution), then the distribution of the axis of TD also must be \texttt{BLOCK(\(n\))} (or \texttt{BLOCK}, as above). Similarly for \texttt{CYCLIC(\(n\))} and \texttt{CYCLIC}. (This just repeats a constraint previously mentioned that the distribution formats of these axes of TA and TD have to be the same.) Note that the constraints that apply to this case require that \(l\) must be the declared lower bound, \(u\) must be the declared upper bound, and \(s\) must be 1; equivalently, the triplet \texttt{\(l\):\(u\):\(s\)} could simply have been written as \texttt{:}. \end{itemize} Note that when the dummy has a natural template, it would be non-conforming to have a scalar \textit{section-subscript} on an array axis corresponding to an axis of TA not distributed \texttt{*}. \end{enumerate} Here is an example: The main program has a two-dimensional array \texttt{TROGGS}, which is to be processed by a subroutine one column at a time. (Perhaps processing the entire array at once would require prohibitive amounts of temporary space.) Each column is to be distributed across many processors. \CODE REAL TROGGS(1024,473) !HPF$ PROCESSORS PMAIN(8) !HPF$ DISTRIBUTE TROGGS(BLOCK,*) ONTO PMAIN DO J=1,473 CALL WILD_THING(TROGGS(:,J)) END DO \EDOC Each column of {\tt TROGGS} has a {\tt BLOCK} distribution. The rules listed above mean that the lack of an explicit interface is legal in this case provided the programmer writes directives in the subprogram as follows (either a descriptive or a prescriptive form of the distribution directive is correct): \CODE SUBROUTINE WILD_THING(GROOVY) REAL GROOVY(1024) !HPF$ PROCESSORS PSUB(8) !HPF$ DISTRIBUTE GROOVY *(BLOCK) ONTO *PSUB \EDOC \begin{users} These conditions are complex. The important things to realize are these: \begin{itemize} \item You don't have to read any of this if you have an explicit interface. So if there is any doubt in your mind, just make sure you have an explicit interface. \item When there is an explicit interface, there are other kinds of pairs of mappings that can legitimately be said to be ``the same''. These restrictions apply only when there is no explicit interface. \end{itemize} \end{users} \section{Restrictions on Pointers and Targets} \label{mapsub:pointers} If, on invocation of a procedure P: (a)~a dummy argument has the \texttt{TARGET} attribute, and (b)~the corresponding actual argument has the \texttt{TARGET} attribute and is not an array section with a vector subscript (and therefore is an object A or a section of an array A), then the program is not HPF-conforming unless: \begin{enumerate} \item No remapping of the actual argument occurs during the call; or \item the remainder of program execution would be unaffected if \begin{enumerate} \item each pointer associated with any portion of the dummy argument or with any portion of A during execution of P were to acquire undefined pointer association status on exit from P; and \item each pointer associated with any portion of A before the call were to acquire undefined pointer association status on entry to P and, if not reassigned during execution of P, were to be restored on exit to the pointer association status it had before entry. \end{enumerate} \end{enumerate} Note that if a dummy argument has the \texttt{TARGET} attribute and no explicit mapping attributes, then the \texttt{INHERIT} attribute is implicitly assumed (see section~\ref{INHERIT-SECTION}); therefore no remapping occurs for such a dummy argument and there is no problem. \begin{rationale} These restrictions are made in order to support the following part of the Fortran standard (in Section 12.4.1.1 of that document) in the face of implicit remapping across the subprogram interface: \begin{quote} If the dummy argument does not have the \texttt{TARGET} or \texttt{POINTER} attribute, any pointers associated with the actual argument do not become associated with the corresponding dummy argument on invocation of the procedure. If the dummy argument has the \texttt{TARGET} attribute and the corresponding actual argument has the \texttt{TARGET} attribute but is not an array section with a vector subscript: \begin{enumerate} \item Any pointers associated with the actual argument become associated with the corresponding dummy argument on invocation of the procedure. \item When execution of the procedure completes, any pointers associated with the dummy argument remain associated with the actual argument. \end{enumerate} If the dummy argument has the \texttt{TARGET} attribute and the corresponding actual argument does not have the \texttt{TARGET} attribute or is an array section with a vector subscript, any pointers associated with the dummy argument become undefined when execution of the procedure completes. \end{quote} \end{rationale} \section{Argument Passing and Sequence Association} For actual arguments in a procedure call, Fortran allows an array element (scalar) to be associated with a dummy argument that is an array. It furthermore allows the shape of a dummy argument to differ from the shape of the corresponding actual array argument, in effect reshaping the actual argument via the subroutine call. Storage sequence properties of Fortran are used to identify the values of the dummy argument. This feature, carried over from FORTRAN 77, has been widely used to pass starting addresses of subarrays, rows or columns of a larger array, to procedures. For HPF arrays that are potentially mapped across processors, this feature is not fully supported. \subsection{Sequence Association Rules} \begin{enumerate} \item When an array element or the name of an assumed-size array is used as an actual argument, the associated dummy argument must be a scalar or specified to be a sequential array. An array-element designator of a nonsequential array must not be associated with a dummy array argument. \item When an actual argument is an array or array section and the corresponding dummy argument differs from the actual argument in shape, then the dummy argument must be declared sequential and the actual array argument must be sequential. \item An object of type character (scalar or array) is nonsequential if it conforms to the requirements of Definition~\ref{seq-var} of Section~\ref{sequence-defs}. If the length of an explicit-length character dummy argument differs from the length of the actual argument, then both the actual and dummy arguments must be sequential. \item Without an explicit interface, a sequential actual may not be associated with a nonsequential dummy and a nonsequential actual may not be associated with a sequential dummy. (This item merely repeats part of Section~\ref{mapsub:ExplicitInterfaces} \end{enumerate} \subsection{Discussion of Sequence Association} When the shape of the dummy array argument and its associated actual array argument differ, the actual argument must not be an expression. There is no HPF mechanism for declaring that the value of an array-valued expression is sequential. In order to associate such an expression as an actual argument with a dummy argument of different rank, the actual argument must first be assigned to a named array variable that is forced to be sequential according to Definition~\ref{seq-var} of Section~\ref{sequence-defs}. \subsection{Examples of Sequence Association} Given the following subroutine fragment: \CODE SUBROUTINE HOME (X) DIMENSION X (20,10) \EDOC By rule 1 \CODE CALL HOME (ET (2,1)) \EDOC is legal only if \texttt{X} is declared sequential in \texttt{HOME} and \texttt{ET} is sequential in the calling routine. Likewise, by rule 2 and 4 \CODE CALL HOME (ET) \EDOC requires either that \texttt{ET} and \texttt{X} are both sequential arrays or that \texttt{ET} and \texttt{X} have the same shape and have the same sequence attribute. Rule 3 addresses a special consideration for objects of type character. Change of the length of character objects across a call, as in \CODE CHARACTER (LEN=44) one_long_word one_long_word = 'Chargoggagoggmanchaugagoggchaubunagungamaugg' CALL webster(one_long_word) SUBROUTINE webster(short_dictionary) CHARACTER (LEN=4) short_dictionary (11) !Note that short_dictionary(3) is 'agog', for example \EDOC \noindent is conceptually legal in FORTRAN 77 and Fortran 95. In HPF, both the actual argument and dummy argument must be sequential. (By the way, ``Chargoggagoggmanchaugagoggchaubunagungamaugg'' is the original Nipmuc name for what is now called ``Lake Webster'' in Massachusetts.) --------------------------------------------------------------------------- To (un)subscribe to this list, send mail to hpff-doc-request@cs.rice.edu. Leave the subject line blank, and in the body put the line (un)subscribe --------------------------------------------------------------------------- From owner-hpff-doc Thu Jun 6 13:38:20 1996 Received: (from daemon@localhost) by cs.rice.edu (8.7.1/8.7.1) id NAA27526 for hpff-doc-out; Thu, 6 Jun 1996 13:38:20 -0500 (CDT) Date: Thu, 6 Jun 1996 13:38:20 -0500 (CDT) Message-Id: <199606061838.NAA27526@cs.rice.edu> From: Piyush Mehrotra Subject: hpff-doc: mapping-base.tex Phone: (804)864-2188 Fax: (804)864-6134 WWW: http://www.icase.edu/~pm Address: ICASE MS 132C NASA Langley Research Center Hampton VA 23681 Sender: owner-hpff-doc Precedence: bulk --------------------------------------------------------------------------- hpff-doc@cs.rice.edu is a mailing list for HPF 2.0 language specification authors and editors. Instructions for adding or deleting yourself from this list appear at the bottom of this message. --------------------------------------------------------------------------- % File: mapping-base.tex % Contents: % Mapping constructs for local variables for HPF 2.0 document, % including % DISTRIBUTE % ALIGN % SEQUENCE % some of pointer mappings % equivalence of mappings % Revision history: % May-10-96 Created by Charles Koelbel, Rice University % (from HPF 1.1 document and HPF 2.0 proposals) % May-22-96 Chapter updated by Piyush Mehrotra % \chapter{Data Mapping} \label{ch-mapping-base} {\em Comments on this chpater should be directed to Piyush Mehrotra ({\tt pm@icase.edu}), Carl Offner ({\tt offner@hpc.pko.dec.com}), Guy Steele ({\tt Guy.Steele@east.sun.com}) and {\tt hpff-doc@cs.rice.edu}. Please use ``{\tt Comments on Mapping}'' as the {\tt Subject:} line. \par } \begin{new} {\em This chapter is based on the old chapter on Data Mappings. I have removed all references to dynamic remapping of data ({\tt DYNAMIC, REDISTRIBUTE} and {\tt REALIGN}), including those in the model section. Also removed all references to mapping of aggregate covers and sequential objects from t eh section on storage and sequence association. I have also replaced all references to Fortran 90 with plain Fortran. -- PM} \end{new} HPF data alignment and distributions directives allow the programmer to advise the compiler how to assign array elements to processor memories. This chapter discusses the basic data mapping features applicable, particularly those that are meaningful within a single scoping unit. Chapter~\ref{ch-mapping-subr} will discuss mapping features that apply when mapped variables are passed as subprogram arguments. \section{Model} HPF adds directives to Fortran to allow the user to advise the compiler on the allocation of data objects to processor memories. The model is that there is a two-level mapping of data objects to memory regions, referred to as ``abstract processors.'' Data objects (typically array elements) are first {\it aligned} relative to one another; this group of arrays is then {\it distributed} onto a rectilinear arrangement of abstract processors. (The implementation then uses the same number, or perhaps some smaller number, of physical processors to implement these abstract processors. This mapping of abstract processors to physical processors is implementation-dependent.) The following diagram illustrates the model: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(600,240)(0,30) \thicklines \put(100,150){\circle{50}} \put(242,150){\circle{50}} \put(383,150){\circle{50}} \put(525,150){\circle{50}} \put(125,150){\vector(1,0){92}} \put(267,150){\vector(1,0){91}} \put(408,150){\vector(1,0){92}} \put(56,190){\shortstack{Arrays or\strut\\other objects\strut}} \put(192,190){\shortstack{Group of\strut\\aligned objects\strut}} \put(328,190){\shortstack{Abstract\strut\\processors as a\strut \\user-declared\strut\\Cartesian mesh\strut}} \put(494,190){\shortstack{Physical\strut\\processors\strut}} \put(150,100){\tt ALIGN} \put(270,100){\tt DISTRIBUTE} \put(400,50){\shortstack{Optional\strut\\implementation-\strut \\dependent\strut\\directive\strut}} \end{picture} \end{center} The underlying assumptions are that an operation on two or more data objects is likely to be carried out much faster if they all reside in the same processor, and that it may be possible to carry out many such operations concurrently if they can be performed on different processors. Fortran provides a number of features, notably array syntax, that make it easy for a compiler to determine that many operations may be carried out concurrently. The HPF directives provide a way to inform the compiler of the recommendation that certain data objects should reside in the same processor: if two data objects are mapped (via the two-level mapping of alignment and distribution) to the same abstract processor, it is a strong recommendation to the implementation that they ought to reside in the same physical processor. There is also a provision for recommending that a data object be stored in multiple locations, which may complicate any updating of the object but makes it faster for multiple processors to read the object. There is a clear separation between directives that serve as specification statements and directives that serve as executable statements (in the sense of the Fortran standards). Specification statements are carried out on entry to a program unit, as if all at once; only then are executable statements carried out. (While it is often convenient to think of specification statements as being handled at compile time, some of them contain specification expressions, which are permitted to depend on run-time quantities such as dummy arguments, and so the values of these expressions may not be available until run time, specifically the very moment that program control enters the scoping unit.) The basic concept is that every array (indeed, every object) is created with {\em some\/} alignment to an entity, which in turn has {\em some\/} distribution onto {\em some\/} arrangement of abstract processors. If the specification statements contain explicit specification directives specifying the alignment of an array {\tt A} with respect to another array {\tt B}, then the distribution of {\tt A} will be dictated by the distribution of {\tt B}; otherwise, the distribution of {\tt A} itself may be specified explicitly. In either case, any such explicit declarative information is used when the array is created. \begin{implementors} This model gives a better picture of the actual amount of work that needs to be done than a model that says ``the array is created in some default location, and then realigned and/or redistributed if there is an explicit directive.'' Using {\tt ALIGN} and {\tt DISTRIBUTE} specification directives doesn't have to cause any more work at run time than using the implementation defaults. \end{implementors} In the case of an allocatable object, we say that the object is created whenever it is allocated. Specification directives for allocatable objects may appear in the {\it specification-part} of a program unit, but take effect each time the array is created, rather than on entry to the scoping unit. Alignment is considered an {\em attribute\/} (in the Fortran sense) of a data object. If an object {\tt A} is aligned with an object {\tt B}, which in turn is already aligned to an object {\tt C}, this is regarded as an alignment of {\tt A} with {\tt C} directly, with {\tt B} serving only as an intermediary at the time of specification. We say that {\tt A} is {\it immediately aligned} with {\tt B} but {\it ultimately aligned} with {\tt C}. If an object is not explicitly aligned with another object, we say that it is ultimately aligned with itself. The alignment relationships form a tree with everything ultimately aligned to the object at the root of the tree; however, the tree is always immediately ``collapsed'' so that every object is related directly to the root. Every object which is the root of an alignment tree has an associated {\em template\/} or index space. Typically, this template has the same rank and size in each dimension as the object associated with it. (The most important exception to this rule is dummy arguments with the {\tt INHERIT} attribute, described in Section~\ref{INHERIT-SECTION}.) We often refer to ``the template for an array,'' which means the template of the object to which the array is ultimately aligned. (When an explicit {\tt TEMPLATE} (see Section~\ref{TEMPLATE-SECTION}) is used, this may be simply the template to which the array is explicitly aligned.) The {\em distribution\/} step of the HPF model technically applies to the template of an array, although because of the close relationship noted above we often speak loosely of the distribution of an array. Distribution partitions the template among a set of abstract processors according to a given pattern. The combination of alignment (from arrays to templates) and distribution (from templates to processors) thus determines the relationship of an array to the processors; we refer to this relationship as the {\em mapping\/} of the array. (These remarks also apply to a scalar, which may be regarded as having an index space whose sole position is indicated by an empty list of subscripts.) Every object is created as if according to some complete set of specification directives; if the program does not include complete specifications for the mapping of some object, the compiler provides defaults. By default an object is not aligned with any other object; it is ultimately aligned with itself. The default distribution is implementation-dependent, but must be expressible as explicit directives for that implementation. Identically declared objects need not be provided with identical default distribution specifications; the compiler may, for example, take into account the contexts in which objects are used in executable code. The programmer may force identically declared objects to have identical distributions by specifying such distributions explicitly. (On the other hand, identically declared processor arrangements {\it are} guaranteed to represent ``the same processors arranged the same way.'' This is discussed in more detail in Section~\ref{PROCESSORS-SECTION}.) Sometimes it is desirable to consider a large index space with which several smaller arrays are to be aligned, but not to declare any array that spans the entire index space. HPF allows one to declare a {\tt TEMPLATE}, which is like an array whose elements have no content and therefore occupy no storage; it is merely an abstract index space that can be distributed and with which arrays may be aligned. An object is considered to be {\it explicitly mapped} if it appears in an HPF mapping directive within the scoping unit in which it is declared; otherwise it is {\it implicitly mapped}. A mapping directive is an {\tt ALIGN}, or {\tt DISTRIBUTE}, or {\tt INHERIT} directive, or any directive that confers an alignment, a distribution, or the {\tt INHERIT} attribute. \section{Syntax of Data Alignment and Distribution Directives} Specification directives in HPF have two forms: specification statements, analogous to the {\tt DIMENSION} and {\tt ALLOCATABLE} statements of Fortran; and an attribute form analogous to type declaration statements in Fortran using the ``{\tt ::}'' punctuation. The attribute form allows more than one attribute to be described in a single directive. HPF goes beyond Fortran in not requiring that the first attribute, or indeed any of them, be a type specifier. \BNF combined-directive \IS combined-attribute-list :: combined-decl-list combined-attribute \IS ALIGN align-attribute-stuff \OR DISTRIBUTE dist-attribute-stuff \OR INHERIT \OR TEMPLATE \OR PROCESSORS \OR DIMENSION ( explicit-shape-spec-list ) combined-decl-list \IS hpf-entity [(explicit-shape spec-list)] \OR object-name hpf-entity \IS processors-name \OR template-name \FNB The {\tt INHERIT} attribute is related to subroutine call conventions and will be discussed in Section~\ref{ch-mapping-subr}. \begin{constraints} \item The same {\it combined-attribute} must not appear more than once in a given {\it combined-directive}. \item If the {\tt DIMENSION} attribute appears in a {\it combined-directive}, any entity to which it applies must be declared with the HPF {\tt TEMPLATE} or {\tt PROCESSORS} type specifier. \end{constraints} The following rules constrain the declaration of various attributes, whether in separate directives or in a combined-directive. If the {\tt DISTRIBUTE} attribute is present, then every name declared in the {\it combined-decl-list} is considered to be a {\it distributee} and is subject to the constraints listed in section~\ref{DISTRIBUTE-SECTION}. If the {\tt ALIGN} attribute is present, then every name declared in the {\it entity-decl-list} is considered to be an {\it alignee} and is subject to the constraints listed in section~\ref{ALIGN-SECTION}. The HPF keywords {\tt PROCESSORS} and {\tt TEMPLATE} play the role of type specifiers in declaring processor arrangements and templates. The HPF keywords {\tt ALIGN}, {\tt DISTRIBUTE}, and {\tt INHERIT} play the role of attributes. Attributes referring to processor arrangements, to templates, or to entities with other types (such as {\tt REAL}) may be combined in an HPF directive without having the type specifier appear. No entity may be given a particular attribute more than once. Dimension information may be specified after an {\it hpf-entity} or in a {\tt DIMENSION} attribute. If both are present, the one after the {\it object-name} overrides the {\tt DIMENSION} attribute (this is consistent with the Fortran standard). For example, in: \CODE !HPF$ TEMPLATE,DIMENSION(64,64) :: A,B,C(32,32),D \EDOC {\tt A}, {\tt B}, and {\tt D} are \( 64 \times 64 \) templates; {\tt C} is \( 32 \times 32 \). Directives mapping a variable must be in the same module scoping unit where the variable is declared. If a specification expression includes a reference to the value of an element of an array specified in the same specification-part, any explicit mapping or {\tt INHERIT} attribute for the array must be completely specified in prior specification-directives. (This restriction is inspired by and extends section 7.1.6.2 of the Fortran standard, which states in part: If a specification expression includes a reference to the value of an element of an array specified in the same specification-part, the array bounds must be specified in a prior declaration.) A comment on asterisks: The asterisk character ``{\tt *}'' appears in the syntax rules for HPF alignment and distribution directives in three distinct roles: \begin{itemize} \item When a lone asterisk appears as a member of a parenthesized list, it indicates either a collapsed mapping, wherein many elements of an array may be mapped to the same abstract processor, or a replicated mapping, wherein each element of an array may be mapped to many abstract processors. See the syntax rules for {\it align-source} and {\it align-subscript} (see Section \ref{ALIGN-SECTION}) and for {\it dist-format} (see Section \ref{DISTRIBUTE-SECTION}). \item When an asterisk appears in an {\it align-subscript-use} expression, it represents the usual integer multiplication operator. \item When an asterisk appears before a left parenthesis ``{\tt (}'' or after the keyword {\tt WITH} or {\tt ONTO}, it indicates that the directive constitutes an assertion about the {\it current} mapping of a dummy argument on entry to a subprogram, rather than a request for a {\it desired} mapping of that dummy argument. This use of the asterisk may appear {\it only} in directives that apply to dummy arguments (see Chapter~\ref{ch-mapping-subr}). \end{itemize} The first two of these uses are discussed in this chapter. The last is a subprogram interface issue and will be detailed in Chapter~\ref{ch-mapping-subr}. \section{DISTRIBUTE Directive} \label{DISTRIBUTE-SECTION} The {\tt DISTRIBUTE} directive specifies a mapping of data objects to abstract processors in a processor arrangement. For example, \CODE REAL SALAMI(10000) !HPF$ DISTRIBUTE SALAMI(BLOCK) \EDOC specifies that the array {\tt SALAMI} should be distributed across some set of abstract processors by slicing it uniformly into blocks of contiguous elements. If there are 50 processors, the directive implies that the array should be divided into groups of 200 elements, with {\tt SALAMI(1:200)} mapped to the first processor, {\tt SALAMI(201:400)} mapped to the second processor, and so on. If there is only one processor, the entire array is mapped to that processor as a single block of 10000 elements. The block size may be specified explicitly: \CODE REAL WEISSWURST(10000) !HPF$ DISTRIBUTE WEISSWURST(BLOCK(256)) \EDOC This specifies that groups of exactly 256 elements should be mapped to successive abstract processors. (There must be at least \( \lceil 10000/256 \rceil = 40 \) abstract processors if the directive is to be satisfied. The fortieth processor will contain a partial block of only 16 elements, namely {\tt WEISSWURST(9985:10000)}.) HPF also provides a cyclic distribution format: \CODE REAL DECK_OF_CARDS(52) !HPF$ DISTRIBUTE DECK_OF_CARDS(CYCLIC) \EDOC If there are 4 abstract processors, the first processor will contain {\tt DECK_OF_CARDS(1:49:4)}, the second processor will contain {\tt DECK_OF_CARDS(2:50:4)}, the third processor will contain {\tt DECK_OF_CARDS(3:51:4)}, and the fourth processor will contain {\tt DECK_OF_CARDS(4:52:4)}. Successive array elements are dealt out to successive abstract processors in round-robin fashion. Distributions may be specified independently for each dimension of a multidimensional array: \CODE INTEGER CHESS_BOARD(8,8), GO_BOARD(19,19) !HPF$ DISTRIBUTE CHESS_BOARD(BLOCK, BLOCK) !HPF$ DISTRIBUTE GO_BOARD(CYCLIC,*) \EDOC The {\tt CHESS_BOARD} array will be carved up into contiguous rectangular patches, which will be distributed onto a two-dimensional arrangement of abstract processors. The {\tt GO_BOARD} array will have its rows distributed cyclically over a one-dimensional arrangement of abstract processors. (The ``{\tt *}'' specifies that {\tt GO_BOARD} is not to be distributed along its second axis; thus an entire row is to be distributed as one object. This is sometimes called ``on-processor'' distribution.) The {\tt DISTRIBUTE} directive may appear only in the {\it specification-part} of a scoping unit and can contain only a {\it specification-expr} as the argument to a {\tt BLOCK} or {\tt CYCLIC} option. Formally, the syntax of the {\tt DISTRIBUTE} directive is: \BNF distribute-directive \IS DISTRIBUTE distributee dist-directive-stuff dist-directive-stuff \IS dist-format-clause [ dist-onto-clause ] dist-attribute-stuff \IS dist-directive-stuff \OR dist-onto-clause distributee \IS object-name \OR template-name dist-format-clause \IS ( dist-format-list ) \OR * ( dist-format-list ) \OR * dist-format \IS BLOCK [ ( int-expr ) ] \OR CYCLIC [ ( int-expr ) ] \OR * dist-onto-clause \IS ONTO dist-target dist-target \IS processors-name \OR * processors-name \OR * \FNB The full syntax is given here for completeness, however some of the forms will only be discussed in Chapter~\ref{ch-mapping-subr}. These ``interprocedural'' forms are: \begin{itemize} \item The last two options of rule~\ref{dist-format-clause-rule} (containing the {\tt *} form) \item The last two options of rule~\ref{dist-target-rule} (containing the {\tt *} form) \end{itemize} \begin{constraints} \item An {\it object-name} mentioned as a {\it distributee} must be a simple name and not a subobject designator. \item An {\it object-name} mentioned as a {\it distributee} may not appear as an {\it alignee}. \item An {\it object-name} mentioned as a {\it distributee} may not have the {\tt POINTER} attribute. \item If a {\it dist-format-list} is specified, its length must equal the rank of each {\it distributee}. \item If both a {\it dist-format-list} and a {\it processors-name} appear, the number of elements of the {\it dist-format-list} that are not ``{\tt *}'' must equal the rank of the named processor arrangement. \item If a {\it processors-name} appears but not a {\it dist-format-list}, the rank of each {\it distributee} must equal the rank of the named processor arrangement. \item If either the {\it dist-format-clause} or the {\it dist-target} in a {\tt DISTRIBUTE} directive begins with ``{\tt *}'' then every {\it distributee} must be a dummy argument. \item Any {\it int-expr} appearing in a {\it dist-format} of a {\tt DISTRIBUTE} directive must be a {\it specification-expr}. \end{constraints} Note that the possibility of a {\tt DISTRIBUTE} directive of the form \ICODE !HPF$ DISTRIBUTE dist-attribute-stuff :: distributee-list \EDOC is covered by syntax rule \ref{combined-directive-rule} for a {\it combined-directive}. Examples: \CODE !HPF$ DISTRIBUTE D1(BLOCK) !HPF$ DISTRIBUTE (BLOCK,*,BLOCK) ONTO SQUARE:: D2,D3,D4 \EDOC The meanings of the alternatives for {\it dist-format} are given below. Define the ceiling division function {\tt CD(J,K)~=~(J+K-1)/K} (using Fortran integer arithmetic with truncation toward zero.) Define the ceiling remainder function {\tt CR(J,K)~=~J-K*CD(J,K)}. The dimensions of a processor arrangement appearing as a {\it dist-target} are said to {\it correspond} in left-to-right order with those dimensions of a {\it distributee} for which the corresponding {\it dist-format} is not {\tt *}. In the example above, processor arrangement {\tt SQUARE} must be two-dimensional; its first dimension corresponds to the first dimensions of {\tt D2}, {\tt D3}, and {\tt D4} and its second dimension corresponds to the third dimensions of {\tt D2}, {\tt D3}, and {\tt D4}. Let \(d\) be the size of a {\it distributee} in a certain dimension and let \(p\) be the size of the processor arrangement in the corresponding dimension. For simplicity, assume all dimensions have a lower bound of 1. Then {\tt BLOCK(\(m\))} means that a {\it distributee} position whose index along that dimension is \(j\) is mapped to an abstract processor whose index along the corresponding dimension of the processor arrangement is {\tt CD(\(j\),\(m\))} (note that \(m \times p \geq d\) must be true), and is position number {\tt \(m\)+CR(\(j\),\(m\))} among positions mapped to that abstract processor. The first {\it distributee} position in abstract processor \(k\) along that axis is position number {\tt 1+\(m\)*(\(k\)-1)}. The block size \(m\) must be a positive integer. {\tt BLOCK} by definition means the same as {\tt BLOCK(CD(\(d\),\(p\)))}. {\tt CYCLIC(\(m\))} means that a {\it distributee} position whose index along that dimension is \(j\) is mapped to an abstract processor whose index along the corresponding dimension of the processor arrangement is {\tt 1+MODULO(CD(\(j\),\(m\))-1,\(p\))}. The first {\it distributee} position in abstract processor \(k\) along that axis is position number {\tt 1+\(m\)*(\(k\)-1)}. The block size \(m\) must be a positive integer. {\tt CYCLIC} by definition means the same as {\tt CYCLIC(1)}. {\tt CYCLIC(\(m\))} and {\tt BLOCK(\(m\))} imply the same distribution when \( m \times p \geq d \), but {\tt BLOCK(\(m\))} additionally asserts that the distribution will not wrap around in a cyclic manner, which a compiler cannot determine at compile time if \(m\) is not constant. Note that {\tt CYCLIC} and {\tt BLOCK} (without argument expressions) do not imply the same distribution unless \( p \geq d \), a degenerate case in which the block size is 1 and the distribution does not wrap around. Suppose that we have 16 abstract processors and an array of length 100: \CODE !HPF$ PROCESSORS SEDECIM(16) REAL CENTURY(100) \EDOC Distributing the array {\tt BLOCK} (which in this case would mean the same as {\tt BLOCK(7)}): \CODE !HPF$ DISTRIBUTE CENTURY(BLOCK) ONTO SEDECIM \EDOC results in this mapping of array elements onto abstract processors: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,200)(0,0) \put(0,175){\makebox(35,25){\small\rm 1}} \put(35,175){\makebox(35,25){\small\rm 2}} \put(70,175){\makebox(35,25){\small\rm 3}} \put(105,175){\makebox(35,25){\small\rm 4}} \put(140,175){\makebox(35,25){\small\rm 5}} \put(175,175){\makebox(35,25){\small\rm 6}} \put(210,175){\makebox(35,25){\small\rm 7}} \put(245,175){\makebox(35,25){\small\rm 8}} \put(280,175){\makebox(35,25){\small\rm 9}} \put(315,175){\makebox(35,25){\small\rm 10}} \put(350,175){\makebox(35,25){\small\rm 11}} \put(385,175){\makebox(35,25){\small\rm 12}} \put(420,175){\makebox(35,25){\small\rm 13}} \put(455,175){\makebox(35,25){\small\rm 14}} \put(490,175){\makebox(35,25){\small\rm 15}} \put(525,175){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){6}{\line(1,0){560}} \thicklines \multiput(0,0)(0,175){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){175}} \put(0,150){\makebox(35,25){\tt 1}} \put(0,125){\makebox(35,25){\tt 2}} \put(0,100){\makebox(35,25){\tt 3}} \put(0,75){\makebox(35,25){\tt 4}} \put(0,50){\makebox(35,25){\tt 5}} \put(0,25){\makebox(35,25){\tt 6}} \put(0,0){\makebox(35,25){\tt 7}} \put(35,150){\makebox(35,25){\tt 8}} \put(35,125){\makebox(35,25){\tt 9}} \put(35,100){\makebox(35,25){\tt 10}} \put(35,75){\makebox(35,25){\tt 11}} \put(35,50){\makebox(35,25){\tt 12}} \put(35,25){\makebox(35,25){\tt 13}} \put(35,0){\makebox(35,25){\tt 14}} \put(70,150){\makebox(35,25){\tt 15}} \put(70,125){\makebox(35,25){\tt 16}} \put(70,100){\makebox(35,25){\tt 17}} \put(70,75){\makebox(35,25){\tt 18}} \put(70,50){\makebox(35,25){\tt 19}} \put(70,25){\makebox(35,25){\tt 20}} \put(70,0){\makebox(35,25){\tt 21}} \put(105,150){\makebox(35,25){\tt 22}} \put(105,125){\makebox(35,25){\tt 23}} \put(105,100){\makebox(35,25){\tt 24}} \put(105,75){\makebox(35,25){\tt 25}} \put(105,50){\makebox(35,25){\tt 26}} \put(105,25){\makebox(35,25){\tt 27}} \put(105,0){\makebox(35,25){\tt 28}} \put(140,150){\makebox(35,25){\tt 29}} \put(140,125){\makebox(35,25){\tt 30}} \put(140,100){\makebox(35,25){\tt 31}} \put(140,75){\makebox(35,25){\tt 32}} \put(140,50){\makebox(35,25){\tt 33}} \put(140,25){\makebox(35,25){\tt 34}} \put(140,0){\makebox(35,25){\tt 35}} \put(175,150){\makebox(35,25){\tt 36}} \put(175,125){\makebox(35,25){\tt 37}} \put(175,100){\makebox(35,25){\tt 38}} \put(175,75){\makebox(35,25){\tt 39}} \put(175,50){\makebox(35,25){\tt 40}} \put(175,25){\makebox(35,25){\tt 41}} \put(175,0){\makebox(35,25){\tt 42}} \put(210,150){\makebox(35,25){\tt 43}} \put(210,125){\makebox(35,25){\tt 44}} \put(210,100){\makebox(35,25){\tt 45}} \put(210,75){\makebox(35,25){\tt 46}} \put(210,50){\makebox(35,25){\tt 47}} \put(210,25){\makebox(35,25){\tt 48}} \put(210,0){\makebox(35,25){\tt 49}} \put(245,150){\makebox(35,25){\tt 50}} \put(245,125){\makebox(35,25){\tt 51}} \put(245,100){\makebox(35,25){\tt 52}} \put(245,75){\makebox(35,25){\tt 53}} \put(245,50){\makebox(35,25){\tt 54}} \put(245,25){\makebox(35,25){\tt 55}} \put(245,0){\makebox(35,25){\tt 56}} \put(280,150){\makebox(35,25){\tt 57}} \put(280,125){\makebox(35,25){\tt 58}} \put(280,100){\makebox(35,25){\tt 59}} \put(280,75){\makebox(35,25){\tt 60}} \put(280,50){\makebox(35,25){\tt 61}} \put(280,25){\makebox(35,25){\tt 62}} \put(280,0){\makebox(35,25){\tt 63}} \put(315,150){\makebox(35,25){\tt 64}} \put(315,125){\makebox(35,25){\tt 65}} \put(315,100){\makebox(35,25){\tt 66}} \put(315,75){\makebox(35,25){\tt 67}} \put(315,50){\makebox(35,25){\tt 68}} \put(315,25){\makebox(35,25){\tt 69}} \put(315,0){\makebox(35,25){\tt 70}} \put(350,150){\makebox(35,25){\tt 71}} \put(350,125){\makebox(35,25){\tt 72}} \put(350,100){\makebox(35,25){\tt 73}} \put(350,75){\makebox(35,25){\tt 74}} \put(350,50){\makebox(35,25){\tt 75}} \put(350,25){\makebox(35,25){\tt 76}} \put(350,0){\makebox(35,25){\tt 77}} \put(385,150){\makebox(35,25){\tt 78}} \put(385,125){\makebox(35,25){\tt 79}} \put(385,100){\makebox(35,25){\tt 80}} \put(385,75){\makebox(35,25){\tt 81}} \put(385,50){\makebox(35,25){\tt 82}} \put(385,25){\makebox(35,25){\tt 83}} \put(385,0){\makebox(35,25){\tt 84}} \put(420,150){\makebox(35,25){\tt 85}} \put(420,125){\makebox(35,25){\tt 86}} \put(420,100){\makebox(35,25){\tt 87}} \put(420,75){\makebox(35,25){\tt 88}} \put(420,50){\makebox(35,25){\tt 89}} \put(420,25){\makebox(35,25){\tt 90}} \put(420,0){\makebox(35,25){\tt 91}} \put(455,150){\makebox(35,25){\tt 92}} \put(455,125){\makebox(35,25){\tt 93}} \put(455,100){\makebox(35,25){\tt 94}} \put(455,75){\makebox(35,25){\tt 95}} \put(455,50){\makebox(35,25){\tt 96}} \put(455,25){\makebox(35,25){\tt 97}} \put(455,0){\makebox(35,25){\tt 98}} \put(490,150){\makebox(35,25){\tt 99}} \put(490,125){\makebox(35,25){\tt 100}} \end{picture} \end{center} Distributing the array {\tt BLOCK(8)}: \CODE !HPF$ DISTRIBUTE CENTURY(BLOCK(8)) ONTO SEDECIM \EDOC results in this mapping of array elements onto abstract processors: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,225)(0,0) \put(0,200){\makebox(35,25){\small\rm 1}} \put(35,200){\makebox(35,25){\small\rm 2}} \put(70,200){\makebox(35,25){\small\rm 3}} \put(105,200){\makebox(35,25){\small\rm 4}} \put(140,200){\makebox(35,25){\small\rm 5}} \put(175,200){\makebox(35,25){\small\rm 6}} \put(210,200){\makebox(35,25){\small\rm 7}} \put(245,200){\makebox(35,25){\small\rm 8}} \put(280,200){\makebox(35,25){\small\rm 9}} \put(315,200){\makebox(35,25){\small\rm 10}} \put(350,200){\makebox(35,25){\small\rm 11}} \put(385,200){\makebox(35,25){\small\rm 12}} \put(420,200){\makebox(35,25){\small\rm 13}} \put(455,200){\makebox(35,25){\small\rm 14}} \put(490,200){\makebox(35,25){\small\rm 15}} \put(525,200){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){7}{\line(1,0){560}} \thicklines \multiput(0,0)(0,200){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){200}} \put(0,175){\makebox(35,25){\tt 1}} \put(0,150){\makebox(35,25){\tt 2}} \put(0,125){\makebox(35,25){\tt 3}} \put(0,100){\makebox(35,25){\tt 4}} \put(0,75){\makebox(35,25){\tt 5}} \put(0,50){\makebox(35,25){\tt 6}} \put(0,25){\makebox(35,25){\tt 7}} \put(0,0){\makebox(35,25){\tt 8}} \put(35,175){\makebox(35,25){\tt 9}} \put(35,150){\makebox(35,25){\tt 10}} \put(35,125){\makebox(35,25){\tt 11}} \put(35,100){\makebox(35,25){\tt 12}} \put(35,75){\makebox(35,25){\tt 13}} \put(35,50){\makebox(35,25){\tt 14}} \put(35,25){\makebox(35,25){\tt 15}} \put(35,0){\makebox(35,25){\tt 16}} \put(70,175){\makebox(35,25){\tt 17}} \put(70,150){\makebox(35,25){\tt 18}} \put(70,125){\makebox(35,25){\tt 19}} \put(70,100){\makebox(35,25){\tt 20}} \put(70,75){\makebox(35,25){\tt 21}} \put(70,50){\makebox(35,25){\tt 22}} \put(70,25){\makebox(35,25){\tt 23}} \put(70,0){\makebox(35,25){\tt 24}} \put(105,175){\makebox(35,25){\tt 25}} \put(105,150){\makebox(35,25){\tt 26}} \put(105,125){\makebox(35,25){\tt 27}} \put(105,100){\makebox(35,25){\tt 28}} \put(105,75){\makebox(35,25){\tt 29}} \put(105,50){\makebox(35,25){\tt 30}} \put(105,25){\makebox(35,25){\tt 31}} \put(105,0){\makebox(35,25){\tt 32}} \put(140,175){\makebox(35,25){\tt 33}} \put(140,150){\makebox(35,25){\tt 34}} \put(140,125){\makebox(35,25){\tt 35}} \put(140,100){\makebox(35,25){\tt 36}} \put(140,75){\makebox(35,25){\tt 37}} \put(140,50){\makebox(35,25){\tt 38}} \put(140,25){\makebox(35,25){\tt 39}} \put(140,0){\makebox(35,25){\tt 40}} \put(175,175){\makebox(35,25){\tt 41}} \put(175,150){\makebox(35,25){\tt 42}} \put(175,125){\makebox(35,25){\tt 43}} \put(175,100){\makebox(35,25){\tt 44}} \put(175,75){\makebox(35,25){\tt 45}} \put(175,50){\makebox(35,25){\tt 46}} \put(175,25){\makebox(35,25){\tt 47}} \put(175,0){\makebox(35,25){\tt 48}} \put(210,175){\makebox(35,25){\tt 49}} \put(210,150){\makebox(35,25){\tt 50}} \put(210,125){\makebox(35,25){\tt 51}} \put(210,100){\makebox(35,25){\tt 52}} \put(210,75){\makebox(35,25){\tt 53}} \put(210,50){\makebox(35,25){\tt 54}} \put(210,25){\makebox(35,25){\tt 55}} \put(210,0){\makebox(35,25){\tt 56}} \put(245,175){\makebox(35,25){\tt 57}} \put(245,150){\makebox(35,25){\tt 58}} \put(245,125){\makebox(35,25){\tt 59}} \put(245,100){\makebox(35,25){\tt 60}} \put(245,75){\makebox(35,25){\tt 61}} \put(245,50){\makebox(35,25){\tt 62}} \put(245,25){\makebox(35,25){\tt 63}} \put(245,0){\makebox(35,25){\tt 64}} \put(280,175){\makebox(35,25){\tt 65}} \put(280,150){\makebox(35,25){\tt 66}} \put(280,125){\makebox(35,25){\tt 67}} \put(280,100){\makebox(35,25){\tt 68}} \put(280,75){\makebox(35,25){\tt 69}} \put(280,50){\makebox(35,25){\tt 70}} \put(280,25){\makebox(35,25){\tt 71}} \put(280,0){\makebox(35,25){\tt 72}} \put(315,175){\makebox(35,25){\tt 73}} \put(315,150){\makebox(35,25){\tt 74}} \put(315,125){\makebox(35,25){\tt 75}} \put(315,100){\makebox(35,25){\tt 76}} \put(315,75){\makebox(35,25){\tt 77}} \put(315,50){\makebox(35,25){\tt 78}} \put(315,25){\makebox(35,25){\tt 79}} \put(315,0){\makebox(35,25){\tt 80}} \put(350,175){\makebox(35,25){\tt 81}} \put(350,150){\makebox(35,25){\tt 82}} \put(350,125){\makebox(35,25){\tt 83}} \put(350,100){\makebox(35,25){\tt 84}} \put(350,75){\makebox(35,25){\tt 85}} \put(350,50){\makebox(35,25){\tt 86}} \put(350,25){\makebox(35,25){\tt 87}} \put(350,0){\makebox(35,25){\tt 88}} \put(385,175){\makebox(35,25){\tt 89}} \put(385,150){\makebox(35,25){\tt 90}} \put(385,125){\makebox(35,25){\tt 91}} \put(385,100){\makebox(35,25){\tt 92}} \put(385,75){\makebox(35,25){\tt 93}} \put(385,50){\makebox(35,25){\tt 94}} \put(385,25){\makebox(35,25){\tt 95}} \put(385,0){\makebox(35,25){\tt 96}} \put(420,175){\makebox(35,25){\tt 97}} \put(420,150){\makebox(35,25){\tt 98}} \put(420,125){\makebox(35,25){\tt 99}} \put(420,100){\makebox(35,25){\tt 100}} \end{picture} \end{center} Distributing the array {\tt BLOCK(6)} is not HPF-conforming because \(6 \times 16 < 100\). Distributing the array {\tt CYCLIC} (which means exactly the same as {\tt CYCLIC(1)}): \CODE !HPF$ DISTRIBUTE CENTURY(CYCLIC) ONTO SEDECIM \EDOC results in this mapping of array elements onto abstract processors: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,200)(0,0) \put(0,175){\makebox(35,25){\small\rm 1}} \put(35,175){\makebox(35,25){\small\rm 2}} \put(70,175){\makebox(35,25){\small\rm 3}} \put(105,175){\makebox(35,25){\small\rm 4}} \put(140,175){\makebox(35,25){\small\rm 5}} \put(175,175){\makebox(35,25){\small\rm 6}} \put(210,175){\makebox(35,25){\small\rm 7}} \put(245,175){\makebox(35,25){\small\rm 8}} \put(280,175){\makebox(35,25){\small\rm 9}} \put(315,175){\makebox(35,25){\small\rm 10}} \put(350,175){\makebox(35,25){\small\rm 11}} \put(385,175){\makebox(35,25){\small\rm 12}} \put(420,175){\makebox(35,25){\small\rm 13}} \put(455,175){\makebox(35,25){\small\rm 14}} \put(490,175){\makebox(35,25){\small\rm 15}} \put(525,175){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){6}{\line(1,0){560}} \thicklines \multiput(0,0)(0,175){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){175}} \put(0,150){\makebox(35,25){\tt 1}} \put(35,150){\makebox(35,25){\tt 2}} \put(70,150){\makebox(35,25){\tt 3}} \put(105,150){\makebox(35,25){\tt 4}} \put(140,150){\makebox(35,25){\tt 5}} \put(175,150){\makebox(35,25){\tt 6}} \put(210,150){\makebox(35,25){\tt 7}} \put(245,150){\makebox(35,25){\tt 8}} \put(280,150){\makebox(35,25){\tt 9}} \put(315,150){\makebox(35,25){\tt 10}} \put(350,150){\makebox(35,25){\tt 11}} \put(385,150){\makebox(35,25){\tt 12}} \put(420,150){\makebox(35,25){\tt 13}} \put(455,150){\makebox(35,25){\tt 14}} \put(490,150){\makebox(35,25){\tt 15}} \put(525,150){\makebox(35,25){\tt 16}} \put(0,125){\makebox(35,25){\tt 17}} \put(35,125){\makebox(35,25){\tt 18}} \put(70,125){\makebox(35,25){\tt 19}} \put(105,125){\makebox(35,25){\tt 20}} \put(140,125){\makebox(35,25){\tt 21}} \put(175,125){\makebox(35,25){\tt 22}} \put(210,125){\makebox(35,25){\tt 23}} \put(245,125){\makebox(35,25){\tt 24}} \put(280,125){\makebox(35,25){\tt 25}} \put(315,125){\makebox(35,25){\tt 26}} \put(350,125){\makebox(35,25){\tt 27}} \put(385,125){\makebox(35,25){\tt 28}} \put(420,125){\makebox(35,25){\tt 29}} \put(455,125){\makebox(35,25){\tt 30}} \put(490,125){\makebox(35,25){\tt 31}} \put(525,125){\makebox(35,25){\tt 32}} \put(0,100){\makebox(35,25){\tt 33}} \put(35,100){\makebox(35,25){\tt 34}} \put(70,100){\makebox(35,25){\tt 35}} \put(105,100){\makebox(35,25){\tt 36}} \put(140,100){\makebox(35,25){\tt 37}} \put(175,100){\makebox(35,25){\tt 38}} \put(210,100){\makebox(35,25){\tt 39}} \put(245,100){\makebox(35,25){\tt 40}} \put(280,100){\makebox(35,25){\tt 41}} \put(315,100){\makebox(35,25){\tt 42}} \put(350,100){\makebox(35,25){\tt 43}} \put(385,100){\makebox(35,25){\tt 44}} \put(420,100){\makebox(35,25){\tt 45}} \put(455,100){\makebox(35,25){\tt 46}} \put(490,100){\makebox(35,25){\tt 47}} \put(525,100){\makebox(35,25){\tt 48}} \put(0,75){\makebox(35,25){\tt 49}} \put(35,75){\makebox(35,25){\tt 50}} \put(70,75){\makebox(35,25){\tt 51}} \put(105,75){\makebox(35,25){\tt 52}} \put(140,75){\makebox(35,25){\tt 53}} \put(175,75){\makebox(35,25){\tt 54}} \put(210,75){\makebox(35,25){\tt 55}} \put(245,75){\makebox(35,25){\tt 56}} \put(280,75){\makebox(35,25){\tt 57}} \put(315,75){\makebox(35,25){\tt 58}} \put(350,75){\makebox(35,25){\tt 59}} \put(385,75){\makebox(35,25){\tt 60}} \put(420,75){\makebox(35,25){\tt 61}} \put(455,75){\makebox(35,25){\tt 62}} \put(490,75){\makebox(35,25){\tt 63}} \put(525,75){\makebox(35,25){\tt 64}} \put(0,50){\makebox(35,25){\tt 65}} \put(35,50){\makebox(35,25){\tt 66}} \put(70,50){\makebox(35,25){\tt 67}} \put(105,50){\makebox(35,25){\tt 68}} \put(140,50){\makebox(35,25){\tt 69}} \put(175,50){\makebox(35,25){\tt 70}} \put(210,50){\makebox(35,25){\tt 71}} \put(245,50){\makebox(35,25){\tt 72}} \put(280,50){\makebox(35,25){\tt 73}} \put(315,50){\makebox(35,25){\tt 74}} \put(350,50){\makebox(35,25){\tt 75}} \put(385,50){\makebox(35,25){\tt 76}} \put(420,50){\makebox(35,25){\tt 77}} \put(455,50){\makebox(35,25){\tt 78}} \put(490,50){\makebox(35,25){\tt 79}} \put(525,50){\makebox(35,25){\tt 80}} \put(0,25){\makebox(35,25){\tt 81}} \put(35,25){\makebox(35,25){\tt 82}} \put(70,25){\makebox(35,25){\tt 83}} \put(105,25){\makebox(35,25){\tt 84}} \put(140,25){\makebox(35,25){\tt 85}} \put(175,25){\makebox(35,25){\tt 86}} \put(210,25){\makebox(35,25){\tt 87}} \put(245,25){\makebox(35,25){\tt 88}} \put(280,25){\makebox(35,25){\tt 89}} \put(315,25){\makebox(35,25){\tt 90}} \put(350,25){\makebox(35,25){\tt 91}} \put(385,25){\makebox(35,25){\tt 92}} \put(420,25){\makebox(35,25){\tt 93}} \put(455,25){\makebox(35,25){\tt 94}} \put(490,25){\makebox(35,25){\tt 95}} \put(525,25){\makebox(35,25){\tt 96}} \put(0,0){\makebox(35,25){\tt 97}} \put(35,0){\makebox(35,25){\tt 98}} \put(70,0){\makebox(35,25){\tt 99}} \put(105,0){\makebox(35,25){\tt 100}} \end{picture} \end{center} Distributing the array {\tt CYCLIC(3)}: \CODE !HPF$ DISTRIBUTE CENTURY(CYCLIC(3)) ONTO SEDECIM \EDOC results in this mapping of array elements onto abstract processors: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(560,250)(0,0) \put(0,225){\makebox(35,25){\small\rm 1}} \put(35,225){\makebox(35,25){\small\rm 2}} \put(70,225){\makebox(35,25){\small\rm 3}} \put(105,225){\makebox(35,25){\small\rm 4}} \put(140,225){\makebox(35,25){\small\rm 5}} \put(175,225){\makebox(35,25){\small\rm 6}} \put(210,225){\makebox(35,25){\small\rm 7}} \put(245,225){\makebox(35,25){\small\rm 8}} \put(280,225){\makebox(35,25){\small\rm 9}} \put(315,225){\makebox(35,25){\small\rm 10}} \put(350,225){\makebox(35,25){\small\rm 11}} \put(385,225){\makebox(35,25){\small\rm 12}} \put(420,225){\makebox(35,25){\small\rm 13}} \put(455,225){\makebox(35,25){\small\rm 14}} \put(490,225){\makebox(35,25){\small\rm 15}} \put(525,225){\makebox(35,25){\small\rm 16}} \thinlines \multiput(0,25)(0,25){8}{\line(1,0){560}} \thicklines \multiput(0,0)(0,225){2}{\line(1,0){560}} \multiput(0,0)(35,0){17}{\line(0,1){225}} \put(0,200){\makebox(35,25){\tt 1}} \put(0,175){\makebox(35,25){\tt 2}} \put(0,150){\makebox(35,25){\tt 3}} \put(35,200){\makebox(35,25){\tt 4}} \put(35,175){\makebox(35,25){\tt 5}} \put(35,150){\makebox(35,25){\tt 6}} \put(70,200){\makebox(35,25){\tt 7}} \put(70,175){\makebox(35,25){\tt 8}} \put(70,150){\makebox(35,25){\tt 9}} \put(105,200){\makebox(35,25){\tt 10}} \put(105,175){\makebox(35,25){\tt 11}} \put(105,150){\makebox(35,25){\tt 12}} \put(140,200){\makebox(35,25){\tt 13}} \put(140,175){\makebox(35,25){\tt 14}} \put(140,150){\makebox(35,25){\tt 15}} \put(175,200){\makebox(35,25){\tt 16}} \put(175,175){\makebox(35,25){\tt 17}} \put(175,150){\makebox(35,25){\tt 18}} \put(210,200){\makebox(35,25){\tt 19}} \put(210,175){\makebox(35,25){\tt 20}} \put(210,150){\makebox(35,25){\tt 21}} \put(245,200){\makebox(35,25){\tt 22}} \put(245,175){\makebox(35,25){\tt 23}} \put(245,150){\makebox(35,25){\tt 24}} \put(280,200){\makebox(35,25){\tt 25}} \put(280,175){\makebox(35,25){\tt 26}} \put(280,150){\makebox(35,25){\tt 27}} \put(315,200){\makebox(35,25){\tt 28}} \put(315,175){\makebox(35,25){\tt 29}} \put(315,150){\makebox(35,25){\tt 30}} \put(350,200){\makebox(35,25){\tt 31}} \put(350,175){\makebox(35,25){\tt 32}} \put(350,150){\makebox(35,25){\tt 33}} \put(385,200){\makebox(35,25){\tt 34}} \put(385,175){\makebox(35,25){\tt 35}} \put(385,150){\makebox(35,25){\tt 36}} \put(420,200){\makebox(35,25){\tt 37}} \put(420,175){\makebox(35,25){\tt 38}} \put(420,150){\makebox(35,25){\tt 39}} \put(455,200){\makebox(35,25){\tt 40}} \put(455,175){\makebox(35,25){\tt 41}} \put(455,150){\makebox(35,25){\tt 42}} \put(490,200){\makebox(35,25){\tt 43}} \put(490,175){\makebox(35,25){\tt 44}} \put(490,150){\makebox(35,25){\tt 45}} \put(525,200){\makebox(35,25){\tt 46}} \put(525,175){\makebox(35,25){\tt 47}} \put(525,150){\makebox(35,25){\tt 48}} \put(0,125){\makebox(35,25){\tt 49}} \put(0,100){\makebox(35,25){\tt 50}} \put(0,75){\makebox(35,25){\tt 51}} \put(35,125){\makebox(35,25){\tt 52}} \put(35,100){\makebox(35,25){\tt 53}} \put(35,75){\makebox(35,25){\tt 54}} \put(70,125){\makebox(35,25){\tt 55}} \put(70,100){\makebox(35,25){\tt 56}} \put(70,75){\makebox(35,25){\tt 57}} \put(105,125){\makebox(35,25){\tt 58}} \put(105,100){\makebox(35,25){\tt 59}} \put(105,75){\makebox(35,25){\tt 60}} \put(140,125){\makebox(35,25){\tt 61}} \put(140,100){\makebox(35,25){\tt 62}} \put(140,75){\makebox(35,25){\tt 63}} \put(175,125){\makebox(35,25){\tt 64}} \put(175,100){\makebox(35,25){\tt 65}} \put(175,75){\makebox(35,25){\tt 66}} \put(210,125){\makebox(35,25){\tt 67}} \put(210,100){\makebox(35,25){\tt 68}} \put(210,75){\makebox(35,25){\tt 69}} \put(245,125){\makebox(35,25){\tt 70}} \put(245,100){\makebox(35,25){\tt 71}} \put(245,75){\makebox(35,25){\tt 72}} \put(280,125){\makebox(35,25){\tt 73}} \put(280,100){\makebox(35,25){\tt 74}} \put(280,75){\makebox(35,25){\tt 75}} \put(315,125){\makebox(35,25){\tt 76}} \put(315,100){\makebox(35,25){\tt 77}} \put(315,75){\makebox(35,25){\tt 78}} \put(350,125){\makebox(35,25){\tt 79}} \put(350,100){\makebox(35,25){\tt 80}} \put(350,75){\makebox(35,25){\tt 81}} \put(385,125){\makebox(35,25){\tt 82}} \put(385,100){\makebox(35,25){\tt 83}} \put(385,75){\makebox(35,25){\tt 84}} \put(420,125){\makebox(35,25){\tt 85}} \put(420,100){\makebox(35,25){\tt 86}} \put(420,75){\makebox(35,25){\tt 87}} \put(455,125){\makebox(35,25){\tt 88}} \put(455,100){\makebox(35,25){\tt 89}} \put(455,75){\makebox(35,25){\tt 90}} \put(490,125){\makebox(35,25){\tt 91}} \put(490,100){\makebox(35,25){\tt 92}} \put(490,75){\makebox(35,25){\tt 93}} \put(525,125){\makebox(35,25){\tt 94}} \put(525,100){\makebox(35,25){\tt 95}} \put(525,75){\makebox(35,25){\tt 96}} \put(0,50){\makebox(35,25){\tt 97}} \put(0,25){\makebox(35,25){\tt 98}} \put(0,0){\makebox(35,25){\tt 99}} \put(35,50){\makebox(35,25){\tt 100}} \end{picture} \end{center} Note that it is perfectly permissible for an array to be distributed so that some processors have no elements. Indeed, an array may be ``distributed'' so that all elements reside on one processor. For example, \CODE !HPF$ DISTRIBUTE CENTURY(BLOCK(256)) ONTO SEDECIM \EDOC results in having only one non-empty block---a partially-filled one at that, having only 100 elements---on processor 1, with processors 2 through 16 having no elements of the array. A {\tt DISTRIBUTE} directive must not cause any data object associated with the {\it distributee} via storage association ({\tt COMMON} or {\tt EQUIVALENCE}) to be mapped such that storage units of a scalar data object are split across more than one abstract processor. See Section \ref{sequence} for further discussion of storage association. The statement form of a {\tt DISTRIBUTE} directive may be considered an abbreviation for an attributed form that happens to mention only one {\it distributee}; for example, \ICODE !HPF$ DISTRIBUTE distributee ( dist-format-list ) ONTO dist-target \EDOC is equivalent to \ICODE !HPF$ DISTRIBUTE ( dist-format-list ) ONTO dist-target :: distributee \EDOC Note that, to prevent syntactic ambiguity, the {\it dist-format-clause} must be present in the statement form, so in general the statement form of the directive may not be used to specify the mapping of scalars. If the {\it dist-format-clause} is omitted from the attributed form, then the language processor may make an arbitrary choice of distribution formats for each template or array. So the directive \CODE !HPF$ DISTRIBUTE ONTO P :: D1,D2,D3 \EDOC means the same as \CODE !HPF$ DISTRIBUTE ONTO P :: D1 !HPF$ DISTRIBUTE ONTO P :: D2 !HPF$ DISTRIBUTE ONTO P :: D3 \EDOC to which a compiler, perhaps taking into account patterns of use of {\tt D1}, {\tt D2}, and {\tt D3} within the code, might choose to supply three distinct distributions such as, for example, \CODE !HPF$ DISTRIBUTE D1(BLOCK, BLOCK) ONTO P !HPF$ DISTRIBUTE D2(CYCLIC, BLOCK) ONTO P !HPF$ DISTRIBUTE D3(BLOCK(43),CYCLIC) ONTO P \EDOC Then again, the compiler might happen to choose the same distribution for all three arrays. In either the statement form or the attributed form, if the {\tt ONTO} clause is present, it specifies the processor arrangement that is the target of the distribution. If the {\tt ONTO} clause is omitted, then a implementation-dependent processor arrangement is chosen arbitrarily for each {\it distributee}. So, for example, \CODE REAL, DIMENSION(1000) :: ARTHUR, ARNOLD, LINUS, LUCY !HPF$ PROCESSORS EXCALIBUR(32) !HPF$ DISTRIBUTE (BLOCK) ONTO EXCALIBUR :: ARTHUR, ARNOLD !HPF$ DISTRIBUTE (BLOCK) :: LINUS, LUCY \EDOC causes the arrays {\tt ARTHUR} and {\tt ARNOLD} to have the same mapping, so that corresponding elements reside in the same abstract processor, because they are the same size and distributed in the same way ({\tt BLOCK}) onto the same processor arrangement ({\tt EXCALIBUR}). However, {\tt LUCY} and {\tt LINUS} do not necessarily have the same mapping because they might, depending on the implementation, be distributed onto differently chosen processor arrangements; so corresponding elements of {\tt LUCY} and {\tt LINUS} might not reside on the same abstract processor. (The {\tt ALIGN} directive provides a way to ensure that two arrays have the same mapping without having to specify an explicit processor arrangement.) In a given environment, for some distributions, there may be no appropriate processor arrangement. \section{ALIGN Directive} \label{ALIGN-SECTION} The {\tt ALIGN} directive is used to specify that certain data objects are to be mapped in the same way as certain other data objects. Operations between aligned data objects are likely to be more efficient than operations between data objects that are not known to be aligned (because two objects that are aligned are intended to be mapped to the same abstract processor). The {\tt ALIGN} directive is designed to make it particularly easy to specify explicit mappings for all the elements of an array at once. While objects can be aligned in some cases through careful use of matching {\tt DISTRIBUTE} directives, {\tt ALIGN} is more general and frequently more convenient. The {\tt ALIGN} directive may appear only in the {\it specification-part} of a scoping unit and can contain only a {\it specification-expr} as a {\it subscript} or in a {\it subscript-triplet}. Formally, the syntax of {\tt ALIGN} is as follows: \BNF align-directive \IS ALIGN alignee align-directive-stuff align-directive-stuff \IS ( align-source-list ) align-with-clause align-attribute-stuff \IS [ ( align-source-list ) ] align-with-clause alignee \IS object-name align-source \IS : \OR * \OR align-dummy align-dummy \IS scalar-int-variable \FNB \begin{constraints} \item An {\it object-name} mentioned as an {\it alignee} must be a simple name and not a subobject designator. \item An {\it object-name} mentioned as an {\it alignee} may not appear as a {\it distributee}. \item An {\it object-name} mentioned as an {\it alignee} may not have the {\tt POINTER} attribute. \item If the {\it alignee} is scalar, the {\it align-source-list} (and its surrounding parentheses) must not appear. In this case the statement form of the directive is not allowed. \item If the {\it align-source-list} is present, its length must equal the rank of the alignee. \item An {\it align-dummy} must be a named variable. \item An object may not have both the {\tt INHERIT} attribute and the {\tt ALIGN} attribute. \end{constraints} Note that the possibility of an {\tt ALIGN} directive of the form \ICODE !HPF$ ALIGN align-attribute-stuff :: alignee-list \EDOC is covered by syntax rule \ref{combined-directive-rule} for a {\it combined-directive}. The statement form of an {\tt ALIGN} directive may be considered an abbreviation of an attributed form that happens to mention only one {\it alignee}: \ICODE !HPF$ ALIGN alignee ( align-source-list ) WITH align-spec \EDOC is equivalent to \ICODE !HPF$ ALIGN ( align-source-list ) WITH align-spec :: alignee \EDOC If the {\it align-source-list} is omitted from the attributed form and the {\it alignee}s are not scalar, the {\it align-source-list} is assumed to consist of a parenthesized list of ``{\tt :}'' entries, equal in number to the rank of the {\it alignees}. Similarly, if the {\it align-subscript-list} is omitted from the {\it align-spec} in either form, it is assumed to consist of a parenthesized list of ``{\tt :}'' entries, equal in number to the rank of the {\it align-target}. So the directive \CODE !HPF$ ALIGN WITH B :: A1, A2, A3 \EDOC means \CODE !HPF$ ALIGN (:,:) WITH B(:,:) :: A1, A2, A3 \EDOC which in turn means the same as \CODE !HPF$ ALIGN A1(:,:) WITH B(:,:) !HPF$ ALIGN A2(:,:) WITH B(:,:) !HPF$ ALIGN A3(:,:) WITH B(:,:) \EDOC \noindent because an attributed-form directive that mentions more than one {\it alignee} is equivalent to a series of identical directives, one for each {\it alignee}; all {\it alignee}s must have the same rank. With this understanding, we will assume below, for the sake of simplifying the description, that an {\tt ALIGN} directive has a single {\it alignee}. Each {\it align-source} corresponds to one axis of the {\it alignee}, and is specified as either ``{\tt :}'' or ``{\tt *}'' or a dummy variable: \begin{itemize} \item If it is ``{\tt :}'', then positions along that axis will be spread out across the matching axis of the {\it align-spec} (see below). \item If it is ``{\tt *}'', then that axis is {\it collapsed}: positions along that axis make no difference in determining the corresponding position within the {\it align-target}. (Replacing the ``{\tt *}'' with a dummy variable name not used anywhere else in the directive would have the same effect; ``{\tt *}'' is merely a convenience that saves the trouble of inventing a variable name and makes it clear that no dependence on that dimension is intended.) \item A dummy variable is considered to range over all valid index values for that dimension of the {\it alignee}. \end{itemize} The {\tt WITH} clause of an {\tt ALIGN} has the following syntax: \BNF align-with-clause \IS WITH align-spec align-spec \IS align-target [ ( align-subscript-list ) ] \OR * align-target [ ( align-subscript-list ) ] align-target \IS object-name \OR template-name align-subscript \IS int-expr \OR align-subscript-use \OR subscript-triplet \OR * align-subscript-use \IS [ [ int-level-two-expr ] add-op ] align-add-operand \OR align-subscript-use add-op int-add-operand align-add-operand \IS [ int-add-operand * ] align-primary \OR align-add-operand * int-mult-operand align-primary \IS align-dummy \OR ( align-subscript-use ) int-add-operand \IS add-operand int-mult-operand \IS mult-operand int-level-two-expr \IS level-2-expr \FNB The full syntax is given here for completeness, however some of the forms will only be discussed in Chapter~\ref{ch-mapping-subr}. These ``interprocedural'' forms are: \begin{itemize} \item The second option of rule~\ref{align-spec-rule} (containing the {\tt *} form) \end{itemize} \begin{constraints} \item An {\it object-name} mentioned as an {\it align-target} must be a simple name and not a subobject designator. \item An {\it align-target} may not have the {\tt OPTIONAL} attribute. \item If the {\it align-spec} in an {\tt ALIGN} directive begins with ``{\tt *}'' then every {\it alignee} must be a dummy argument. \item Each {\it align-dummy} may appear at most once in an {\it align-subscript-list}. \item An {\it align-subscript-use} expression may contain at most one occurrence of an {\it align-dummy}. \item An {\it align-dummy} may not appear anywhere in the {\it align-spec} except where explicitly permitted to appear by virtue of the grammar shown above. Paraphrased, one may construct an {\it align-subscript-use} by starting with an {\it align-dummy} and then doing additive and multiplicative things to it with any integer expressions that contain no {\it align-dummy}. \item A {\it subscript} in an {\it align-subscript} may not contain occurrences of any {\it align-dummy}. \item An {\it int-add-operand}, {\it int-mult-operand}, or {\it int-level-two-expr} must be of type integer. \end{constraints} The syntax rules for an {\it align-subscript-use} take account of operator precedence issues, but the basic idea is simple: an {\it align-subscript-use} is intended to be a linear function of a single occurrence of an {\it align-dummy}. For example, the following {\it align-subscript-use} expressions are valid, assuming that {\tt J}, {\tt K}, and {\tt M} are {\it align-dummy}s and {\tt N} is not an {\it align-dummy}: \begin{flushleft}\tt \begin{tabular}{l@{\quad}l@{\quad}l@{\quad}l@{\quad}l@{\quad}l} J & J+1 & 3-K & 2*M & N*M & 100-3*M \\ -J & +J & -K+3 & M+2**3 & M+N & -(4*7+IOR(6,9))*K-(13-5/3) \\ M*2 & N*(M-N) & 2*(J+1) & 5-K+3 & 10000-M*3 & 2*(3*(K-1)+13)-100 \end{tabular} \end{flushleft} The following expressions are not valid {\it align-subscript-use} expressions: \begin{flushleft}\tt \begin{tabular}{l@{\quad}l@{\quad}l@{\quad}l@{\quad}l@{\quad}l} J+J & J-J & 3*K-2*K & M*(N-M) & 2*J-3*J+J & 2*(3*(K-1)+13)-K \\ J*J & J+K & 3/K & 2**M & M*K & K-3*M \\ K-J & IOR(J,1) & -K/3 & M*(2+M) & M*(M-N) & 2**(2*J-3*J+J) \end{tabular} \end{flushleft} The {\it align-spec} must contain exactly as many {\it subscript-triplets} as the number of colons (``{\tt :}'') appearing in the {\it align-source-list}. These are matched up in corresponding left-to-right order, ignoring, for this purpose, any {\it align-source} that is not a colon and any {\it align-subscript} that is not a {\it subscript-triplet}. Consider a dimension of the {\it alignee} for which a colon appears as an {\it align-source} and let the lower and upper bounds of that array be {\it LA} and {\it UA}. Let the corresponding subscript triplet be {\it LT\/}:{\it UT\/}:{\it ST} or its equivalent. Then the colon could be replaced by a new, as-yet-unused dummy variable, say {\tt J}, and the subscript triplet by the expression {\tt (J-{\it LA})*{\it ST}+{\it LT}} without affecting the meaning of the directive. Moreover, the axes must conform, which means that \[ \max(0,UA-LA+1) \ = \ \max(0,\lceil (UT-LT+1) / ST \rceil) \] must be true. (This is entirely analogous to the treatment of array assignment.) To simplify the remainder of the discussion, we assume that every colon in the {\it align-source-list} has been replaced by new dummy variables in exactly the fashion just described, and that every ``{\tt *}'' in the {\it align-source-list} has likewise been replaced by an otherwise unused dummy variable. For example, \CODE !HPF$ ALIGN A(:,*,K,:,:,*) WITH B(31:,:,K+3,20:100:3) \EDOC may be transformed into its equivalent \CODE !HPF$ ALIGN A(I,J,K,L,M,N) WITH B(I-LBOUND(A,1)+31, & !HPF$ L-LBOUND(A,4)+LBOUND(B,2),K+3,(M-LBOUND(A,5))*3+20) \EDOC with the attached requirements \begin{center} {\tt SIZE(A,1)~.EQ.~UBOUND(B,1)-30} \\ {\tt SIZE(A,4)~.EQ.~SIZE(B,2)} \\ {\tt SIZE(A,5)~.EQ.~(100-20+3)/3} \end{center} Thus we need consider further only the case where every {\it align-source} is a dummy variable and no {\it align-subscript} is a {\it subscript-triplet}. Each dummy variable is considered to range over all valid index values for the corresponding dimension of the {\it alignee}. Every combination of possible values for the index variables selects an element of the {\it alignee}. The {\it align-spec} indicates a corresponding element (or section) of the {\it align-target} with which that element of the {\it alignee} should be aligned; this indication may be a function of the index values, but the nature of this function is syntactically restricted (as discussed above) to linear functions in order to limit the complexity of the implementation. Each {\it align-dummy} variable may appear at most once in the {\it align-spec} and only in certain rigidly prescribed contexts. The result is that each {\it align-subscript} expression may contain at most one {\it align-dummy} variable and the expression is constrained to be a linear function of that variable. (Therefore skew alignments are not possible.) An asterisk ``{\tt *}'' as an {\it align-subscript} indicates a replicated representation. Each element of the {\it alignee} is aligned with every position along that axis of the {\it align-target}. \begin{rationale} It may seem strange to use ``{\tt *}'' to mean both collapsing and replication; the rationale is that ``{\tt *}'' always stands conceptually for a dummy variable that appears nowhere else in the statement and ranges over the set of indices for the indicated dimension. Thus, for example, \CODE !HPF$ ALIGN A(:) WITH D(:,*) \EDOC means that a copy of {\tt A} is aligned with every column of {\tt D}, because it is conceptually equivalent to \ICODE for every legitimate index j, align A(:) with D(:,j) \EDOC just as \CODE !HPF$ ALIGN A(:,*) WITH D(:) \EDOC is conceptually equivalent to \ICODE for every legitimate index j, align A(:,j) with D(:) \EDOC Note, however, that while HPF syntax allows \CODE !HPF$ ALIGN A(:,*) WITH D(:) \EDOC to be written in the alternate form \CODE !HPF$ ALIGN A(:,J) WITH D(:) \EDOC it does {\it not} allow \CODE !HPF$ ALIGN A(:) WITH D(:,*) \EDOC to be written in the alternate form \CODE !HPF$ ALIGN A(:) WITH D(:,J) \EDOC because that has another meaning (only a variable appearing in the {\it align-source-list} following the {\it alignee} is understood to be an {\it align-dummy}, so the current value of the variable {\tt J} is used, thus aligning {\tt A} with a single column of {\tt D}). Replication allows an optimizing compiler to arrange to read whichever copy is closest. (Of course, when a replicated data object is written, all copies must be updated, not just one copy. Replicated representations are very useful for use as small lookup tables, where it is much faster to have a copy in each physical processor but without giving it an extra dimension that is logically unnecessary to the algorithm.) \end{rationale} By applying the transformations given above, all cases of an {\it align-subscript} may be conceptually reduced to either an {\it int-expr} (not involving an {\it align-dummy}) or an {\it align-subscript-use} and the {\it align-source-list} may be reduced to a list of index variables with no ``{\tt *}'' or ``{\tt:}''. An {\it align-subscript-list} may then be evaluated for any specific combination of values for the {\it align-dummy} variables simply by evaluating each {\it align-subscript} as an expression. The resulting subscript values must be legitimate subscripts for the {\it align-target}. (This implies that the {\it alignee} is not allowed to ``wrap around'' or ``extend past the edges'' of an {\it align-target}.) The selected element of the {\it alignee} is then considered to be aligned with the indicated element of the {\it align-target}; more precisely, the selected element of the {\it alignee} is considered to be ultimately aligned with the same object with which the indicated element of the {\it align-target} is currently ultimately aligned (possibly itself). More examples of {\tt ALIGN} directives: \CODE INTEGER D1(N) LOGICAL D2(N,N) REAL, DIMENSION(N,N):: X,A,B,C,AR1,AR2A,P,Q,R,S !HPF$ ALIGN X(:,*) WITH D1(:) !HPF$ ALIGN (:,*) WITH D1:: A,B,C,AR1,AR2A !HPF$ ALIGN WITH D2, DYNAMIC:: P,Q,R,S \EDOC Note that, in a {\it alignee-list}, the alignees must all have the same rank but need not all have the same shape; the extents need match only for dimensions that correspond to colons in the {\it align-source-list}. This turns out to be an extremely important convenience; one of the most common cases in current practice is aligning arrays that match in distributed (``parallel'') dimensions but may differ in collapsed (``on-processor'') dimensions: \CODE REAL A(3,N), B(4,N), C(43,N), Q(N) !HPF$ DISTRIBUTE Q(BLOCK) !HPF$ ALIGN (*,:) WITH Q:: A,B,C \EDOC Here there are processors (perhaps {\tt N} of them) and arrays of different sizes (3, 4, 43) within each processor are required. As far as HPF is concerned, the numbers 3, 4, and 43 may be different, because those axes will be collapsed. Thus array elements with indices differing only along that axis will all be aligned with the same element of {\tt Q} (and thus be specified as residing in the same processor). In the following examples, each directive in the group means the same thing, assuming that corresponding axis upper and lower bounds match: \CODE !Second axis of X is collapsed !HPF$ ALIGN X(:,*) WITH D1(:) !HPF$ ALIGN X(J,*) WITH D1(J) !HPF$ ALIGN X(J,K) WITH D1(J) !Replicated representation along second axis of D3 !HPF$ ALIGN X(:,:) WITH D3(:,*,:) !HPF$ ALIGN X(J,K) WITH D3(J,*,K) !Transposing two axes !HPF$ ALIGN X(J,K) WITH D2(K,J) !HPF$ ALIGN X(J,:) WITH D2(:,J) !HPF$ ALIGN X(:,K) WITH D2(K,:) !But there isn't any way to get rid of *both* index variables; ! the subscript-triplet syntax alone cannot express transposition. !Reversing both axes !HPF$ ALIGN X(J,K) WITH D2(M-J+1,N-K+1) !HPF$ ALIGN X(:,:) WITH D2(M:1:-1,N:1:-1) !Simple case !HPF$ ALIGN X(J,K) WITH D2(J,K) !HPF$ ALIGN X(:,:) WITH D2(:,:) !HPF$ ALIGN (J,K) WITH D2(J,K):: X !HPF$ ALIGN (:,:) WITH D2(:,:):: X !HPF$ ALIGN WITH D2:: X \EDOC \section{Allocatable Arrays and Pointers} \label{ALLOCATABLE-SECTION} A variable with the {\tt ALLOCATABLE} attribute may appear as an {\it alignee} in an {\tt ALIGN} directive or as a {\it distributee} in a {\tt DISTRIBUTE} directive. Such directives do not take effect immediately, however; they take effect each time the array is allocated by an {\tt ALLOCATE} statement, rather than on entry to the scoping unit. The values of all specification expressions in such a directive are determined once on entry to the scoping unit and may be used multiple times (or not at all). For example: \CODE SUBROUTINE MILLARD_FILLMORE(N,M) REAL, ALLOCATABLE, DIMENSION(:) :: A, B !HPF$ ALIGN B(I) WITH A(I+N) !HPF$ DISTRIBUTE A(BLOCK(M*2)) N = 43 M = 91 ALLOCATE(A(27)) ALLOCATE(B(13)) ... \EDOC The values of the expressions {\tt N} and {\tt M*2} on entry to the subprogram are conceptually retained by the {\tt ALIGN} and {\tt DISTRIBUTE} directives for later use at allocation time. When the array {\tt A} is allocated, it is distributed with a block size equal to the retained value of {\tt M*2}, not the value 182. When the array {\tt B} is allocated, it is aligned relative to {\tt A} according to the retained value of {\tt N}, not its new value 43. Note that it would have been incorrect in the {\tt MILLARD_FILLMORE} example to perform the two {\tt ALLOCATE} statements in the opposite order. In general, when an object {\tt X} is created it may be aligned to another object {\tt Y} only if {\tt Y} has already been created or allocated. The following example illustrates several related cases. \CODE SUBROUTINE WARREN_HARDING(P,Q) REAL P(:) REAL Q(:) REAL R(SIZE(Q)) REAL, ALLOCATABLE :: S(:),T(:) !HPF$ ALIGN P(I) WITH T(I) !Nonconforming !HPF$ ALIGN Q(I) WITH *T(I) !Nonconforming !HPF$ ALIGN R(I) WITH T(I) !Nonconforming !HPF$ ALIGN S(I) WITH T(I) ALLOCATE(S(SIZE(Q))) !Nonconforming ALLOCATE(T(SIZE(Q))) \EDOC The {\tt ALIGN} directives are not HPF-conforming because the array {\tt T} has not yet been allocated at the time that the various alignments must take place. The four cases differ slightly in their details. The arrays {\tt P} and {\tt Q} already exist on entry to the subroutine, but because {\tt T} is not yet allocated, one cannot correctly prescribe the alignment of {\tt P} or describe the alignment of {\tt Q} relative to {\tt T}. (See Chapter~\ref{ch-mapping-subr} for a discussion of prescriptive and descriptive directives.) The array {\tt R} is created on subroutine entry and its size can correctly depend on the {\tt SIZE} of {\tt Q}, but the alignment of {\tt R} cannot be specified in terms of the alignment of {\tt T} any more than its size can be specified in terms of the size of {\tt T}. It {\it is} permitted to have an alignment directive for {\tt S} in terms of {\tt T}, because the alignment action does not take place until {\tt S} is allocated; however, the first {\tt ALLOCATE} statement is nonconforming because {\tt S} needs to be aligned but at that point in time {\tt T} is still unallocated. When an array is allocated, it will be aligned to an existing template if there is an explicit {\tt ALIGN} directive for the allocatable variable. If there is no explicit {\tt ALIGN} directive, then the array will be ultimately aligned with itself. It is forbidden for any other object to be ultimately aligned to an array at the time the array becomes undefined by reason of deallocation. All this applies regardless of whether the name originally used in the {\tt ALLOCATE} statement when the array was created had the {\tt ALLOCATABLE} attribute or the {\tt POINTER} attribute. Pointers cannot be explicitly mapped in HPF and thus can be associated with objects which are not explicilty mapped. When used for allocation, the compiler may choose any arbitrary mapping for data allocated through the pointer. Explicit mapping of pointers is allowed under the approved extensions - see Section~\ref{POINTERS-SECTION} for details. Also, the the relationship of pointers and sequence attributes is described in Section~\ref{sequence}. \section{PROCESSORS Directive} \label{PROCESSORS-SECTION} The {\tt PROCESSORS} directive declares one or more rectilinear processor arrangements, specifying for each one its name, its rank (number of dimensions), and the extent in each dimension. It may appear only in the {\it specification-part} of a scoping unit. Every dimension of a processor arrangement must have nonzero extent; therefore a processor arrangement cannot be empty. In the language of section 14.1.2 of the Fortran standard, processor arrangements are local entities of class (1); therefore a processor arrangement may not have the same name as a variable, named constant, internal procedure, etc., in the same scoping unit. Names of processor arrangements obey the same rules for host and use association as other names in the long list in section 12.1.2.2.1 of the Fortran standard. A processor arrangement declared in a module has the default accessibility of the module. \begin{rationale} Because the name of a processor arrangement is not a first-class entity in HPF, but must appear only in directives, it cannot appear in an {\it access-stmt} ({\tt PRIVATE} or {\tt PUBLIC}). If directives ever become full-fledged Fortran statements rather than structured comments, then it would be appropriate to allow the accessibility of a processor arrangement to be controlled by listing its name in an {\it access-stmt}. \end{rationale} If two processor arrangements have the same shape, then corresponding elements of the two arrangements are understood to refer to the same abstract processor. (It is anticipated that implementation-dependent directives provided by some HPF implementations could overrule the default correspondence of processor arrangements that have the same shape.) If directives collectively specify that two objects be mapped to the same abstract processor at a given instant during the program execution, the intent is that the two objects be mapped to the same physical processor at that instant. The intrinsic functions {\tt NUMBER_OF_PROCESSORS} and {\tt PROCESSORS_SHAPE} may be used to inquire about the total number of actual physical processors used to execute the program. This information may then be used to calculate appropriate sizes for the declared abstract processor arrangements. \BNF processors-directive \IS PROCESSORS processors-decl-list processors-decl \IS processors-name [ ( explicit-shape-spec-list ) ] \FNB Examples: \CODE !HPF$ PROCESSORS P(N) !HPF$ PROCESSORS Q(NUMBER_OF_PROCESSORS()), & !HPF$ R(8,NUMBER_OF_PROCESSORS()/8) !HPF$ PROCESSORS BIZARRO(1972:1997,-20:17) !HPF$ PROCESSORS SCALARPROC \EDOC \noindent If no shape is specified, then the declared processor arrangement is conceptually scalar. \begin{rationale} A scalar processor arrangement may be useful as a way of indicating that certain scalar data should be kept together but need not interact strongly with distributed data. Depending on the implementation architecture, data distributed onto such a processor arrangement may reside in a single ``control'' or ``host'' processor (if the machine has one), or may reside in an arbitrarily chosen processor, or may be replicated over all processors. For target architectures that have a set of computational processors and a separate scalar host computer, a natural implementation is to map every scalar processor arrangement onto the host processor. For target architectures that have a set of computational processors but no separate scalar ``host'' computer, data mapped to a scalar processor arrangement might be mapped to some arbitrarily chosen computational processor or replicated onto all computational processors. \end{rationale} An HPF compiler is required to accept any {\tt PROCESSORS} declaration in which the product of the extents of each declared processor arrangement is equal to the number of physical processors that would be returned by the call {\tt NUMBER_OF_PROCESSORS()}. It must also accept all declarations of scalar {\tt PROCESSOR} arrangements. Other cases may be handled as well, depending on the implementation. For compatibility with the Fortran attribute syntax, an optional ``{\tt ::}'' may be inserted. The shape may also be specified with the {\tt DIMENSION} attribute: \CODE !HPF$ PROCESSORS :: RUBIK(3,3,3) !HPF$ PROCESSORS, DIMENSION(3,3,3) :: RUBIK \EDOC As in Fortran, an {\it explicit-shape-spec-list} in a {\it processors-decl} will override an explicit {\tt DIMENSION} attribute: \CODE !HPF$ PROCESSORS, DIMENSION(3,3,3) :: & !HPF$ RUBIK, RUBIKS_REVENGE(4,4,4), SOMA \EDOC Here {\tt RUBIKS_REVENGE} is \( 4 \times 4 \times 4 \) while {\tt RUBIK} and {\tt SOMA} are each \( 3 \times 3 \times 3 \). (By the rules enunciated above, however, such a statement may not be completely portable because no HPF language processor is required to handle shapes of total sizes 27 and 64 simultaneously.) Returning from a subprogram causes all processor arrangements declared local to that subprogram to become undefined. It is not HPF-conforming for any array or template to be distributed onto a processor arrangement at the time the processor arrangement becomes undefined unless at least one of two conditions holds: \begin{itemize} \item The array or template itself becomes undefined at the same time by virtue of returning from the subprogram. \item Whenever the subprogram is called, the processor arrangement is always locally defined in the same way, with identical lower bounds, and identical upper bounds. \begin{rationale} Note that the second condition is slightly less stringent than requiring all expressions to be constant. This allows calls to {\tt NUMBER_OF_PROCESSORS} or {\tt PROCESSORS_SHAPE} to appear without violating the condition. \end{rationale} \end{itemize} Variables in {\tt COMMON} or having the {\tt SAVE} attribute may be mapped to a locally declared processor arrangement, but because the first condition cannot hold for such variables (they don't become undefined), the second condition must be observed. This allows {\tt COMMON} variables to work properly through the customary strategy of putting identical declarations in each scoping unit that needs to use them, while allowing the processor arrangements to which they may be mapped to depend on the value returned by {\tt NUMBER_OF_PROCESSORS}. (See Section~\ref{sequence} for further information on mapping common variables.) \begin{implementors} It may be desirable to have a way for the user to specify at compile time the number of physical processors on which the program is to be executed. This might be specified either by a implementation-dependent directive, for example, or through the programming environment (for example, as a UNIX command-line argument). Such facilities are beyond the scope of the HPF specification, but as food for thought we offer the following illustrative hypothetical examples: \CODE !Declaration for multiprocessor by ABC Corporation !ABC$ PHYSICAL PROCESSORS(8) !Declaration for mpp by XYZ Incorporated !XYZ$ PHYSICAL PROCESSORS(65536) !Declaration for hypercube machine by PDQ Limited !PDQ$ PHYSICAL PROCESSORS(2,2,2,2,2,2,2,2,2,2) !Declaration for two-dimensional grid machine by TLA GmbH !TLA$ PHYSICAL PROCESSORS(128,64) !One of the preceding might affect the following !HPF$ PROCESSORS P(NUMBER_OF_PROCESSORS()) \EDOC It may furthermore be desirable to have a way for the user to specify the precise mapping of the processor arrangement declared in a {\tt PROCESSORS} statement to the physical processors of the executing hardware. Again, this might be specified either by a implementation-dependent directive or through the programming environment (for example, as a UNIX command-line argument); such facilities are beyond the scope of the HPF specification, but as food for thought we offer the following illustrative hypothetical example: \CODE !PDQ$ PHYSICAL PROCESSORS(2,2,2,2,2,2,2,2,2,2,2,2,2) !HPF$ PROCESSORS G(8,64,16) !PDQ$ MACHINE LAYOUT G(:GRAY(0:2),:GRAY(6:11),:BINARY(3:5,12)) \EDOC This might specify that the first dimension of {\tt G} should use hypercube axes 0, 1, 2 with a Gray-code ordering; the second dimension should use hypercube axes 6 through 11 with a Gray-code ordering; and the third dimension should use hypercube axes 3, 4, 5, and 12 with a binary ordering. \end{implementors} \section{TEMPLATE Directive} \label{TEMPLATE-SECTION} The {\tt TEMPLATE} directive declares one or more templates, specifying for each the name, the rank (number of dimensions), and the extent in each dimension. It must appear in the {\it specification-part} of a scoping unit. In the language of section 14.1.2 of the Fortran standard, templates are local entities of class (1); therefore a template may not have the same name as a variable, named constant, internal procedure, etc., in the same scoping unit. Template names obey the rules for host and use association as other names in the list in section 12.1.2.2.1 of the Fortran standard. A template declared in a module has the default accessibility of the module. \begin{rationale} Because the name of a template is not a first-class entity in HPF, but must appear only in directives, it cannot appear in an {\it access-stmt} ({\tt PRIVATE} or {\tt PUBLIC}). If directives ever become full-fledged Fortran statements rather than structured comments, then it would be appropriate to allow the accessibility of a template to be controlled by listing its name in an {\it access-stmt}. \end{rationale} A template is simply an abstract space of indexed positions; it can be considered as an ``array of nothings'' (as compared to an ``array of integers,'' say). %%%If an array is a cat, then a template is a Cheshire %%%cat, and the index space is the grin. A template may be used as an abstract {\it align-target} that may then be distributed. \BNF template-directive \IS TEMPLATE template-decl-list template-decl \IS template-name [ ( explicit-shape-spec-list ) ] \FNB Examples: \CODE !HPF$ TEMPLATE A(N) !HPF$ TEMPLATE B(N,N), C(N,2*N) !HPF$ TEMPLATE DOPEY(100,100),SNEEZY(24),GRUMPY(17,3,5) \EDOC If the ``{\tt ::}'' syntax is used, then the declared templates may optionally be distributed in the same {\it combined-directive}. In this case all templates declared by the directive must have the same rank so that the {\tt DISTRIBUTE} attribute will be meaningful. The {\tt DIMENSION} attribute may also be used. \CODE !HPF$ TEMPLATE, DISTRIBUTE(BLOCK,*) :: & !HPF$ WHINEY(64,64),MOPEY(128,128) !HPF$ TEMPLATE, DIMENSION(91,91) :: BORED,WHEEZY,PERKY \EDOC Templates are useful in the particular situation where one must align several arrays relative to one another but there is no need to declare a single array that spans the entire index space of interest. For example, one might want four \( N \times N \) arrays aligned to the four corners of a template of size \( (N+1) \times (N+1) \): \CODE !HPF$ TEMPLATE, DISTRIBUTE(BLOCK, BLOCK) :: EARTH(N+1,N+1) REAL, DIMENSION(N,N) :: NW, NE, SW, SE !HPF$ ALIGN NW(I,J) WITH EARTH( I , J ) !HPF$ ALIGN NE(I,J) WITH EARTH( I ,J+1) !HPF$ ALIGN SW(I,J) WITH EARTH(I+1, J ) !HPF$ ALIGN SE(I,J) WITH EARTH(I+1,J+1) \EDOC Templates may also be useful in making assertions about the mapping of dummy arguments (see Chapter~\ref{ch-mapping-subr}). Unlike arrays, templates cannot be in {\tt COMMON}. So two templates declared in different scoping units will always be distinct, even if they are given the same name. The only way for two program units to refer to the same template is to declare the template in a module that is then used by the two program units. Templates are not passed through the subprogram argument interface. The template to which a dummy argument is aligned is always distinct from the template to which the actual argument is aligned, though it may be a copy (see Section \ref{INHERIT-SECTION}). On exit from a subprogram, an HPF implementation arranges that the actual argument is aligned with the same template with which it was aligned before the call. Returning from a subprogram causes all templates declared local to that subprogram to become undefined. It is not HPF-conforming for any variable to be aligned to a template at the time the template becomes undefined unless at least one of two conditions holds: \begin{itemize} \item The variable itself becomes undefined at the same time by virtue of returning from the subprogram. \item Whenever the subprogram is called, the template is always locally defined in the same way, with identical lower bounds, identical upper bounds, and identical distribution information (if any) onto identically defined processor arrangements (see Section \ref{PROCESSORS-SECTION}). \begin{rationale} (Note that this second condition is slightly less stringent than requiring all expressions to be constant. This allows calls to {\tt NUMBER_OF_PROCESSORS} or {\tt PROCESSORS_SHAPE} to appear without violating the condition.) \end{rationale} \end{itemize} \noindent Variables in {\tt COMMON} or having the {\tt SAVE} attribute may be mapped to a locally declared template, but because the first condition cannot hold for such variable (they don't become undefined), the second condition must be observed. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Material from the old chapter 7 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Storage and Sequence Association} \label{sequence} HPF allows the mapping of data objects across multiple processors in order to improve parallel performance. Fortran specifies relationships between the storage for data objects associated through {\tt COMMON} and {\tt EQUIVALENCE} statements, and the order of array elements during association at procedure boundaries between actual arguments and dummy arguments. Otherwise, the location of data is not constrained by the language. {\tt COMMON} and {\tt EQUIVALENCE} statements constrain the alignment of different data items based on the underlying model of storage units and storage sequences: \begin{quotation} {\em Storage association is the association of two or more data objects that occurs when two or more storage sequences share or are aligned with one or more storage units.} --- Fortran Standard (14.6.3.1) \end{quotation} The model of storage association is a single linearly addressed memory, based on the traditional single address space, single memory unit architecture. This model can cause severe inefficiencies on architectures where storage for data objects is mapped. Sequence association refers to the order of array elements that Fortran requires when an array expression or array element is associated with a dummy array argument: \begin{quotation} {\em The rank and shape of the actual argument need not agree with the rank and shape of the dummy argument, \ldots} --- Fortran Standard (12.4.1.4) \end{quotation} As with storage association, sequence association is a natural concept only in systems with a linearly addressed memory. As an aid to porting FORTRAN 77 codes, HPF allows codes that rely on sequence and storage association to be valid in HPF. Some modification to existing FORTRAN 77 codes may nevertheless be necessary. This section explains the relationship between HPF data mapping and sequence and storage association. \subsection{Storage Association} \subsubsection{Definitions} \label{sequence-defs} \begin{enumerate} \item {\tt COMMON} blocks are either {\it sequential} or {\it nonsequential}, as determined by either explicit directive or compiler default. A sequential {\tt COMMON} block has a single common block storage sequence (Fortran Standard 5.5.2.1). \item An {\it aggregate variable group} is a collection of variables whose individual storage sequences are parts of a single storage sequence. Variables associated by {\tt EQUIVALENCE} statements or by a combination of {\tt EQUIVALENCE} and {\tt COMMON} statements form an aggregate variable group. The variables of a sequential {\tt COMMON} block form a single aggregate variable group. \item The {\it size} of an aggregate variable group is the number of storage units in the group's storage sequence (Fortran Standard 14.6.3.1). \item \label{seq-var} Data objects are either {\it sequential} or {\it nonsequential}. A data object is {\it sequential} if and only if any of the following holds: \begin{enumerate} \item it appears in a sequential {\tt COMMON} block; \item it is a member of an aggregate variable group; \item it is an assumed-size array; \item it is a component of a derived type with the Fortran {\tt SEQUENCE} attribute; or \item it is declared to be sequential in an HPF {\tt SEQUENCE} directive. \end{enumerate} A sequential object can be storage associated or sequence associated; nonsequential objects cannot. \item A {\tt COMMON} block contains a sequence of {\it components}. Each component is either an aggregate variable group, or a variable that is not a member of any aggregate variable group. Sequential {\tt COMMON} blocks contain a single component. Nonsequential {\tt COMMON} blocks may contain several components that may be sequential variables or aggregate variable groups or may be nonsequential. \end{enumerate} \subsubsection{Examples of Definitions} \CODE IMPLICIT REAL (A-Z) COMMON /FOO/ A(100), B(100), C(100), D(100), E(100) DIMENSION X(100), Y(150), Z(200) !Example 1: EQUIVALENCE ( A(1), Z(1) ) !Four components: (A, B), C, D, E !Sizes are: 200, 100, 100, 100 !Example 2: EQUIVALENCE ( A(51), X(1) ) ( B(100), Y(1) ) !Two components (A, B, C, D), E !Sizes are: 400, 100 !Example 3: !HPF$ SEQUENCE /FOO/ !The COMMON has one component, (A, B, C, D, E) !Size is 500 \EDOC \noindent The {\tt COMMON} block {\tt /FOO/} is nonsequential in Examples 1 and 2. Aggregate variable groups are shown as components in parentheses. \subsection {Sequence Directive} A {\tt SEQUENCE} directive is defined to allow a user to declare explicitly that data objects or {\tt COMMON} blocks are to be treated by the compiler as sequential. ({\tt COMMON} blocks are by default nonsequential. Data objects are nonsequential unless Definition~\ref{seq-var} applies.) Some implementations may supply an optional compilation environment where the {\tt SEQUENCE} directive is applied by default. For completeness in such an environment, HPF defines a {\tt NO SEQUENCE} directive to allow a user to establish that the usual nonsequential default should apply to a scoping unit, or selected data objects and {\tt COMMON} blocks within the scoping unit. \BNF sequence-directive \IS SEQUENCE [ [ :: ] association-name-list ] \OR NO SEQUENCE [ [ :: ] association-name-list ] association-name \IS object-name \OR / [ common-block-name ] / \FNB \begin{constraints} \item An object name or {\tt COMMON} block name may appear at most once in a {\it sequence-directive} within any scoping unit. \item Only one sequence directive with no {\it association-name-list} is permitted in the same scoping unit. \end{constraints} A pointer declared with the {\tt SEQUENCE} attribute can be only be associated with sequential objects. \subsubsection {Storage Association Rules} \begin{enumerate} \item A {\it sequence-directive} with an empty {\it association-name-list} is treated as if it contained the name of all implicitly mapped objects and {\tt COMMON} blocks in the scoping unit which cannot otherwise be determined to be sequential or nonsequential by their language context. \item A sequential object may not be explicitly mapped. \item No explicit mapping may be given for a component of a derived type having the Fortran {\tt SEQUENCE} attribute. Note that this rule is applicable only under the approved extensions since components of derived types cannot be explicitly mapped in HPF. \item No explicit mapping may be given for a dummy argument that is an assumed size array. \item If a {\tt COMMON} block is nonsequential, then all of the following must hold: \begin{enumerate} \item Every occurrence of the {\tt COMMON} block has exactly the same number of components with each corresponding component having a storage sequence of exactly the same size; \item If a component is a nonsequential variable in {\it any} occurrence of the {\tt COMMON} block, then it must be nonsequential with identical type, shape, and mapping attributes in {\it every} occurrence of the {\tt COMMON} block; and \item Every occurrence of the {\tt COMMON} block must be nonsequential. \end{enumerate} \end{enumerate} \subsubsection{Storage Association Discussion} \begin{users} Under these rules, variables in a {\tt COMMON} block can be mapped as long as the components of the {\tt COMMON} block are the same in every scoping unit that declares the {\tt COMMON} block. Correct Fortran programs will not necessarily be correct without modification in HPF. The use of {\tt EQUIVALENCE} with {\tt COMMON} blocks can impact the mappability of data objects in subtle ways. To allow maximum optimization for performance, the HPF default for data objects is to consider them mappable. In order to get correct separate compilation for subprograms that use {\tt COMMON} blocks with different aggregate variable groups in different scoping units, it will be necessary to insert the HPF {\tt SEQUENCE} directive. As a check-list for a user to determine the status of a data object or {\tt COMMON} block, the following questions can be applied, in order: \begin{itemize} \item Does the object appear in some explicit language context which dictates sequential (e.g. {\tt EQUIVALENCE}) or nonsequential (e.g. array-valued function result variable)? \item If not, does the object appear in an explicit mapping directive? \item If not, does the object or {\tt COMMON} block name appear in the list of names on a {\tt SEQUENCE} or {\tt NO SEQUENCE} directive? \item If not, does the scoping unit contain a nameless {\tt SEQUENCE} or {\tt NO SEQUENCE}? \item If not, is the compilation affected by some special implementation-dependent environment which dictates that names default to {\tt SEQUENCE}? \item If not, then the compiler will consider the object or {\tt COMMON} block name non-sequential and is free to apply data mapping optimizations disregarding Fortran sequence and storage association. \end{itemize} \end{users} \begin{implementors} In order to protect the user and to facilitate portability of older codes, two implementation options are strongly recommended. First, every implementation should supply some mechanism to verify that the type and shape of every mappable array and the sizes of aggregate variable groups in {\tt COMMON} blocks are the same in every scoping unit unless the {\tt COMMON} blocks are declared to be sequential. This same check should also verify that identical mappings have been selected for the variables in {\tt COMMON} blocks. Implementations without interprocedural information can use a link-time check. The second implementation option recommended is a mechanism to declare that data objects and {\tt COMMON} blocks for a given compilation should be considered sequential unless declared otherwise. The purpose of this feature is to permit compilation of large old libraries or subprograms where storage association is known to exist without requiring that the code be modified to apply the HPF {\tt SEQUENCE} directive to every {\tt COMMON} block. \end{implementors} --------------------------------------------------------------------------- To (un)subscribe to this list, send mail to hpff-doc-request@cs.rice.edu. Leave the subject line blank, and in the body put the line (un)subscribe --------------------------------------------------------------------------- From owner-hpff-doc Thu Jun 6 13:38:33 1996 Received: (from daemon@localhost) by cs.rice.edu (8.7.1/8.7.1) id NAA27553 for hpff-doc-out; Thu, 6 Jun 1996 13:38:33 -0500 (CDT) Date: Thu, 6 Jun 1996 13:38:33 -0500 (CDT) Message-Id: <199606061838.NAA27553@cs.rice.edu> From: Piyush Mehrotra Subject: hpff-doc: mapping-base.tex Phone: (804)864-2188 Fax: (804)864-6134 WWW: http://www.icase.edu/~pm Address: ICASE MS 132C NASA Langley Research Center Hampton VA 23681 Sender: owner-hpff-doc Precedence: bulk --------------------------------------------------------------------------- hpff-doc@cs.rice.edu is a mailing list for HPF 2.0 language specification authors and editors. Instructions for adding or deleting yourself from this list appear at the bottom of this message. --------------------------------------------------------------------------- % File: mapping-ext.tex % Contents: % Approved Extensions for data mapping for HPF 2.0 document, including % GenBlock % indirect % range % shadow % subsets % derived types % more on pointers. % Revision history: % May-10-96 Created by Charles Koelbel, Rice University % (From HPF 2.0 proposals) \chapter{Approved Extensions for Data Mapping} \label{ch-mapping-ext} {\em Comments on this chapter should be directed to Piyush Mehrotra ({\tt pm@icase.edu}), Carl Offner ({\tt offner@hpc.pko.dec.com}), and {\tt hpff-doc@cs.rice.edu}. Please use ``{\tt Comments on Extended Mapping}'' as the {\tt Subject:} line. \par } {\em This chapter contains two extended-dist-formats, one for the new distribution formats and one for the shadow width declarations. Something needs to be done. -- PM} This chapter describes a set of data mapping features which extend the capabilities provided by the base set as described in Chapter~\ref{ch-mapping-base}. These extensions can be divided into two categories. The first set of extensions provide the user a greater control over the mapping of the data. These include directives for dynamic remapping of data which allow the user to redistribute and realign at runtime data which has been declared {\tt DYNAMIC}. The {\tt ONTO} clause used in the {\tt DISTRIBUTE} directive is extended to allow direct distribution to subsets of processors. Explicit mapping of pointers and components of derived types are also introduced. Two new distributions are included, the {\tt GEN\_BLOCK} distribution which generalizes the block distribution and the {\tt INDIRECT} which allows the mapping of individual array elements to be specified through a mapping array. The programmer can use the second set of extensions to provide the compiler with information useful for generating efficient code. This category includes the range directives which allows the user to specify the range of distribution that a dynamically distributed array, a pointer or a formal argument may have. The shadow directive allows the user to specify the amount of additional space required on a processor to accommodate non-local elements in a nearest-neighbor computation. Since this chapter deals with extensions, we repeat some of the sections of Chapter~\ref{ch-mapping-base} and~\ref{ch-mapping-subr} providing new rules and extending old ones where necessary. %When extending a rule we use {\tt extended-xx} %as the new rule name with the understanding %that it will replace the old rule name, {\tt xx}, everywhere. \section{Extended Model} The fundamental model for allocation of data to abstract processors still remains a two-level mapping as described in Chapter~\ref{ch-mapping-base}. However, it is extended to allow the dynamic remapping of the data objects as illustrated by the following diagram: \begin{center} \setlength{\unitlength}{0.01in} \begin{picture}(600,240)(0,30) \thicklines \put(100,150){\circle{50}} \put(242,150){\circle{50}} \put(383,150){\circle{50}} \put(525,150){\circle{50}} \put(125,150){\vector(1,0){92}} \put(267,150){\vector(1,0){91}} \put(408,150){\vector(1,0){92}} \put(56,190){\shortstack{Arrays or\strut\\other objects\strut}} \put(192,190){\shortstack{Group of\strut\\aligned objects\strut}} \put(328,190){\shortstack{Abstract\strut\\processors as a\strut \\user-declared\strut\\Cartesian mesh\strut}} \put(494,190){\shortstack{Physical\strut\\processors\strut}} \put(135,50){\shortstack{{\tt ALIGN}\strut\\(static) or\strut \\{\tt REALIGN}\strut\\(dynamic)\strut}} \put(265,50){\shortstack{{\tt DISTRIBUTE}\strut\\(static) or\strut \\{\tt REDISTRIBUTE}\strut\\(dynamic)\strut}} \put(400,50){\shortstack{Optional\strut\\implementation-\strut \\dependent\strut\\directive\strut}} \end{picture} \end{center} Thus, objects can now be remapped at execution time using the executable directives {\tt REALIGN} and {\tt REDISTRIBUTE} Any object that is the root of an alignment tree, i.e., is not explicitly aligned to another object, can be explicitly redistributed. Redistributing such an object causes all objects ultimately aligned with it to be also redistributed so as to maintain the alignment relationships. Any object that is not a root of an alignment tree can be explicitly realigned but not explicitly redistributed. Such a realignment does not change the mapping of any other object. Note that such remapping of data may require communication among the processors. By analogy with the Fortran 90 {\tt ALLOCATABLE} attribute, HPF includes the attribute {\tt DYNAMIC}. It is not permitted to {\tt REALIGN} an array that has not been declared {\tt DYNAMIC}. Similarly, it is not permitted to {\tt REDISTRIBUTE} an array or template that has not been declared {\tt DYNAMIC}. Saved local variables, variables in common, and variables accessed by use association must not be implicitly remapped - e.g., by having variable distribution formats or being aligned with entities having variable distribution formats. Of these three categories of variables, only variables accessed by use association may have the {\tt DYNAMIC} attribute. \section{Syntax of Data Alignment and Distribution Directives} As in the case of other mapping directives, the executable directives {\tt REALIGN} and {\tt REDISTRIBUTE} also come in two forms (statement form and attribute form) but may not be combined with other attributes in a single directive. The {\tt RANGE} attribute may be combined with other attributes in a single directive. \BNF extend-combined-attribute \IS ALIGN align-attribute-stuff \OR DISTRIBUTE dist-attribute-stuff \OR DYNAMIC \OR INHERIT \OR TEMPLATE \OR PROCESSORS \OR DIMENSION ( explicit-shape-spec-list ) \OR RANGE range-attr-stuff-list \FNB \section{REDISTRIBUTE Directive} \label{REDISTRIBUTE-SECTION} The {\tt REDISTRIBUTE} directive is similar to the {\tt DISTRIBUTE} directive but is considered executable. An array (or template) may be redistributed at any time, provided it has been declared {\tt DYNAMIC} (see Section \ref{DYNAMIC-SECTION}). Any other arrays currently ultimately aligned with an array (or template) when it is redistributed are also remapped to reflect the new distribution, in such a way as to preserve alignment relationships (see Section \ref{ALIGN-SECTION}). (This can require a lot of computational and communication effort at run time; the programmer must take care when using this feature.) The {\tt DISTRIBUTE} directive may appear only in the {\it specification-part} of a scoping unit. The {\tt REDISTRIBUTE} directive may appear only in the {\it execution-part} of a scoping unit. The principal difference between {\tt DISTRIBUTE} and {\tt REDISTRIBUTE} is that {\tt DISTRIBUTE} must contain only a {\it specification-expr} as the argument to a distribution format, e.g., {\tt BLOCK} or {\tt CYCLIC}, whereas in {\tt REDISTRIBUTE} such an argument may be any integer expression. Another difference is that {\tt DISTRIBUTE} is an attribute, and so can be combined with other attributes as part of a {\it combined-directive}, whereas {\tt REDISTRIBUTE} is not an attribute (although a {\tt REDISTRIBUTE} statement may be written in the style of attributed syntax, using ``{\tt ::}'' punctuation). Formally, the syntax of the {\tt DISTRIBUTE} and {\tt REDISTRIBUTE} directives is: \BNF redistribute-directive \IS REDISTRIBUTE distributee dist-directive-stuff \OR REDISTRIBUTE dist-attribute-stuff :: distributee-list \FNB \begin{constraints} \item A {\it distributee} that appears in a {\tt REDISTRIBUTE} directive must have the {\tt DYNAMIC} attribute (see Section \ref{DYNAMIC-SECTION}). \item Neither the {\it dist-format-clause} nor the {\it dist-target} in a {\tt REDISTRIBUTE} may begin with ``{\tt *}''. \end{constraints} A {\tt REDISTRIBUTE} directive must not cause any data object associated with the {\it distributee} via storage association ({\tt COMMON} or {\tt EQUIVALENCE}) to be mapped such that storage units of a scalar data object are split across more than one abstract processor. See Section \ref{sequence} for further discussion of storage association. The statement form of a {\tt REDISTRIBUTE} directive may be considered an abbreviation for an attributed form that happens to mention only one {\it distributee}; for example, \CODE !HPF$ REDISTRIBUTE distributee ( dist-format-list ) ONTO dist-target \EDOC is equivalent to \CODE !HPF$ REDISTRIBUTE ( dist-format-list ) ONTO dist-target :: distributee \EDOC \section{REALIGN Directive} \label{REALIGN-SECTION} The {\tt REALIGN} directive is similar to the {\tt ALIGN} directive but is considered executable. An array (or template) may be realigned at any time, provided it has been declared {\tt DYNAMIC} (see Section \ref{DYNAMIC-SECTION}) Unlike redistribution (see Section \ref{DISTRIBUTE-SECTION}), realigning a data object does not cause any other object to be remapped. (However, realignment of even a single object, if it is large, could require a lot of computational and communication effort at run time; the programmer must take care when using this feature.) The {\tt ALIGN} directive may appear only in the {\it specification-part} of a scoping unit. The {\tt REALIGN} directive is similar but may appear only in the {\it execution-part} of a scoping unit. The principal difference between {\tt ALIGN} and {\tt REALIGN} is that {\tt ALIGN} must contain only a {\it specification-expr} as a {\it subscript} or in a {\it subscript-triplet}, whereas in {\tt REALIGN} such subscripts may be any integer expressions. Another difference is that {\tt ALIGN} is an attribute, and so can be combined with other attributes as part of a {\it combined-directive}, whereas {\tt REALIGN} is not an attribute (although a {\tt REALIGN} statement may be written in the style of attributed syntax, using ``{\tt ::}'' punctuation). Formally, the syntax of {\tt REALIGN} is as follows: \BNF realign-directive \IS REALIGN alignee align-directive-stuff \OR REALIGN align-attribute-stuff :: alignee-list \FNB \begin{constraints} \item Any {\it alignee} that appears in a {\tt REALIGN} directive must have the {\tt DYNAMIC} attribute (see Section \ref{DYNAMIC-SECTION}). \item If the {\tt align-target} specified in the {\tt align-with-clause} has the {\tt DYNAMIC} attribute, then each {\tt alignee} must also have the {\tt DYNAMIC} attribute. \item An object may not have both the {\tt INHERIT} attribute and the {\tt ALIGN} attribute. (However, an object with the {\tt INHERIT} attribute may appear as an {\it alignee} in a {\tt REALIGN} directive, provided that it does not appear as a {\it distributee} in a {\tt DISTRIBUTE} or {\tt REDISTRIBUTE} directive.) \end{constraints} \section{DYNAMIC Directive} \label{DYNAMIC-SECTION} The {\tt DYNAMIC} attribute specifies that an object may be dynamically realigned or redistributed. \BNF dynamic-directive \IS DYNAMIC alignee-or-distributee-list alignee-or-distributee \IS alignee \OR distributee \FNB \begin{constraints} \item An object in {\tt COMMON} may not be declared {\tt DYNAMIC} and may not be aligned to an object (or template) that is {\tt DYNAMIC}. (To get this kind of effect, Fortran 90 modules must be used instead of {\tt COMMON} blocks.) \item An object with the {\tt SAVE} attribute may not be declared {\tt DYNAMIC} and may not be aligned to an object (or template) that is {\tt DYNAMIC}. \end{constraints} A {\tt REALIGN} directive may not be applied to an {\it alignee} that does not have the {\tt DYNAMIC} attribute. A {\tt REDISTRIBUTE} directive may not be applied to a {\it distributee} that does not have the {\tt DYNAMIC} attribute. A {\tt DYNAMIC} directive may be combined with other directives, with the attributes stated in any order, consistent with the Fortran 90 attribute syntax. Examples: \CODE !HPF$ DYNAMIC A,B,C,D,E !HPF$ DYNAMIC:: A,B,C,D,E !HPF$ DYNAMIC, ALIGN WITH SNEEZY:: X,Y,Z !HPF$ ALIGN WITH SNEEZY, DYNAMIC:: X,Y,Z !HPF$ DYNAMIC, DISTRIBUTE(BLOCK, BLOCK) :: X,Y !HPF$ DISTRIBUTE(BLOCK, BLOCK), DYNAMIC :: X,Y \EDOC The first two examples mean exactly the same thing. The next two examples mean exactly the same second thing. The last two examples mean exactly the same third thing. The three directives \CODE !HPF$ TEMPLATE A(64,64),B(64,64),C(64,64),D(64,64) !HPF$ DISTRIBUTE(BLOCK, BLOCK) ONTO P:: A,B,C,D !HPF$ DYNAMIC A,B,C,D \EDOC may be combined into a single directive as follows: \CODE !HPF$ TEMPLATE, DISTRIBUTE(BLOCK, BLOCK) ONTO P, & !HPF$ DIMENSION(64,64),DYNAMIC :: A,B,C,D \EDOC An {\tt ALLOCATABLE} object may also be given the {\tt DYNAMIC} attribute. If an {\tt ALLOCATE} statement is immediately followed by {\tt REDISTRIBUTE} and/or {\tt REALIGN} directives, the meaning in principle is that the array is first created with the statically declared mapping, if any, then immediately remapped. In practice there is an obvious optimization: create the array in the processors to which it is about to be remapped, in a single step. HPF implementors are strongly encouraged to implement this optimization and HPF programmers are encouraged to rely upon it. Here is an example: \CODE REAL,ALLOCATABLE(:,:) :: TINKER, EVERS !HPF$ DYNAMIC :: TINKER, EVERS REAL, ALLOCATABLE :: CHANCE(:) !HPF$ DISTRIBUTE(BLOCK),DYNAMIC :: CHANCE ... READ 6,M,N ALLOCATE(TINKER(N*M,N*M)) !HPF$ REDISTRIBUTE TINKER(CYCLIC, BLOCK) ALLOCATE(EVERS(N,N)) !HPF$ REALIGN EVERS(:,:) WITH TINKER(M::M,1::M) ALLOCATE(CHANCE(10000)) !HPF$ REDISTRIBUTE CHANCE(CYCLIC) \EDOC While {\tt CHANCE} is by default always allocated with a {\tt BLOCK} distribution, it should be possible for a compiler to notice that it will immediately be remapped to a {\tt CYCLIC} distribution. Similar remarks apply to {\tt TINKER} and {\tt EVERS}. (Note that {\tt EVERS} is mapped in a thinly-spread-out manner onto {\tt TINKER}; adjacent elements of {\tt EVERS} are mapped to elements of {\tt TINKER} separated by a stride {\tt M}. This thinly-spread-out mapping is put in the lower left corner of {\tt TINKER}, because {\tt EVERS(1,1)} is mapped to {\tt TINKER(M,1)}.) \section{Remapping and Subprogram Interfaces} \label{DYNAMIC-DUMMY-SECTION} {\em Part of this section needs to go into the Chapter on Subprogram Interfaces since it deals with the issues of implicit remapping across procedure boundaries.} The rules on the interaction of the {\tt REALIGN} and {\tt REDISTRIBUTE} directives with a subprogram argument interface are: \begin{enumerate} \item A dummy argument may be declared {\tt DYNAMIC}. However, it is subject to the general restrictions concerning the use of the name of an array to stand for its associated template. \item If an array or any section thereof is accessible by two or more paths, it is not HPF-conforming to remap it through any of those paths. For example, if an array is passed as an actual argument, it is forbidden to realign that array, or to redistribute an array or template to which it was aligned at the time of the call, until the subprogram has returned from the call. This prevents nasty aliasing problems. An example: \CODE MODULE FOO REAL A(10,10) !HPF$ DYNAMIC :: A END PROGRAM MAIN USE FOO CALL SUB(A(1:5,3:9)) END SUBROUTINE SUB(B) USE FOO REAL B(:,:) ... !HPF$ REDISTRIBUTE A !Nonconforming ... END \EDOC Situations such as this are forbidden, for the same reasons that an assignment to {\tt A} at the statement marked ``Nonconforming'' would also be forbidden. In general, in {\it any} situation where assignment to a variable would be nonconforming by reason of aliasing, remapping of that variable by an explicit {\tt REALIGN} or {\tt REDISTRIBUTE} directive is also forbidden. \end{enumerate} An overriding principle is that any mapping or remapping of arguments is not visible to the caller. This is true whether such remapping is implicit (in order to conform to prescriptive directives, which may themselves be explicit or implicit) or explicit (specified by {\tt REALIGN} or {\tt REDISTRIBUTE} directives). When the subprogram returns and the caller resumes execution, all objects accessible to the caller after the call are mapped exactly as they were before the call. It is not possible for a subprogram to change the mapping of any object in a manner visible to its caller, not even by means of {\tt REALIGN} and {\tt REDISTRIBUTE}. \begin{implementors} There are several implementation strategies for achieving this behavior. For example, one may be able to use a copy-in/copy-out strategy for arguments that require remapping on subprogram entry. Alternatively, one may be able to remap the actual argument on entry and remap again on exit to restore the original mapping. \end{implementors} %---------------------------------------------------------------------------- % Mapping to subsets proposal %-------------------------------------------------------------------------- \section{Mapping to Processor Subsets} \label{mapping-proc-subset} This extension allows objects to be directly distributed to processor subsets by allowing a processor subset to be specified where a processor could be named, e.g., in a {\tt DISTRIBUTE} directive. The specified subset must be a proper subset of the named processor arrangement. Formally the syntax of the extended {\it dist-target} is as follows: \BNF extended-dist-target \IS processor-name [( section-subscript-list )] \OR * processor-name [( section-subscript-list )] \OR * \FNB \begin{constraints} \item The {\it section-subscript}s in the {\it section-subscript-list} may not be {\it vector-subscript}s and are restricted to either {\it subscript}s or {\it subscript-triplet}s. \item In the {\it section-subscript-list}, the number of {\it section-subscript}s must equal the rank of the {\it processor-name}. \end{constraints} \CODE !Example 1 !HPF$ PROCESSORS P(10) REAL A(100) !HPF$ DISTRIBUTE A(BLOCK) ONTO P(2:5) !Example 2 !HPF$ PROCESSORS Q(10,10) REAL A(100,100) !HPF$ DISTRIBUTE B(BLOCK,BLOCK) ONTO Q(5:10,5:10) \EDOC In Example 1, the array A is distributed by block across the processors P(2) to P(5) while in t eh second example, the array B is distributed across the lower right quadrant of the processor array Q. \begin{users} This extension is most useful in conjunction with the tasking construct, see Section~\ref{tasking}, which allows multiple independent phases of a computation to execute simultaneously on different subsets of processors. A similar situation arises when the code uses multiple data structures which can be computed in parallel where the computation on each individual object also exhibits parallelism, e.g., the multiple blocks in a multi-block grid used in some fluid dynamics calculation. Here, the individual blocks have to be distributed over subsets of processors to exploit both levels of parallelism. \end{users} \section{Pointers} \label{POINTERS-SECTION} Under the approved extensions, pointers can be explicitly mapped. Formally, this implies that the constraints that a {\it distributee} and an {\it alignee} may not have the {\tt POINTER} attribute as stated in Sections~\ref{DISTRIBUTE-SECTION} and~\ref{ALIGN-SECTION} respectively, have to be removed. As is the case of an allocatable object, the mapping specification for a pointer does not take effect immediately but plays a role when the pointer becomes pointer associated with a target either through allocation or through pointer assignment. When a pointer with an explicit mapping is used in an {\tt ALLOCATE} statement, the data is allocated with the specified mapping. For example: \CODE REAL, POINTER, DIMENSION(:) :: A, B !HPF$ ALIGN B WITH A !HPF$ DISTRIBUTE A(BLOCK) ONTO P ... ALLOCATE(A(100)) ALLOCATE(B(50)) ... ALLOCATE(B(200)) ! Nonconforming \EDOC Pointer {\it A} is declared to have a {\tt BLOCK} distribution while pointer {\tt B} is declared to be identically aligned with {\it A}. When {\it A} is allocated, it is created with a block distribution on the processor arrangement {\it P}. When {\it B} is allocated, it is aligned with the first 50 elements of {\it A}. Note, that the allocation statements may not occur in the opposite order, since an object may be aligned to another only if it has already been created or allocated. Also, the second allocation for {\it B} is nonconforming, since a larger object, {\it B} here, cannot be aligned with a smaller object, {\it A} in this case. A pointer with an explicit mapping can be pointer associated with a target through a pointer assignment statement if and only if the target has an explicit compatible mapping. Two objects are considered to have a compatible mapping if and only if the corresponding elements are mapped to the same abstract processor. Since a pointer does not have an inherent size only a rank, the compatibility of mappings may only be determinable when the assignment statement is executed. For example: \CODE !HPF$ PROCESSORS P(10) REAL, POINTER, DIMENSION(:) :: A REAL, TARGET:: B, C DIMENSION B(100), C(200) !HPF$ DISTRIBUTE A(BLOCK(10)) ONTO P !HPF$ DISTRIBUTE (BLOCK) ONTO P :: B, C ... A => B ! Conforming A => C ! Nonconforming ... \EDOC Here, {\it A} can be pointer associated with the array {\it B} since the latter has a size of 100 and thus when block distributed on the ten processor arrangement, its mapping is compatible with a {\tt BLOCK(10)} mapping onto the same processor arrangement. On the other hand, the size of {\it C} makes its mapping incompatible with that specified for {\it A}. Also, if a pointer has a partially specified (or underspecified) mapping, it can be pointer associated with a target as long as inheriting the rest of the mapping from the target provides a complete specification which is compatible with the mapping of the target. For example, the following pointer assignment is valid even though no processor arrangement is specified for the pointer: \CODE REAL, POINTER, DIMENSION(:) :: A REAL, TARGET, DIMENSION(100) :: B !HPF$ DISTRIBUTE A(BLOCK) !HPF$ DISTRIBUTE (BLOCK) ONTO P :: B ... A => B ! Conforming ... \EDOC If a dummy argument is a pointer with an explicit mapping, then the actual argument must be a pointer with an explicit compatible mapping as described above. In order to allow pointers to point to target with any explicit mappings we introduce the concept of pointers with transrcipitive mappings. For example: \CODE REAL, POINTER, DIMENSION(:) :: A !HPF$ DISTRIBUTE * :: A REAL, TARGET, DIMENSION(100) :: B, C !HPF$ DISTRIBUTE B(BLOCK), C(CYCLIC) ... A => B ! Conforming A => C ! Conforming ... \EDOC Here, the {\tt *} is used to indicate a transcriptive distribution for the pointer {\it A} and thus it can be pointer associated with both targets {\it B} and {\it C} distributed by {\tt BLCOK} and {\tt CYCLIC} respectively. To allow pointers to have a transcriptive distribution, we have to change the constraint for {\it dist-format-clause} as specified in Section~\ref{DISTRIBUTE-SECTION}, to read as follows: \begin{constraints} \item If either the {\it dist-format-clause} or the {\it dist-target} in a {\tt DISTRIBUTE} directive begins with ``{\tt *}'' then every {\it distributee} must be a dummy argument, {\em except if the {\em distributee} has the {\tt POINTER} attribute.} \end{constraints} Note, that pointers with the transcriptive distribution can be pointer associated only with targets which have explicit mappings. When such pointers are used in an {\tt ALLOCATE} statement, the compiler may choose any arbitrary mapping for the allocated data. A range declaration (see Section~\ref{range}) can be used to restrict the set of target distribution formats. A pointer with an explicit mapping, as described above, may also have the {\tt DYNAMIC} attribute. Such pointers can be pointer associated with targets if and only if they have the {\tt DYNAMIC} attribute. Thus, the target associated with the pointer may be remapped using a {\tt REDISTRIBUTE} or {\tt REALIGN} statement under the following restrictions. An pointer may be used in {\tt REALIGN} and {\tt REDISTRIBUTE} as an {\it alignee}, {\it align-target}, or {\it distributee} if and only if it is currently associated with a whole array, not an array section. One may remap an object by using a pointer as an {\it alignee} or {\it distributee} only if the object was created by {\tt ALLOCATE} but is not an {\tt ALLOCATABLE} array. Any directive that remaps an object constitutes an assertion on the part of the programmer that the remainder of program execution would be unaffected if all pointers associated with any portion of the object were instantly to acquire undefined pointer association status, except for the one pointer, if any, used to indicate the object in the remapping directive. \begin{implementors} If HPF directives were ever to be absorbed as actual Fortran statements, the previous paragraph could be written as ``Remapping an object causes all pointers associated with any portion of the object to have undefined pointer association status, except for the one pointer, if any, used to indicate the object in the remapping directive.'' The more complicated wording here is intended to avoid any implication that the remapping directives, in the form of structured comment annotations, have any effect on the execution semantics, as opposed to the execution speed, of the annotated program.) \end{implementors} {\em Question: What about the case when both the actual and formal are pointers and are declared dynamic - is remapping reflected when the rpocedure returns? An allocation of teh summy would certainly be carried back to the calling procedure. -- PM} \section{Mapping of Derived Type Components} %----------------------------------------------------------------------------- % Derive type mappings, apparent date Jan 18, 1996 %----------------------------------------------------------------------------- An {\tt ALIGN}, {\tt DISTRIBUTE}, or {\tt DYNAMIC} directive may appear within a {\it derived-type-def} wherever a {\it component-def-stmt} may appear. Every {\it alignee} or {\it distributee} within such a directive must be the name of a component defined within that {\it derived-type-def}. To allow mapping of the structure components, the rules have to be extended as follows: \BNF extended-distributee \IS object-name \OR template-name \OR structure-component \FNB A derived type is said to be an {\it explicitly mapped type} if any of its components is explicitly mapped or if any of its components is of an explicitly mapped type. \begin{constraints} \item A component of a derived type may be explicitly distributed only if the type of the component is not an explicitly mapped type. \item A variable or array of a derived type may be explicitly distributed only if the derived type is not an explicitly mapped type. \item A {\it distributee} in a {\tt DISTRIBUTE} directive may not be a {\it structure-component}, only a {\it component-name}. \item A {\it distributee} that is a {\it structure-component} may occur only in a {\tt REDISTRIBUTE} directive and every {\it part-ref} except the rightmost must be scalar (rank zero). The rightmost {\it part-name} in the {\it structure-component} must have the {\tt DYNAMIC} attribute. \end{constraints} \BNF extended-alignee \IS object-name \OR structure-component \FNB \begin{constraints} \item A component of a derived type may be explicitly aligned only if the type of the component is not an explicitly mapped type. \item A variable or array of a derived type may be explicitly aligned only if the derived type is not an explicitly mapped type. \item An {\it alignee} in an {\tt ALIGN} directive may not be a {\it structure-component}. \item An {\it alignee} that is a {\it structure-component} may occur only in a {\tt REALIGN} directive and every {\it part-ref} except the rightmost must be scalar (rank zero). The rightmost {\it part-name} in the {\it structure-component} must have the {\tt DYNAMIC} attribute. \end{constraints} The above constraints imply that components of structures can be mapped within the derived type definition itself such that any objects of that type are created with the specified mapping. Structure components which have been declared {\tt DYNAMIC} may be remapped using the {\tt REDISTRIBUTE} or {\tt REALIGN} directive. Example: \CODE TYPE SIMPLE REAL S(100) !HPF$ DISTRIBUTE S(BLOCK) END TYPE SIMPLE !HPF$ TEMPLATE, DISTRIBUTE(BLOCK, *) :: HAIRY_TEMPLATE(47,73) TYPE COMPLICATED INTEGER SIZE REAL RV(100,100), KV(100,100), QV(47,73) ! Arrays RV, KV, and QV may be mapped !HPF$ DISTRIBUTE (BLOCK, BLOCK):: RV, KV !HPF$ ALIGN WITH HAIRY_TEMPLATE :: QV TYPE(SIMPLE) SV(100) ! The following directive is not valid because SIMPLE ! is an explicitly mapped type. !HPF$ DISTRIBUTE SV(BLOCK) END TYPE COMPLICATED TYPE(COMPLICATED) LOTSOF(20) ! The following directive is not valid because COMPLICATED ! is an explicitly mapped type. !HPF$ DISTRIBUTE LOTSOF(BLOCK) \EDOC Here, a component of the derived type {\tt SIMPLE} has been mapped, thus objects of this type, e.g., {\tt SV} in type {\tt COMPLICATED}, cannot be distributed. The array {\tt LOTSOF} cannot be distributed for the same reason. Components of structures can be remapped using the {\tt REDISTRIBUTE} or {\tt REALIGN} directive if they have been declared {\tt DYNAMIC}. FOr example, the following code fragment can be used to allocate and map multiple blocks (called {\tt SUBGRID} here) of a multi-block grid: \CODE !HPF$ PROCESSORS P( number_of_processors() ) TYPE SUBGRID INTEGER SIZE INTEGER LO, HI ! target subset of processors REAL, POINTER BL(:) !HPF$ DYNAMIC BL END TYPE SUBGRID TYPE (SUBGRID), ALLOCATABLE :: GRID(:) READ (*,*) SUBGRID_COUNT ALLOCATE GRID(SUBGRID_COUNT) DO I = 1, SUBGRID_COUNT READ(*,*) GRID(I)%SIZE END DO ! Compute processor subsets for each subgrid, setting the LO and HI values CALL FIGURE_THE_PROCS ( GRID, number_of_processors()) ! Allocate each subgrid and distribute to the computed processors subset DO I = 1, SUBGRID_COUNT ALLOCATE( GRID(I)%BL( GRID(I)%SIZE ) ) !HPF$ REDISTRIBUTE GRID(I)%BL(BLOCK) ONTO P( GRID(I)%LO : GRID(I)%HI ) END DO \EDOC %----------------------------------------------------------------------------- % Indirect mapping proposal, apparent date Jan 18, 1996 %---------------------------------------------------------------------------- \section{New Distribution Formats} \label{dist-formats} This section describes two new distribution formats. Formally, we can extend the syntax as follows: \BNF extended-dist-format \IS BLOCK [ ( int-expr ) ] \OR CYCLIC [ ( int-expr ) ] \OR GEN\_BLOCK (int-array) \OR INDIRECT (int-array) \OR * \FNB \begin{constraints} \item An {\it int-array} appearing in a {\it extended-dist-format} of a {\tt DISTRIBUTE} or {\tt REDISTRIBUTE} directive must be an integer array of rank 1. \item An {\it int-array} appearing in a {\it extended-dist-format} of a {\tt DISTRIBUTE} directive must be a {\it restricted-expr}. \item The size of any {\it int-array} appearing with a {\tt GEN\_BLOCK} distribution must be equal to the extent of the corresponding dimension of the target processor arrangement. \item The size of any {\it int-array} appearing with an {\tt INDIRECT} distribution must be equal to the extent of the corresponding dimension of the {\it distributee} to which the distribution is to be applied. \end{constraints} The ``generalized'' block distribution, {\tt GEN\_BLOCK}, allows contiguous segments of an array, of possibly unequal sizes, to be mapped onto processors. The sizes of the segments are specified by values of a user-defined integer mapping array, one value per target processor of the mapping. That is, the {\it ith} element of the mapping array specifies the size of the block to be stored on the {\it ith} processor of the target processor arrangement. Thus, the values of the mapping arrays are restricted to be non-negative numbers and their sum must be greater than or equal to the extent of the corresponding dimension the array being distributed. The mapping array has to be a restricted expression when used in the {\tt DISTRIBUTE} directive, but can be an array variable in a {\tt REDISTRIBUTE} directive. In the latter case, changing the value of the map array after the directive has been executed will not change the mapping of the distributed array. Let {\it l} and {\it u} be the lower and upper bounds of the dimension of the {\it distributee}, {\it MAP} be the mapping array and let {\it BS(i):BE(i)} be the resultant elements mapped to the {\it ith} processor in the corresponding dimension of the target processor arrangements. Then, \begin{eqnarray*} BS (1) & = & l,\\ BE (i) & = & max( BS(i)+MAP(i)-1, u),\\ BS (i) & = & BE(i-1) + 1.\\ \end{eqnarray*} Example: \CODE PARAMETER (S = /2,25,20,0,8,65/) !HPF$ PROCESSORS P(6) REAL A(100), B(200), new(6) !HPF$ DISTRIBUTE A( GEN_BLOCK( S) ) ONTO P !HPF$ DYNAMIC B ... new = ... !HPF$ REDISTRIBUTE ( B( GEN_BLOCK(new) ) \EDOC Given the above specification, array elements A(1:2) are mapped on P(1), A(3:27) are mapped on P(2), A(28:37) are mapped on P(3), no elements are mapped on P(4), A(37:45) are mapped on P(5), and A(46:100) are mapped on P(6). The array {\it B} is distributed based on the array {\it new} whose values are computed at runtime. \begin{implementors} Accessing elements of an array distributed using the generalized block distribution may require accessing the values of the mapping array at runtime. However, since the size of such an array is equal to that of the processor arrangement, it can in most cases be replicated over all processors. For dynamic arrays, an independent copy of the mapping array will have to be maintained internally so that a change in the values of the mapping array does not affect the access of the distributed array. \end{implementors} There are many scientific applications in which the structure of the underlying domain is such that it does not map directly onto Fortran data structures. For example, in many CFD applications a unstructured mesh (consisting of triangles in 2D or tetrahedra in 3d) is used to represent the underlying domain. The nodes of such a mesh are generally represented by a one-dimensional array while another is used to represent their interconnections. Mapping such arrays using the structured distribution mechanisms, {\tt BLOCK} and {\tt CYCLIC}, results in mappings in which unrelated elements are mapped onto the same processor. This in turn leads to massive amounts of unnecessary communication. What is required is a mechanism to map a related set of arbitrary array elements onto the same processor. The {\tt INDIRECT} distribution provides such a mechanism. The {\tt INDIRECT} distribution allows a many-to-one mapping of elements of a dimension of a data array to a dimension of the target processor arrangement. An integer array is used to specify the target processor of each individual element of the array dimension being distributed. That is, the {\it ith} element of the mapping array provides the processor number onto which the {\it ith} array element is to be mapped. Since the mapping array maps array elements onto processor elements, the extent of the mapping array must match the extent of the dimension of the array it is distributing. Also, the values of the mapping array must lie between the lower and upper bound of the target dimension of the processor arrangement. The mapping array has to be a restricted expression when used in the {\tt DISTRIBUTE} directive, but can be an array variable in a {\tt REDISTRIBUTE} directive. In the latter case, changing the value of the mapping array after the directive has been executed will not change the mapping of the distributed array. Example: \CODE !HPF$ PROCESSORS P(4) REAL A(100), B(50) INTEGER map1(100), map2(50) PARAMETER (map1 = /1,3,4,3,3,2,1,4, ..../) !HPF$ DYNAMIC B !HPF$ DISTRIBUTE A( INDIRECT(map1) ) ONTO P !HPF$ DISTRIBUTE map2(BLOCK) ONTO P map2 = ... !HPF$ DISTRIBUTE B( INDIRECT(map2) ) ONTO P .... \EDOC Here, the array {\it A} is distributed statically using the constant array {\it map1}. Thus: \hspace{0.5in} \parbox{3in}{ A(1) is mapped onto P(1),\\ A(2) is mapped onto P(3),\\ A(3) is mapped onto P(4),\\ A(4) is mapped onto P(3),\\ A(5) is mapped onto P(3),\\ A(6) is mapped onto P(2),\\ A(7) is mapped onto P(1),\\ A(5) is mapped onto P(4), and so on.\\ } The array {\it B} is declared dynamic and is redistributed using the mapping array {\it map2}. \begin{implementors} In general, the {\tt INDIRECT} distribution is going to be used in the {\tt REDISTRIBUTE} directive with an array variable as the map array. Also, since the size of the mapping array must be the same as the array being distributed, it will itself be distributed most likely using the {\tt BLOCK} distribution. This raises several issues. To correctly implement this distribution, the runtime system should maintain a (distributed) copy of the mapping array so that if the program modifies the mapping array, the distribution does not change. Using an array variable as a mapping array implies that the location of each element of the array will not be known until runtime. Thus, a communication maybe required to figure out the location of a specific array element. \end{implementors} %----------------------------------------------------------------------------- % Shadow regions proposal, apparent date Feb 7, 1996 %----------------------------------------------------------------------------- \section{Range Directive} \label{range} The {\tt RANGE} attribute is used to restrict the possible distribution formats for an object or template that has the {\tt DYNAMIC} attricute or a transcriptive distribution format, or an object with the{\tt POINTER} attribute. \BNF range-directive \IS RANGE ranger range-attr-stuff-list ranger \IS object-name \OR template-name range-attr-stuff \IS ( range-attr-list ) range-attr \IS range-dist-format \OR ALL range-dist-format \IS BLOCK [ ()] \OR CYCLIC [ () ] \OR GEN\_BLOCK \OR INDIRECT \OR * \FNB \begin{constraints} \item The {\it ranger} must have the {\tt DYNAMIC} attribute, the {\tt POINTER} attribute, the {\tt INHERIT} attribute and no explicit {\tt DISTRIBUTE}, or it must have been specified with a {\it dist-format-clause} of {\tt *} in a {\tt DISTRIBUTE} or combined directive. \item The length of any {\it range-attr-list} must be equal to the rank of the {\it ranger}. \item The {\it ranger} must not appear as an alignee in an {\tt ALIGN} or {\tt REALIGN} directive. \end{constraints} If a {\it ranger} is given the {\tt RANGE} attribute, every {\it range-attr} in at least one of the {\it range-attr-stuff}s must be compatible with the distribution format of the corresponding dimension of the: \begin{enumerate} \item template of the {\it ranger}, if the {\it ranger} is transcriptevly distributed; \item target of the {\it ranger}, if the {\it ranger} has the {\tt POINTER} attribute, and is associated; or \item the {\it ranger} if the {\it ranger} has the {\tt DYNAMIC} attribute. \end{enumerate} This compatibility must be maintained by any {\tt DISTRIBUTE} or {\tt REDISTRIBUTE} directive in which the {\it ranger} appears as a distributee, or if the ranger has the {\tt POINTER} attribute, for any target with which the {\it ranger} becomes associated. A distribution format of \begin{enumerate} \item {\tt BLOCK} is compatible with a {\it range-dist-format} of {\tt BLOCK}, {\tt BLOCK()} or {\tt CYCLIC()}; \item {\tt BLOCK(n)} is compatible with a {\it range-dist-format} of {\tt BLOCK()} or {\tt CYCLIC()}; \item {\tt CYCLIC} is compatible with a {\it range-dist-format} of {\tt CYCLIC} or {\tt CYCLIC()}; \item {\tt CYCLIC(n)} is compatible with a {\it range-dist-format} of {\tt CYCLIC()}; \item {\tt GEN\_BLOCK} is compatible with a {\it range-dist-format} of {\tt GEN\_BLOCK}; \item {\tt INDIRECT} is compatible with a {\it range-dist-format} of {\tt INDIRECT}; \item {\tt *} is compatible with a {\it range-dist-format} of {\tt *}. \end{enumerate} All distribution formats are compatible with a {\it range-dist-format} of {\tt ALL}. \section{Shadow Width Declarations} In compiling nearest-neighbor code (as is used, for instance, in discretizing partial differential equations or implementing convolutions), a standard technique is to allocate storage on each processor for the local array section that includes additional space for the elements that have to be moved in from its neighboring processors. This additional storage is referred to as ``shadow edges.'' There are conceptually two shadow edges for each array dimension: one at the low end of the local array section, and the other at the high end. In a single routine, the compiler can tell which arrays require shadow edges, and allocate this additional space accordingly. However, since the width of the shadow area is dependent on the size of the computational stencil being used, an array may require different shadow widths in different routines. Thus, without interprocedural analysis, an array argument may need to be copied into a space with the appropriate shadow width on each procedure call. A similar data motion would be required to copy the data back to its original location on exit form the subroutine. This unnecessary data motion can be avoided by allowing the user to specify the required shadow width when the array is declared. Formally, the syntax for declaring shadow widths is as follows: \BNF extended-dist-format \IS BLOCK [ ( int-expr [, shadow-spec-or-int-expr] ) ] \OR BLOCK ( shadow-spec ) \OR CYCLIC [ ( int-expr [, shadow-spec-or-int-expr] ) ] \OR CYCLIC ( shadow-spec ) \OR GEN\_BLOCK (int-array [, shadow-spec-or-int-expr] ) \OR INDIRECT (int-array) \OR * shadow-spec-or-int-expr \IS shadow-spec \OR int-expr shadow-spec \IS SHADOW = int-expr \OR LOW\_SHADOW = int-expr [, HIGH\_SHADOW = int-expr] \OR HIGH\_SHADOW = int-expr [, LOW\_SHADOW = int-expr] \FNB \begin{constraints} \item Any {\it int-expr} appearing in a {\it shadow-spec} or in a {\it shadow-spec-or-int-expr} must be a constant {\it specification-expr} with value greater than equal to {\it 0}. \item The absence of a {\it shadow-spec} or a {\it shadow-spec-or-int-expr} is equivalent to {\tt SHADOW = 0}. \item The specification {\tt SHADOW = w} (where {\tt w} is an {\it int-expr}) is equivalent to {\tt LOW\_SHADOW = w, HIGH\_SHADOW = w} \item A {\it shadow-spec-or-int-expr} that is just an {\it int-expr} (say {\tt w}) is equivalent to {\tt SHADOW = w}. \item Specifying only {\tt LOW\_SHADOW} implies {\tt HIGH\_SHADOW = 0}. \item Specifying only {\tt HIGH\_SHADOW} implies {\tt LOW\_SHADOW = 0}. \end{constraints} Thus, the directive \CODE !HPF$ DISTRIBUTE(BLOCK(n, w)) :: A \EDOC \noindent specifies that the array {\it A} is distributed by {\tt BLOCK(n)} and is to have a shadow width of {\it w} on both sides. If {\it A} is a dummy argument, this gives the compiler enough information to inhibit unnecessary data motion at procedure calls. The shadow width parameter can also be referenced by using the keyword {\tt SHADOW}; this is really a preferred usage, and is necessary if the distribution is {\tt BLOCK}, rather than {\tt BLOCK(n)}. So for instance, the following are equivalent: \CODE !HPF$ DISTRIBUTE(BLOCK(n, w)) :: A !HPF$ DISTRIBUTE(BLOCK(n, SHADOW = w)) :: A \EDOC A {\tt BLOCK} distributed dimension with a shadow width would be indicated simply as \CODE !HPF$ DISTRIBUTE(BLOCK(SHADOW = w)) :: A \EDOC In a similar fashion, \CODE !HPF$ DISTRIBUTE(CYCLIC(n, w)) :: A !HPF$ DISTRIBUTE(CYCLIC(n, SHADOW = w)) :: A \EDOC \noindent are both equivalent and refer to a {\tt CYCLIC(n)} distribution with a shadow width of {\it w}. Using the keywords {\tt LOW\_SHADOW} and {\tt HIGH\_SHADOW}, different shadow widths can be specified for the low end and high end of a dimension. For example: \CODE REAL, DIMENSION (1000) :: A !HPF$ DISTRIBUTE(BLOCK(LOW_SHADOW = 1, HIGH_SHADOW = 2)) :: A .... FORALL (i = 2, 998) A(i) = 0.25* (A(i) + A(i-1) + A(i+1) + A(i+2)) \EDOC \noindent specifies that only one non-local element is needed at the lower end while two are needed at the high end. --------------------------------------------------------------------------- To (un)subscribe to this list, send mail to hpff-doc-request@cs.rice.edu. Leave the subject line blank, and in the body put the line (un)subscribe --------------------------------------------------------------------------- From owner-hpff-doc Thu Jun 27 22:23:46 1996 Received: (from daemon@localhost) by cs.rice.edu (8.7.1/8.7.1) id WAA22372 for hpff-doc-out; Thu, 27 Jun 1996 22:23:46 -0500 (CDT) Received: from coral.llnl.gov (coral.llnl.gov [134.9.1.2]) by cs.rice.edu (8.7.1/8.7.1) with ESMTP id WAA22363 for ; Thu, 27 Jun 1996 22:23:43 -0500 (CDT) Received: (from zosel@localhost) by coral.llnl.gov (8.7.5/8.7.3/LLNL-Jun96) id UAA12175 for hpff-doc@cs.rice.edu; Thu, 27 Jun 1996 20:23:41 -0700 (PDT) Date: Thu, 27 Jun 1996 20:23:41 -0700 (PDT) From: Mary E Zosel Message-Id: <199606280323.UAA12175@coral.llnl.gov> To: hpff-doc@cs.rice.edu Subject: hpff-doc: IMPORTANT: document review Mime-Version: 1.0 Content-Type: text/plain; charset=X-roman8 Content-Transfer-Encoding: 7bit Sender: owner-hpff-doc Precedence: bulk --------------------------------------------------------------------------- hpff-doc@cs.rice.edu is a mailing list for HPF 2.0 language specification authors and editors. Instructions for adding or deleting yourself from this list appear at the bottom of this message. --------------------------------------------------------------------------- To: HPF document readers and writers A draft of many of the chapters of the hpf document will be posted by Chuck soon at Rice. He will send a message saying when and where. Please note - what is there is incomplete for three reasons: - some chapters I simply couldn't find all the things that were messing up my latex ... I'm far from an expert ... - and for some other chapters - for who-know-what reason, the files I extracted from the original tar file sent in May were truncated. I discovered a couple of these in time to ask CHK for new copies ... the others I just put a sentence at the end saying that was all I had. - and some chapters just did not show up --- what is in the document (if there at all) is an old chapter. :-( The chapters that I could not latex were: library.tex library-ext.tex hpf-local-ext.tex fortran-local-ext.tex and c-interop-ext.tex The source for these chapters is in the tar file that will be posted. THE LINE IN HPF-REPORT.TEX THAT INCLUDES THESE FILES HAS BEEN COMMENTED OUT - PENDED RESOLUTION OF WHATEVER CAUSED MY PROBLEMS ... I was finding things like extra or missing "}" ... and finally got tired of hunting. This is a state of considerable disarray - and relative disaster for getting a document turnaround. PLEASE - PLEASE - PLEASE - PLEASE - if you are a chapter writer - look at what is and is not included. PLEASE try to get a clean run of your chapter - in the context of the hpf-report ... (not with your own macros, etc ... that was part of the problem). if you don't understand hpf-report.tex - ask me. - if you are a chapter reader see what comments you can send to the chapter writer ASAP. Maybe if you grok latex, you can get a printout of the chapters I failed to include. - if you are a writer - incorporate any corrections you get ASAP and either send and updated copy of your chapter to Theresa for duplication (by July 3) OR bring 20 copies to the meeting (or both). ================== HERE IS A REMINDER OF WHAT YOU COMMITTED TO DO: (note - some of the chapter numbers may be different because of the deleted chapters) DOCUMENT OUTLINE and NAMES Section I Introduction Chapter 0: Front, ack, etc writer: Mary reader: Chuck Chapter 1: Overview - terms and concepts - new document/language structure, Fortran language base, F95 features, HPF 1.1 deletions, etc. writer: David and Carl reader: Bob, Rob, Chuck, Jerry Section II HPF 2.0 Chapter 2: Mappings - distribute, align, sequence, some of pointer. Material from most of v1.1 chapter 3 and some of chapter 7. writer: Piyush, Carl, GLS reader: Saday, Guy R. Chapter 3: Mapping across Subprogram Interface. Material from v1.1 chapters 3 and 7. writer/reader - same as chapter 2. Chapter 4: Independent and Reduce writer: Chuck reader: Rob and Jay Chapter 5: HPF Library (plus sort-up and sort-down) writer: Rob reader: Carol Chapter 6: Extrinsics writer: David reader: Mary Chapter 7: Portability and Efficiency Issues (forall examples) writer: Andy, Chuck reader: Larry, Carl, Henry, Guy R. Section III Extended Features Chapter 8: More mappings (and related subprogram interface issues) {should this be two chapters as in Section II?} GenBlock, indirect, range, shadow, subsets, derived type, more on pointers. writer: Piyush, Carl reader: Saday, Guy R. Chapter 9: ON, TASK, RESIDENT writer: Chuck, Jaspal reader: Jay Chapter 10: New Library - generalized transpose, extended inquiries, new mappings. writer: Rob reader: Henry Chapter 11: Async I/O writer: Larry reader: Alok Chapter 12: HPF_LOCAL writer: David reader: Carl, Carol, Saday Chapter 13: HPF_SERIAL writer: David reader: Carl, Carol, Saday Chapter 14: PROVISIONAL TEXT - F77_LOCAL (don't really know where and if this goes yet). writer: Carol reader: ? Chapter 15: C Interoperability writer: Henry and Andy reader: Scott, Jerry Section IV Appendixes A - BNF (GLS) B - Extrinsic Extrinsics - (policy, mechanism, HPF_CRAFT) writer: Mary, Andy reader: Bob C- Subset writer: Mary reader: Carol Glossary and Bibliography later. PLEASE - Chapter writers - note those features that should be defined in the glossary and put the appropriate latex on the key places they are used. Sweep Team: Piyush, Carol, Henry, Rob, Mary --------------------------------------------------------------------------- To (un)subscribe to this list, send mail to hpff-doc-request@cs.rice.edu. Leave the subject line blank, and in the body put the line (un)subscribe --------------------------------------------------------------------------- From owner-hpff-doc Thu Jun 27 23:04:48 1996 Received: (from daemon@localhost) by cs.rice.edu (8.7.1/8.7.1) id XAA23053 for hpff-doc-out; Thu, 27 Jun 1996 23:04:48 -0500 (CDT) Received: from [128.42.5.213] (pasyn-85.rice.edu [128.42.5.213]) by cs.rice.edu (8.7.1/8.7.1) with SMTP id XAA23047; Thu, 27 Jun 1996 23:04:44 -0500 (CDT) X-Sender: chk@titan.cs.rice.edu Message-Id: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Date: Thu, 27 Jun 1996 23:06:54 -0600 To: Mary E Zosel From: chk@cs.rice.edu (Chuck Koelbel) Subject: hpff-doc: Re: IMPORTANT: document review Cc: hpff-doc@cs.rice.edu Sender: owner-hpff-doc Precedence: bulk --------------------------------------------------------------------------- hpff-doc@cs.rice.edu is a mailing list for HPF 2.0 language specification authors and editors. Instructions for adding or deleting yourself from this list appear at the bottom of this message. --------------------------------------------------------------------------- >To: HPF document readers and writers > >A draft of many of the chapters of the hpf document will be >posted by Chuck soon at Rice. He will send a message saying >when and where. > Getting the draft: Everything is available by anonymous FTP from titan.cs.rice.edu. After connecting to Rice, cd to /public/HPFF/hpf2.0-draft The individual LaTeX files are available in this directory and its subdirectories (such as Elemental). hpf-report.dvi is also available in this directory. It is a 200-page DVI file, so exercise due care when printing it. A GZIP'ed TAR file of the entire document (including the .tex and .dvi files) is in Releases/27-jun-96.tar.gz. I hope that's clear. I'm leaving town tomorrow, and won't have a way to fix things if they're broke. Chuck --------------------------------------------------------------------------- To (un)subscribe to this list, send mail to hpff-doc-request@cs.rice.edu. Leave the subject line blank, and in the body put the line (un)subscribe ---------------------------------------------------------------------------