Subject: hpff-doc: f77-ext/alias f77-local-library-ext/and multiple apologies

From: Carol Munroe <[email-masked]>
Date: Mon, 7 Oct 96 13:22:41 EDT

---------------------------------------------------------------------------
[email-masked] is a mailing list for HPF 2.0 language specification
authors and editors.  Instructions for adding or deleting yourself from this
list appear at the bottom of this message.
---------------------------------------------------------------------------
Here is my latest version of the Fortran 77 Local Library external
extrinsic, or whatever, currently described in hpf-report.tex as
Annex G. I apologize (#1) for its being so late.

I have been using the filename "f77-local-library-ext.tex", to avoid
confusing myself with the other F77_LOCAL piece in chapter 12, but it is
actually referred to as "fortran77-ext.tex" in the current hpf-report.tex
(in \includeonly and \include lines). I apologize (#2) for the name change.

I also apologize (#3) for the fact that it repeats programming
examples currently included in Chapter 12's section introducing
the F77_LOCAL interface. I am in the process of simplifying or
cutting out the other copies (but am open to suggestions).

Carol

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%file f77-local-library-ext.tex

\section{The FORTRAN~77 Local Library}
\label{ch-f7-local-library-ext}  

The HPF standard now describes an {\tt
EXTRINSIC(LANGUAGE='F77',MODEL='LOCAL')} interface ({\tt
EXTRINSIC(F77\_LOCAL)}, for short), similar in characteristics to the {\tt
EXTRINSIC('HPF\_LOCAL')} and {\tt EXTRINSIC('FORTRAN\_LOCAL')} interfaces
(see Section~\ref{ch-f77-local-ext} for its description). This section
describes a set of library routines to make it easier to make use of the
{\tt EXTRINSIC(F77\_LOCAL)} interface on many types of distributed arrays, for
any implementation on which this extension is implemented.  These library
routines can facilitate, for example, a portable blend of global data
parallel code with preexisting FORTRAN~77-based code using explicit message
passing calls for interprocessor communication.

The FORTRAN~77 Local Library interface described in this section has been
implemented for Thinking Machines Corporation's TMHPF Global Works Systems.
Additional documentation, requests, and suggestions can be made to

\begin{tabbing}
---------\=---------\=---\=---\=---\=------------\=\kill
\> Thinking Machines Corporation \\
\> 14 Crosby Drive \\
\> Bedford, MA 01730 \\
\> (617) 276-0400 \\
\> [email-masked] \\
\end{tabbing}

\subsection{Introduction}

The basic constraints for {\tt EXTRINSIC(MODEL='LOCAL')} together with these
argument passing options define the nature of the {\tt F77\_LOCAL}
interface: how control is to be transferred from a global HPF procedure to
a set of local procedures described by an {\tt EXTRINSIC(F77\_LOCAL)}
procedure interface and how data can be passed between the two types of
procedure: by reference or by descriptor, and with or without temporary
local reordering of data to satisfy FORTRAN~77 provisions for sequential,
contiguous storage of array data in Fortran array element order.
These alternative methods of argument passing can be obtained by use
of the mapping attributes {\tt LAYOUT('F77\_ARRAY')} (the default) vs.
{\tt LAYOUT('HPF\_ARRAY')}, and the attributes {\tt PASS\_BY('*')} (the
default) vs. {\tt PASS\_BY('HPF\_HANDLE')}.

However, to take advantage of the option allowing one to pass global HPF
array 'handles' to local FORTRAN~77 procedures and then obtain information
locally about how the local portion of a given parallel array is actually
distributed requires special inquiry routines comparable to the HPF Local
Library of functions. Since this library is not only described as a module,
but uses many features such as array-valued functions and optional
arguments not available in FORTRAN~77 code, it is recommended that a
modified FORTRAN~77 interface to this library be provided in the manner
described below.

Furthermore, there is the problem of describing local portions of parallel
arrays in the FORTRAN~77 code used in each local routine called from a
global HPF one. Since assumed-shape syntax may not be used, explicit shape
arrays are required. But it is common for global distribution of arbitrary
sized arrays to result in local portions of arrays that do not have
constant shapes on all processors, and the actual extents in each processor
cannot always be predicted in advance. In order to allow programmers to
obtain axis extent information at run time from the HPF global caller, a
special subgrid inquiry subroutine is provided. A FORTRAN~77 callable
version of the same routine is also described below, for flexibility in
programming.

\subsection{Summary}

\begin{itemize}

\item{One HPF-callable subgrid inquiry subroutine}

\subitem {\tt HPF\_SUBGRID\_INFO}

\item{A set of FORTRAN~77-callable inquiry functions}

\subitem {\tt F77\_SUBGRID\_INFO}
\subitem {\tt F77\_GLOBAL\_ALIGNMENT}
\subitem {\tt F77\_GLOBAL\_DISTRIBUTION}
\subitem {\tt F77\_GLOBAL\_TEMPLATE}
\subitem {\tt F77\_ABSTRACT\_TO\_PHYSICAL}
\subitem {\tt F77\_PHYSICAL\_TO\_ABSTRACT}
\subitem {\tt F77\_LOCAL\_TO\_GLOBAL}
\subitem {\tt F77\_GLOBAL\_TO\_LOCAL}
\subitem {\tt F77\_LOCAL\_BLKCNT}
\subitem {\tt F77\_LOCAL\_LINDEX}
\subitem {\tt F77\_LOCAL\_UINDEX}
\subitem {\tt F77\_GLOBAL\_SHAPE}
\subitem {\tt F77\_GLOBAL\_SIZE}
\subitem {\tt F77\_SHAPE}
\subitem {\tt F77\_SIZE}
\subitem {\tt F77\_MY\_PROCESSOR}

\end{itemize}

\subsection{HPF-Callable Subgrid Inquiry Subroutine}

The following subroutine can be implemented as a supplement to the standard
HPF LIBRARY module in order to provide additional processor-by-processor
detail on locally subgrid of distributed arrays. This information is
particularly useful when passing such arrays to {\tt
EXTRINSIC(MODE='LOCAL')} procedures written in FORTRAN~77, since in that
language array sizes must be stated explicitly in the local procedure, but
may be expressed in terms of arguments to the procedure that are passed at
run time (``adjustable shape arrays''). Thus the subgrid parameters
obtained from this routine can be passed as arguments to the local routines
and used there to describe the extents of the locally visible portions of
global HPF arrays.

{\tt HPF\_SUBGRID\_INFO (ARRAY, IERR, DIM, LB, UB, STRIDE, 
                 LB\_EMBED, UB\_EMBED, AXIS\_MAP)}

\begin{itemize}
\item Description.  Subgrid-inquiry subroutine.

\item {\tt ARRAY}  is an array of any type, size, shape, or distribution.
  It is an {\tt INTENT (IN)} argument.

\item {\tt IERR}  is a scalar integer.  It is an INTENT (OUT) argument.
  Its return value is zero upon successful return and nonzero
  otherwise.  Errors result if local subgrids cannot be expressed
  as array sections of {\tt ARRAY}.  If LB\_EMBED, UB\_EMBED, or AXIS\_MAP is specified,
  then errors also result if the compiler does not organize the
  local data in serial memory by sequence associating a larger
  ``embedding'' array.

\item {\tt DIM} (optional)  is a scalar integer.  Is is an {\tt INTENT (IN)}
  argument.

\item {\tt LB}, {\tt UB}, {\tt STRIDE},
  {\tt LB\_EMBED}, {\tt UB\_EMBED}, {\tt AXIS\_MAP}
  (all optional)  are
  integer arrays.  They are {\tt INTENT (OUT)} arguments.  If
  {\tt DIM} is specified, they are rank-one arrays of size
  {\tt NUMBER\_OF\_PROCESSORS()} and {\tt BLOCK} distribution.  If {\tt DIM}
  is not specified, there is a second, collapsed axis
  whose extent is at least the rank of the global array.  The return
  values are the lower and upper bounds and strides of the
  array sections describing the local data (in terms of global indices),
  the lower and upper bounds of the embedding arrays (again, in terms of
  global indices), and the axes of the embedding arrays to
  which the axes of {\tt ARRAY} are mapped.
\end{itemize}

\subsubsection{Restricted Form of the HPF Subgrid Inquiry Form}

In the common case that local subgrids are array sections of the global
user array, a simpler form of this routine may be used:

{\tt HPF\_SUBGRID\_INFO(ARRAY, IERR, DIM, LB, UB, STRIDE)}
\begin{itemize}
\item {\tt ARRAY} is an array of any type, size, shape, or
     distribution

\item {\tt IERR}  is an error-status return variable, which
     is zero upon successful return and nonzero otherwise

\item {\tt DIM}   is an optional argument indicating the axis
     along which return values are desired --- if no axis is
     specified, values are returned for all axes

\item {\tt LB}, {\tt UB}, {\tt STRIDE}   are all optional,
     {\tt INTENT (OUT)}, integer arrays.  If a {\tt DIM} argument
     is given, they are rank-one arrays of size
     {\tt NUMBER\_OF\_PROCESSORS()} and {\tt BLOCK} distribution.
     If no {\tt DIM} argument is given, there is a
     second, collapsed axis whose extent is at least the rank
     of the global array.
\end{itemize}

If {\tt HPF\_SUBGRID\_INFO()} is called and the elements of {\tt ARRAY}
that are local to any particular processor are not representable as an array
section of the global user array, then a nonzero value is returned for
{\tt IERR}.  Otherwise, if {\tt LB}, {\tt UB}, or {\tt STRIDE} is
specified, then the lower bounds, upper bounds, and/or strides,
respectively, that describe the local array sections are returned in terms
of one-based, global coordinates.

\subsubsection{More General Subgrid Inquiries Using Embedding Arrays}

Even if the global compiler does not sequence-associate local elements of
an array, it may ``embed'' these elements in a larger array section of
equal rank that {\it is} sequence-associated in local, serial memory.
For example, axes may be padded with ghost cells for stencil optimizations
or to achieve same-size subgrids on all nodes.  If this is the case, then
{\tt HPF\_SUBGRID\_INFO()} may still used to determine the local mapping
of the data through the use of additional arguments:

{\tt HPF\_SUBGRID\_INFO
(ARRAY, IERR, DIM, LB, UB, STRIDE, LB\_EMBED, UB\_EMBED, AXIS\_MAP)}
\begin{itemize}
\item {\tt LB\_EMBED}, {\tt UB\_EMBED}, {\tt AXIS\_MAP}
     These additional arguments are all optional,
     {\tt INTENT (OUT)}, integer arrays of the same rank and
     shape as {\tt LB}, {\tt UB}, and {\tt STRIDE}.
\end{itemize}

In this mapping, local memory has been allocated for a larger array section,
with coordinates {\tt (LB\_EMBEDI:UB\_EMBEDI:STRIDEI)} along axis {\tt I}.
The coordinates of the {\it actual} computational elements are limited to
the subset {\tt (LB:UB:STRIDEI)}.  The sequence association is generalized
to an arbitrary mapping of axes.  Here, {\tt AXIS\_MAP} numbers the axes
from fastest varying to slowest varying.

If {\tt LB\_EMBED}, {\tt UB\_EMBED}, or {\tt AXIS\_MAP} is specified
in a call to {\tt HPF\_SUBGRID\_INFO} but {\tt ARRAY} does not satisfy
the assumptions of this mapping model, then a nonzero value is returned for
{\tt IERR}.

\subsection{FORTRAN~77-Callable Inquiry Functions}

Here the F77-callable inquiry functions are described briefly.  These provide
essentially the same capability as the combination of the HPF intrinsic
array inquiry functions such as {\tt SHAPE} and {\tt SIZE} and {\tt
HPF\_LOCAL\_LIBRARY} inquiry routines, plus the convenient addition of
local subgrid information, provided by the subroutine {\tt
F77\_SUBGRID\_INFO}, the local counterpart of the globally callable {\tt
HPF\_SUBGRID\_INFO} described above. 

In all of the following:
\begin{itemize}
\item {\tt ARRAY}  is some dummy argument passed in from
  the HPF caller using the {\tt MAP\_TO(HPF\_ARRAY)}
  attribute and declared within the FORTRAN~77 local subroutine
  as an assumed-size, rank-one, integer array.  It is an
  {\tt INTENT (IN)} argument.

\item {\tt DIM}  is a scalar integer.  Is is an {\tt INTENT (IN)} argument.
  This argument specifies a particular axis of the global
  array associated with {\tt ARRAY} or, if {\tt DIM = -1}, inquiry is
  for all axes.

\item An ``inquiry result'' is an {\tt INTENT (OUT)} argument.  If {\tt DIM = -1},
  it is a rank-one array of size equal to at least the rank of
  the global array associated with {\tt ARRAY}, returning information
  associated with all axes.  If {\tt DIM} is positive, the ``inquiry
  result'' is a scalar, returning information only for the axis
  indicated by {\tt DIM}.

\item The arguments are defined in the same way as for the
  corresponding HPF or HPF\_LOCAL routines unless otherwise noted. See the
description of {\tt HPF\_SUBGRID\_INFO} above and
Section~\ref{HPF-LOCAL-LIBRARY} for full specifications of the
similarly-named {\tt HPF\_LOCAL\_LIBRARY} procedures.

\end{itemize}

\subsection{
{\tt F77\_SUBGRID\_INFO
(ARRAY, IERR1, IERR2, DIM, LB, UB, STRIDE, LB\_EMBED, UB\_EMBED, AXIS\_MAP)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF subroutine
  {\tt HPF\_SUBGRID\_INFO}.

\item {\tt IERR1}  is a scalar integer.  It is an {\tt INTENT (OUT)} argument.
  Its return value is zero if {\tt LB}, {\tt UB}, and {\tt STRIDE} were determined
  successfully and nonzero otherwise.

\item {\tt IERR2}  is a scalar integer.  It is an {\tt INTENT (OUT)} argument.
  Its return value is zero if {\tt LB\_EMBED} and {\tt UB\_EMBED} were
  determined successfully and nonzero otherwise.

\item {\tt LB}, {\tt UB}, {\tt STRIDE},
  {\tt LB\_EMBED}, {\tt UB\_EMBED}, {\tt AXIS\_MAP}
  are ``inquiry results'' of
  type integer.  They are the lower and upper bounds and strides
  of the array sections describing the local data (in terms of global indices),
  the lower and upper bounds of the embedding arrays (again, in terms of
  global indices), and the axes of the embedding arrays to which the
  axes of {\tt ARRAY} are mapped.
\end{itemize}

\subsection{
{\tt F77\_GLOBAL\_ALIGNMENT
(ALIGNEE, LB, UB, STRIDE, AXIS\_MAP, IDENTITY\_MAP, DYNAMIC, NCOPIES)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL subroutine
  {\tt GLOBAL\_ALIGNMENT}.  All but the first are {\tt INTENT (OUT)}
  arguments whose return values are as specified by the
  corresponding HPF routine.

\item {\tt ALIGNEE}  is a dummy argument passed in from global
  HPF.  It is an {\tt INTENT (IN)} argument.

\item {\tt LB}, {\tt UB}, {\tt STRIDE}, {\tt AXIS\_MAP}  are
  integer arrays of rank one.  Their
  size must be at least equal to the rank of the global HPF array
  associated with {\tt ALIGNEE}.

\item {\tt IDENTITY\_MAP}, {\tt DYNAMIC}  are scalar logicals.

\item {\tt NCOPIES}  is a scalar integer.
\end{itemize}

\subsection{
{\tt F77\_GLOBAL\_DISTRIBUTION
(DISTRIBUTEE, AXIS\_TYPE, AXIS\_INFO, PROCESSORS\_RANK, PROCESSORS\_SHAPE)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL subroutine
  {\tt GLOBAL\_DISTRIBUTION}.  All but the first are {\tt INTENT (OUT)}
  arguments whose return values are as specified by the
  corresponding HPF routine.

\item {\tt DISTRIBUTEE}  is a dummy argument passed in from global
  HPF.  It is an {\tt INTENT (IN)} argument.

\item {\tt AXIS\_TYPE}  is a {\tt CHARACTER*9} array of rank one.  Its size must be
  at least equal to the rank of the global HPF array associated with
  {\tt DISTRIBUTEE}.

\item {\tt AXIS\_INFO}  is an integer array of rank one.  Its size must be at
  least equal to the rank of the global HPF array associated with
  {\tt DISTRIBUTEE}.

\item {\tt PROCESSORS\_RANK}  is an integer scalar.

\item {\tt PROCESSORS\_SHAPE}  is an integer array of rank one.  Its size must
  be at least equal to the value returned by {\tt PROCESSORS\_RANK}.
\end{itemize}

\subsection{
{\tt F77\_GLOBAL\_TEMPLATE
(ALIGNEE, TEMPLATE\_RANK, LB, UB, AXIS\_TYPE, AXIS\_INFO, NUMBER\_ALIGNED, DYNAMIC)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL subroutine
  {\tt GLOBAL\_TEMPLATE}.  All but the first are {\tt INTENT (OUT)} arguments
  whose return values are as specified by the corresponding HPF
  routine.

\item {\tt ALIGNEE}  is a dummy argument passed in from global
  HPF.  It is an {\tt INTENT (IN)} argument.

\item {\tt TEMPLATE\_RANK}  is a scalar integer.

\item {\tt LB}, {\tt UB}, {\tt AXIS\_INFO}  are integer arrays of rank one.
  Their size must
  be at least equal to the rank of the align-target to which the global
  HPF array associated with {\tt ALIGNEE} is ultimately aligned.

\item {\tt AXIS\_TYPE}  is a {\tt CHARACTER*10} array of rank one.  Its size must be
  at least equal to the rank of the align-target to which the global
  HPF array associated with {\tt ALIGNEE} is ultimately aligned.

\item {\tt NUMBER\_ALIGNED}  is a scalar integer.

\item {\tt DYNAMIC}  is a scalar logical.
\end{itemize}

\subsection{
{\tt F77\_ABSTRACT\_TO\_PHYSICAL(ARRAY, INDEX, PROC)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL subroutine
  {\tt ABSTRACT\_TO\_PHYSICAL}.

\item {\tt INDEX}  is a rank-one, {\tt INTENT (IN)}, integer array.

\item {\tt PROC}  is a scalar, {\tt INTENT (OUT)}, integer.
\end{itemize}

\subsection{
{\tt F77\_PHYSICAL\_TO\_ABSTRACT(ARRAY, PROC, INDEX)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL subroutine
  {\tt PHYSICAL\_TO\_ABSTRACT}.

\item {\tt PROC}  is a scalar, {\tt INTENT (IN)}, integer.

\item {\tt INDEX}  is a rank-one, {\tt INTENT (OUT)}, integer array.
\end{itemize}

\subsection{
{\tt F77\_LOCAL\_TO\_GLOBAL(ARRAY, L\_INDEX, G\_INDEX)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL subroutine
  {\tt LOCAL\_TO\_GLOBAL}.

\item {\tt L\_INDEX}  is a rank-one, {\tt INTENT (IN)}, integer array.

\item {\tt G\_INDEX}  is a rank-one, {\tt INTENT (OUT)}, integer array.
\end{itemize}

\subsection{
{\tt F77\_GLOBAL\_TO\_LOCAL(ARRAY, G\_INDEX, L\_INDEX, LOCAL, NCOPIES, PROCS)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL subroutine
  {\tt GLOBAL\_TO\_LOCAL}.

\item {\tt G\_INDEX}  is a rank-one, {\tt INTENT (IN)}, integer array.

\item {\tt L\_INDEX}  is a rank-one, {\tt INTENT (OUT)}, integer array.

\item {\tt LOCAL}  is a scalar, {\tt INTENT (OUT)}, logical.

\item {\tt NCOPIES}  is a scalar, {\tt INTENT (OUT)}, integer.

\item {\tt PROCS}  is a rank-one, integer array whose size is at
  least the number of processors that hold copies of the
  identified element.
\end{itemize}

\subsection{
{\tt F77\_LOCAL\_BLKCNT(L\_BLKCNT, ARRAY, DIM, PROC)}
}
\begin{itemize}
\item Description.  FORTRAN~77interface-callable version of HPF\_LOCAL function
  {\tt LOCAL\_BLKCNT}.

\item {\tt L\_BLKCNT} is an ``inquiry result'' of type integer.

\item {\tt PROC} is a scalar integer.  It must be a valid processor
  number or, if {\tt PROC = -1}, the value returned by {\tt F77\_MY\_PROCESSOR()}
  is implied.
\end{itemize}

\subsection{
{\tt F77\_LOCAL\_LINDEX(L\_LINDEX, ARRAY, DIM, PROC)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL function
  {\tt LOCAL\_LINDEX}.

\item {\tt L\_LINDEX} is a rank-one, integer array of size equal to
  at least the value returned by {\tt F77\_LOCAL\_BLKCNT()}.

\item {\tt DIM} may not be -1.

\item {\tt PROC} is a scalar integer.  It must be a valid processor
  number or, if {\tt PROC = -1}, the value returned by {\tt F77\_MY\_PROCESSOR()}
  is implied.
\end{itemize}

\subsection{
{\tt F77\_LOCAL\_UINDEX(L\_UINDEX, ARRAY, DIM, PROC)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL function
  {\tt LOCAL\_UINDEX}.

\item {\tt L\_UINDEX} is a rank-one, integer array of size equal to
  at least the value returned by {\tt F77\_LOCAL\_BLKCNT()}.

\item {\tt DIM} may not be -1.

\item {\tt PROC} is a scalar integer.  It must be a valid processor
  number or, if {\tt PROC = -1}, the value returned by {\tt F77\_MY\_PROCESSOR()}
  is implied.
\end{itemize}

\subsection{
{\tt F77\_GLOBAL\_SHAPE(SHAPE, ARRAY)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL function
  {\tt GLOBAL\_SHAPE}.

\item {\tt SHAPE}  is a rank-one, integer array of size equal to at
  least the rank of the global array associated with {\tt ARRAY}.
  Its return value is the shape of that global array.
\end{itemize}

\subsection{
{\tt F77\_GLOBAL\_SIZE(SIZE, ARRAY, DIM)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF\_LOCAL function
  {\tt GLOBAL\_SIZE}.

\item {\tt SIZE}  is a scalar integer equal to the extent of axis {\tt DIM}
  of the global array associated with {\tt ARRAY} or, if {\tt DIM = -1},
  the total number of elements in that global array.
\end{itemize}

\subsection{
{\tt F77\_SHAPE(SHAPE, ARRAY)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF intrinsic {\tt SHAPE()},
  as it would behave as called from HPF\_LOCAL.

\item {\tt SHAPE}  is a rank-one, integer array of size equal to at
  least the rank of the subgrid associated with {\tt ARRAY}.
  Its return value is the shape of that subgrid.
\end{itemize}

\subsection{
{\tt F77\_SIZE(SIZE, ARRAY, DIM)}
}
\begin{itemize}
\item Description.  FORTRAN~77-callable version of HPF intrinsic {\tt SIZE()},
  as it would behave as called from HPF\_LOCAL.

\item {\tt SIZE}  is a scalar integer equal to the extent of axis {\tt DIM}
  of the subgrid associated with {\tt ARRAY} or, if {\tt DIM = -1},
  the total number of elements in that subgrid.
\end{itemize}

\subsection{
{\tt F77\_MY\_PROCESSOR(MY\_PROC)}
}
\begin{itemize}
\item Description.  FORTRAN~77-Callable version of HPF\_LOCAL function
  {\tt MY\_PROCESSOR}.

\item {\tt MY\_PROC}  is a scalar, {\tt INTENT (OUT)}, integer.  Its value is
  the identifying number of the physical processor from which
  this call is made.
\end{itemize}

\subsection{Programming Example Using {\tt HPF\_SUBGRID\_INFO)}}

\subsubsection{HPF Caller}

\begin{verbatim}
              program example

        ! declare the data array and a verification copy
              integer, parameter :: nx = 100, ny = 100
              real, dimension(nx,ny) :: x, y
        !hpf$ distribute(block,block) :: x, y

        ! the global sum will be computed
        ! by forming partial sums on the processors
              real partial_sum(number_of_processors())
        !hpf$ distribute partial_sum(block)

        ! local subgrid parameters are declared per processor
        ! for a rank-two array
              integer, dimension(number_of_processors(),2) ::
             & lb, ub, number
        !hpf$ distribute(block,*) :: lb, ub, number

        ! define interfaces

              interface

                extrinsic(f77_local) subroutine local1
             &   ( lb1, ub1, lb2, ub2, x )
                integer, dimension(:) :: lb1, ub1, lb2, ub2
                real x(:,:)
        !hpf$ distribute(block) :: lb1, ub1, lb2, ub2
        !hpf$ distribute(block,block) :: x
                end

                extrinsic(f77_local) subroutine local2(n,x,r)
                integer n(:)
                real x(:,:), r(:)
        !hpf$ distribute n(block)
        !hpf$ distribute x(block,block)
        !hpf$ distribute r(block)
                end

              end interface

        ! determine result using only global HPF

              ! initialize values
              forall (i=1:nx,j=1:ny) x(i,j) = i + (j-1) * nx
              ! determine and report global sum
              print *, 'global HPF result: ',sum(x)

        ! determine result using local subroutines

              ! initialize values ( assume stride = 1 )
              call HPF_subgrid_info( y, ierr, lb=lb, ub=ub )
              if (ierr.ne.0) stop 'error!'
              call local1( lb(:,1), ub(:,1), lb(:,2), ub(:,2), y )

              ! determine and report global sum
              number = ub - lb + 1
              call local2 ( number(:,1) * number(:,2) , y , partial_sum )
              print *, 'F77_LOCAL result #1 : ',sum(partial_sum)

              end
\end{verbatim}

\subsubsection{FORTRAN~77 callee}

\begin{verbatim}
              subroutine local1( lb1, ub1, lb2, ub2, x )

              real x ( lb1 : ub1 , lb2 : ub2 )

        ! get the global extent of the first axis
        ! this is an HPF_LOCAL type of inquiry routine with an ``F77_'' prefix
              call F77_global_size(nx,x,1)

        ! initialize elements of the array
              do j = lb2, ub2
                do i = lb2, ub2
                  x(i,j) = i + (j-1) * nx
                end do
              end do

              end

              subroutine local2(n,x,r)

        ! here, the correspondence to the global indices is not important
        ! only the total size of the subgrid is passed in
              real x(n)

              r = 0.
              do i = 1, n
                r = r + x(i)
              end do

              end
\end{verbatim}

\subsection{Programming Example Using F77-Callable Inquiry Subroutines}

This example performs only the initialization of the above example.  It
illustrates use of the F77-callable inquiry routines and the
addressing of data in terms of ``embedding arrays.''

\subsubsection{HPF Caller}

\begin{verbatim}
              program example

              integer, parameter :: nx = 100, ny = 100
              real, dimension(nx,ny) :: y
        !hpf$ distribute(block,block) :: y

        ! local subgrid parameters are declared per processor
        ! for a rank-two array
              integer, dimension(number_of_processors(),2) ::
             & lb, ub, lb_embed, ub_embed
        !hpf$ distribute(block,*) :: lb, ub, lb_embed, ub_embed

        ! define interfaces

              interface

                extrinsic(f77_local) subroutine local1(
             &   lb1, ub1, lb_embed1, ub_embed1,
             &   lb2, ub2, lb_embed2, ub_embed2, x )
                integer, dimension(:) ::
             &   lb1, ub1, lb_embed1, ub_embed1,
             &   lb2, ub2, lb_embed2, ub_embed2
                real, dimension(:,:), map_to(no_change) :: x
        !hpf$ distribute(block) :: lb1, ub1, lb_embed1, ub_embed1
        !hpf$ distribute(block) :: lb2, ub2, lb_embed2, ub_embed2
        !hpf$ distribute(block,block) :: x
                end

              end interface

        ! initialize values
        ! ( assume stride = 1 and no axis permutation )

              call HPF_subgrid_info( y, ierr,
             & lb=lb, lb_embed=lb_embed,
             & ub=ub, ub_embed=ub_embed)
              if (ierr.ne.0) stop 'error!'

              call local1(
             & lb(:,1), ub(:,1), lb_embed(:,1), ub_embed(:,1),
             & lb(:,2), ub(:,2), lb_embed(:,2), ub_embed(:,2), y )

              end
\end{verbatim}

\subsubsection{FORTRAN~77 callee}

\begin{verbatim}
              subroutine local1(
             & lb1, ub1, lb_embed1, ub_embed1,
             & lb2, ub2, lb_embed2, ub_embed2, x )

        ! the subgrid has been passed in its "embedded" form
              real x ( lb_embed1 : ub_embed1 , lb_embed2 : ub_embed2 )

        ! get the global extent of the first axis
        ! this is an HPF_LOCAL type of inquiry routine with an ``F77_'' prefix
              call F77_global_size(nx,x,1)

        ! otherwise, initialize elements of the array
        ! loop only over actual array elements
              do j = lb2, ub2
                do i = lb2, ub2
                  x(i,j) = i + (j-1) * nx
                end do
              end do

              end
\end{verbatim}

---------------------------------------------------------------------------
To (un)subscribe to this list, send mail to [email-masked].
Leave the subject line blank, and in the body put the line
(un)subscribe <email-address>
---------------------------------------------------------------------------