Subject: hpff-interpret: external issues: extrinsic(f77_local)

From: [email-masked]
Date: Mon, 4 Mar 1996 16:02:32 -0500

---------------------------------------------------------------------------
[email-masked] is a mailing list for corrections,
clarifications, and interpretations related to High Performance Fortran.
Instructions for adding or deleting yourself from this list appear at the
bottom of this message.
---------------------------------------------------------------------------

Hi,

I am a research associate with Jack Dongarra.  I have been looking at providing
an HPF interface to ScaLAPACK, our linear algebra Fortran77 message passing
library.  In the course of this work, I've run into several problems involved
in calling an extrinsic routine.  Most of these problems seem to be in the
present generation of compilers.  However, I've run into one that seems to be
a function of the HPF specification, which I believe could be fixed by a small
addition to the extrinsic section.  I include below a small latex note on the
issue.  I would be happy to provide further information or help if I can.

I have also been reading the HPFF minutes for information on how HPF is going
to look in the future.  I was rather alarmed to note that kernel HPF appears
to be heading towards no support for cyclic(n).  I assume the committee is
assuming that algorithmic blocking will ameliorate the performance hit for
such a decision?  I know that our own library does not yet support algorithmic
blocking for many routines; where it is absent, a cyclic(1) distribution yields
minimal performance . . .  I am not abreast of all of the literature on the
subject, but it is not clear to me that all algorithms can efficiently use
algorithmic blocking . . .

Perhaps I don't understand what kernel HPF really is.  Is it HPF2.0's subset HPF?
Or is it features the compiler should support efficiently, with other features
supported but not with the same degree of optimality?

Thanks for your help,
Clint

\documentstyle[11pt]{article}
\topmargin .5in
\pagestyle{myheadings}
\markboth{\underline{\bf DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT}}
         {\underline{\bf DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT}}

\textwidth=6in
\textheight=8.5in
\hoffset = -.6in
\voffset = -.6in

\title{Passing multidimensional arrays from HPF to extrinsic routines}

\author{
R. Clint Whaley
\thanks{
Dept. of Computer Sciences, Univ. of TN,
Knoxville, TN 37996, {\tt [email-masked]}
}
}

\begin{document}
\maketitle

\begin{abstract}
Under the present extrinsic interface rules, it is not possible to portably
pass multidimensional arrays from HPF to a language such as Fortran77 or C.
In this paper we discuss the problem and suggest some solutions.
\end{abstract}

\vspace*{1in}

\centerline{ \bf \Large \underline{DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT} }
\newpage

We believe that an important area of interfacing HPF routines and local
Fortran77 has been overlooked.  The problem involves the translation of
a HPF/F90 assumed {\em shape} array to a F77 assumed {\em size} array.

This problem may be overcome in F90 by specifying an explicit interface.
This approach is illustrated in figure~\ref{fig-f902f77}.
Obviously, if the compiler has not already stored the indicated (sub)array
in an array whose first dimension is equal to the {\tt SIZE} of the array,
it will have to allocate space and copy the array.

\begin{figure}
\begin{verbatim}
SUBROUTINE F90_TO_F77(A)
REAL, INTENT(INOUT) :: A(:,:)

INTEGER :: M, N, LDA

INTERFACE
   SUBROUTINE F77ROUT(M, N, A, LDA)
   INTEGER, INTENT(IN) :: M, N, LDA
   REAL, INTENT(INOUT) :: A(LDA,*)
   END SUBROUTINE F77ROUT
END INTERFACE

M = SIZE(A, 1)
N = SIZE(A, 2)
LDA = M

CALL F77ROUT(M, N, A, LDA)

RETURN
END
\end{verbatim}
\caption{A F90 routine passing a 2D array to a F77 routine\label{fig-f902f77}}
\end{figure}

Note that in distributed memory terms, leading dimension (\verb+LDA+ in the above
example) is an inherently {\em local} quantity: it indicates the memory stride
between elements in a row.  Thus the F90 approach cannot be directly utilized in
HPF, since the interface described by HPF is {\em global}.

This oversight means that only 1D arrays may be standardly passed from HPF to
external languages such as C or Fortran77 (this is obviously true because it is
impossible to determine how the compiler has locally laid out the matrix, and
thus it will be impossible to do indexing on such arrays).  This means that
codes wishing to pass arrays with greater than 1 dimension will have to be
compiler specific (actually, compiler version specific).

Compilers supporting extrinsic(f90\_local) should not have this problem.  There,
the user may pass the HPF matrix to a f90\_local routine which takes the matrix
as assumed shape.  The f90\_local routine may then use the code in
figure~\ref{fig-f902f77} to pass the local array to the Fortran77 code.

Compilers which support only f77\_local will not have this option.  This makes
f77\_local less useful as an interface.  Obviously, specific extrinsics are
not part of the HPF standard, so it is not surprising that this issue has not
been addressed by HPF.  However, just as the local library can be part of
the standard for those compilers supporting hpf\_local, a solution to this
problem can be adopted as part of the standard for those compilers choosing
to support f77\_local.

We propose a two-fold solution to this problem.  The first takes the form of
an added distribution directive: \\
{\tt !HPF\$ ASSUMMED\_SIZE :: A(LD1, LD2, $\dots$, *)} \\
This command, which is legal only in an interface to an extrinsic routine
accepting assumed size arrays, asserts that the dimensions of the local
matrix match the dimensions specified by the parameters {LD1, LD2, \dots}.

This is the minimal extension required to make multidimensional arrays usable
across languages.  The user can fill in minimal values for {\tt LDx} by the
appropriate call to {\tt SIZE} from an {\tt HPF\_LOCAL} routine (assuming the
compiler supports {\tt HPF\_LOCAL}).  This fix has a weakness in
common with that used by F90, however.  To whit, if the compiler is currently
storing the array with a non-minimal dimension, a data copy will be dictated by
the interface which, at least theoretically, could have been avoided.

In today's compilers, data copies occur on most subarray accesses, so this point
is not as significant as it might be.  However, as compilers become more efficient,
this should no longer be the case, and the implied data copy in forcing the user
to ``guess'' the dimension presently being used may become significant.  We
therefore believe that the HPF standard should encourage vendors supporting
extrinsic interfaces to include the routine:\\
{\tt INTEGER FUNCTION LOCAL\_DIM(ARRAY, DIM)}\\
\verb+   +Returns the local number of elements of ARRAY along the axis DIM.

With this routine, the user can query the dimensions of the array the compiler
will use if array ARRAY is passed to an extrinsic routine.  This should allow
the user to find the correct dimensions to avoid dictating a data copy, if he
so chooses.
\end{document}
---------------------------------------------------------------------------
To (un)subscribe to this list, send mail to
[email-masked].
Leave the subject line blank, and in the body put the line
(un)subscribe <email-address>
---------------------------------------------------------------------------