Subject: hpff-interpret: HPF: dimensioning a processor arrangement by number_of_processors()

From: Chuck Koelbel <[email-masked]>
Date: Sun, 13 Oct 1996 22:54:34 -0500

---------------------------------------------------------------------------
[email-masked] is a mailing list for corrections,
clarifications, and interpretations related to High Performance Fortran.
Instructions for adding or deleting yourself from this list appear at the
bottom of this message.
---------------------------------------------------------------------------

I'm forwarding this question (originally sent to the comp-fortran-90 list):

>Date: Fri, 11 Oct 1996 12:39:54 -0400
>From: Israel Gale <[email-masked]>
>To: [email-masked]
>Cc: [email-masked]
>Subject: HPF: dimensioning a processor arrangement by number_of_processors()
>Reply-To: [email-masked]
>X-Unsub: To leave, send text 'leave comp-fortran-90'
>	 to [email-masked]
>Sender: [email-masked]
>Precedence: list
>
>I have recently come across an annoying problem in HPF:  If you dimension
>a processor arrangement by number_of_processors(), the processor
>arrangement is an automatic data object according to the F90 standard, and
>is illegal in a main program.
>
>Examples like:
>
>!hpf$ processors P(number_of_processors())
>
>are constantly bandied about in official HPF publications (like the HPF
>Handbook), which tells me that this illegal example was designed to be one
>of the main intended uses of number_of_processors().
>
>One work-around that has been suggested is to write a main program that is
>nothing but a wrapper for a procedure which contains your "real" main
>program.  Automatic data objects are permitted in procedures, so the
>problem goes away.
>
>Does anyone see any disadvantages to this approach?
>
>-Israel Gale
> [email-masked]
>

Statement 1:
I'm not sure what constitutes an "official" publication, since HPFF is not
part of any recognized national or international standards body.  And the
Handbook, while written by members of HPFF, was not approved by the HPFF
committee.  But it is true that this sort of example has shown up in a lot
of places.

Statement 2:
Another workaround is to use a parameter statment.
	PARAMETER (NP=NUMBER_OF_PROCESSORS())
	!HPF$ PROCESSORS PROCS(NP)
I believe this is allowed everywhere.

Statement 3:
I don't see any disadvantage to your approach.

Of course, I'm sure others will have some additional comments.

						Chuck

**********************************************************************
Charles Koelbel				CITI/CRPC, MS 41
Center for Research on Parallel Computation		Rice University
Rice University				6100 Main Street
[email-masked]				Houston, TX 77005
**********************************************************************

---------------------------------------------------------------------------
To (un)subscribe to this list, send mail to
[email-masked].
Leave the subject line blank, and in the body put the line
(un)subscribe <email-address>
---------------------------------------------------------------------------

---

From: [email-masked] (Michael Hennecke)
Date: Mon, 14 Oct 1996 11:14:18 +0200 (CES)

> I believe this is allowed everywhere.

No, this is even worse and also flagged by all the compilers I have: 
An <initialization> must contain an <initialisation-expr> (as the name 
implies :-), and NUMBER_OF_PROCESSORS() isn't one.

An <explicit-shape-spec> may contain a <specification-expr>, which is
much less restrictive. Only if the bounds are NONCONSTANT specification
expressions does the above problem arise. For example

  PROGRAM constant
    REAL :: a(100)
    REAL :: b(SIZE(a))
  END PROGRAM constant

is completely legal Fortran. I think it must be stated in the HPF spec 
that NUMBER_OF_PROCESSORS() belongs to the list of intrinsics which may
be used in a <<restricted expression>>. This list is in F95, 7.1.6.2., and
the HPF spec may wish to extend this list.

Best regards,
Michael

 ======================================================================
  Michael Hennecke      http://www.uni-karlsruhe.de/~Michael.Hennecke/ 
 ----------------------------------------------------------------------
  University of Karlsruhe         RFC822: [email-masked] 
  Computing Center (G20.21 R210)               No longer on BITNET :-(
  Zirkel 2  *  P.O. Box 69 80                 Phone: +49 721  608-4862 
  D-76128  Karlsruhe                               Fax: +49 721  32550 
 ======================================================================
---------------------------------------------------------------------------
To (un)subscribe to this list, send mail to
[email-masked].
Leave the subject line blank, and in the body put the line
(un)subscribe <email-address>
---------------------------------------------------------------------------

---

From: Larry Meadows <[email-masked]>
Date: Mon, 14 Oct 1996 09:59:33 -0700 (PDT)

> implies :-), and NUMBER_OF_PROCESSORS() isn't one.

I'm almost certain that it was the intent of the Forum for
number_of_processors() to be a *constant* expression.

This program compiles fine with pghpf:

!hpf$ processors p(number_of_processors())
	real a(100)
	common /junk/ a
!hpf$ distribute (block) onto p:: a
	a = 100
	print *,a
	end

It also compiles fine with IBM's xlhpf.

Digital's compiler, however, says:

f90: Error: bug.f90, line 1: Explicit-shape-specs in a PROCESSORS directive must be constant expressions in a main program.   [P]
!hpf$ processors p(number_of_processors())

lfm

---

From: Piyush Mehrotra <[email-masked]>
Date: Mon, 14 Oct 1996 14:05:06 -0400 (EDT)

Here is the relevant text from Section 5 (page 91) Version 1.1 of 
the HPF document:

    The values returned by the system inquiry intrinsic functions remain
    constant for the duration of one program execution.  Thus, {\tt
    NUMBER_OF_PROCESSORS} and {\tt PROCESSORS_SHAPE} have values that are
    restricted expressions and may be used wherever any other Fortran
    restricted expression may be used.  In particular, {\tt
    NUMBER_OF_PROCESSORS} may be used in a specification expression.

    The values of system inquiry functions may not occur in initialization
    expressions, because they may not be assumed to be constants.   In
    particular, HPF programs may be compiled to run on machines whose
    configurations are not known at compile time.

Thus, it was clearly the intent of the forum to *not* make it a constant
expression (see reasoning above).

	- Piyush

---

From: Israel Gale <[email-masked]>
Date: Mon, 14 Oct 1996 14:00:24 -0400

> !hpf$ processors p(number_of_processors())

Let me re-phrase Larry's comment:  I think Larry is interpreting the HPF
Specification to relax the F90 standard and allow number_of_processors()
wherever a constant expression is allowed, even if it is not actually
known at compile time.  In other words, automatic data objects are not
allowed in main programs, unless they are dimensioned with
number_of_processors(), in which case they _are_ allowed.  This is
apparently IBM's interpretation, as well. 

Digital's compiler interprets the F90 standard strictly, and gives the
error message that Larry reported, whenever the value of
number_of_processors() is not known at compile time.

Digital has a compile-time option that produces an executable that is
optimized for a specified number of processors.  When this option is used,
the value number_of_processors() _is_ known at compile time, and we treat
it as a constant, allowing number_of_processors() to dimension data
objects even in a main program.

If I got this right, it means that our differing implementations have
uncovered an ambiguity in the HPF Specification.

-Israel Gale
 [email-masked]

---

From: Israel Gale <[email-masked]>
Date: Mon, 14 Oct 1996 14:30:14 -0400

>     NUMBER_OF_PROCESSORS} may be used in a specification expression.

I'm afraid the language of the spec does little to clarify this issue.
The spec does not distinguish between main programs and procedures.  It is
not clearly stated here that this was meant to relax the F90 rule on
automatic data objects.

-Israel Gale
 [email-masked]

---

From: Dave Watson <[email-masked]>
Date: Tue, 15 Oct 1996 14:01:26 +0100 (BST)

> not clearly stated here that this was meant to relax the F90 rule on

Read in cojunction with section 5.1.2.4.1 "Explicit-shape array" of
the F90 standard I believe that text from Section 5 is unambiguous
and that NUMBER_OF_PROCESSORS is not allowed to dimension an array in the
main program. The relevant constraint is:

"  An explicit-shape array whose bounds depend on the values of non-constant
expressions must be a dummy argment, a function result or an automatic
array of a procedure."
           ---------

If NUMBER_OF_PROCESSORS were to be considered a constant expression this
would require that both SAVE'd and COMMON arrays could also be dimensioned
with expressions dependent on NUMBER_OF_PROCESSORS. I do not think that 
this is intended.  

cheers

 Dave Watson

---

From: <[email-masked]> (Henry Zongaro)
Date: Tue, 15 Oct 1996 10:32:56 -0400 (EDT)

Hello,

> not clearly stated here that this was meant to relax the F90 rule on

     Sorry I'm late jumping into this thread - yesterday was Thanksgiving Day
here in Canada.

     Interpretation Item 13 against HPF 1.1 deals with this issue.  The final
reponse was to delete the paragraph that Piyush cites above and the paragraph
that follows it, and to replace them with the following text:

       "The Fortran 90 definition of restricted expression is extended to
        permit references to the HPF system inquiry intrinsic functions.
        In particular, at the end of the numbered list in 7.1.6.2
        [of Fortran 90], add:

           (11) A reference to one of the system inquiry functions
                NUMBER_OF_PROCESSORS and PROCESSORS_SHAPE, where any
                argument is a restricted expression.

        A variable which appears in a restricted expression in an HPF directive
        in the scoping unit of a module or main program must be an implied-DO
        variable, or must be an argument in a reference to an array inquiry
        function, bit inquiry function, character inquiry function, kind
        inquiry function or numeric inquiry function."

     NUMBER_OF_PROCESSORS or PROCESSORS_SHAPE were not added to the list of
functions permitted in constant expressions or initialization expressions, so
they are not permitted as part of a specification expression for array bounds
or character lengths in main program or module scoping units.  So there's no
problem with ordinary data objects here.

     We don't want template or processor arrangement bounds to be similarly
restricted.  Since they are not automatic objects (templates and processors
are not objects, so they can't be automatic objects), there's no problem with
declaring them to have bounds that are not constant specification expressions
in the main program or a module scoping unit.  The added restriction that
appears beneath item (11) ensures that the only non-constant parts of such an
expression come in through references to NUMBER_OF_PROCESSORS or
PROCESSORS_SHAPE.  The following problematic example is prohibited by the
above wording:

        module mod1
          integer :: i
        end module mod1

        module mod2
          use mod1
  !hpf$   processors proc(i, number_of_processors()/i)
        end module mod2

        program p
          use mod1
          i = 1
          call sub
          i = 0
          call sub
        end program p

        subroutine sub
          use mod2             ! Do the bounds of proc change?
        end subroutine sub

     Hope this helps.

Thanks,

Henry

---

From: David Presberg <[email-masked]>
Date: Tue, 15 Oct 1996 11:13:42 -0400

Folks --

I am not clear as to why the discussion has shifted from the original
concern:

  > ...If you dimension
     ^^^^^^^^^^^^^^^^^^^^^
  >is illegal in a main program.   
     [from Message-Id: <v03007825ae87686573fc@[128.42.5.118]>, Koelbel
     (for Israel Gale <[email-masked]>), Sun, 13 Oct 1996 22:54:34 -0500;
     my emphasis]

to a concern about dimensioning an ARRAY using number_of_processors().

The latter issue is clearly covered by the definitions in the F90
(soon to be F95) standard, and would hang on the status of
number_of_processors() as defined in HPF ("constant expression"
vs. "restricted expression", both of which are semantic notions, not
strictly c.f.g. general class names).  Note that there are non-HPF
inquiry functions in F90 that fall in the category "constant
expression" when their arguments are appropriate, and in
"specification expression" similarly.

If HPFF proposed to change the text of HPF 1.1, Section 5.2, page 91,
lines 39 .. 43, (quoted by Chuck Koelbel) to assert that
number_of_processors() was in the "constant expression" category per
F90, Section 7.1.6.1, much of the discussion about ARRAY declarations
would shift accordingly.  And the note about that function controlling
the declared extents of COMMON arrays (why the extra issue about
SAVE?) would have to be accomodated by HPF vendors' run time systems:
they either would have the infrastructure in place to accomodate it or
not, and would vote accordingly in HPFF.  (I give a talk about HPF in
which I have the aside: "HPF COMMON is not your grandfather's ...")

But a "processor arrangement" is entirely an HPF idea, and there isn't
any declared, instantiated, program object that corresponds.  Again, I
fully understand how various compilation strategies (and supporting
run time systems) could prefer that all the details of a processor
arrangement be known at compile time, and some others might be capable of
handling the more dynamic (from run to run) unknown when processor
arrangement depends on the encountered number of processors at the
particular run.  But I don't see how that distinction is any different
than using number_of_processors() in, say, the int-expr of a
dist-format (see HPF 1.1, section 3.2, syntax rule H309, page 26,
lines 41 .. 44 (approx)).  That is, one can pick apart a
number_of_processors() quantity of "nodes" to set up one's favorite
BLOCK(N) parameter.

Do the compilers (more than one was implicated) that refuse:

  !HPF$ PROCESSORS R(8, NUMBER_OF_PROCESSORS()/8)

also refuse:

  !HPF$ DISTRIBUTE A(BLOCK(8), BLOCK(NUMBER_OF_PROCESSORS()/8) )

??

-- Pres
- ------------------------------------------------------------------------
- David L. Presberg, Parallel Computing Specialist, CTC/ST/Tools
-   Cornell Theory Center, 740 Frank H.T. Rhodes Hall, Cornell University,
- Ithaca, NY 14853-3801, USA  607-254-8861  [email-masked]
- Nickname:  Pres
- ------------------------------------------------------------------------
---------------------------------------------------------------------------
To (un)subscribe to this list, send mail to
[email-masked].
Leave the subject line blank, and in the body put the line
(un)subscribe <email-address>
---------------------------------------------------------------------------

---

From: Israel Gale <[email-masked]>
Date: Tue, 15 Oct 1996 11:45:11 -0400

Henry Zongaro <[email-masked]> writes:

>      Interpretation Item 13 against HPF 1.1 deals with this issue. . . .
> 
> PROCESSORS_SHAPE.  

Thanks to Henry for pointing out this interpretation that we somehow
missed.  To date, this has not been a problem for our users, probably
because our users generally use our feature that allows them to provide
number of processors information on the compile-time command line (we then
treat number_of_processors() as a constant).

We will, of course, incorporate this interpretation into our next release.

-Israel Gale
 [email-masked]

---

From: Dave Watson <[email-masked]>
Date: Tue, 15 Oct 1996 17:55:21 +0100 (BST)

> "specification expression" similarly.

The  non-HPF inquiry functions in F90 that fall in the category "constant
expression" when their arguments are appropriate are evaluable by a
compiler at compile time. The same is not true for number_of_processors().

> SAVE?) 

SAVE'd objects are restricted in F90 similarly to objects in COMMON

> would have to be accomodated by HPF vendors' run time systems:

I agree that non-sequential COMMON in HPF is a different beast to 
sequential F90 common. My difficulty with accepting number_of_processors() 
as a constant is the interaction with the sequential aspects of F90.

If HPFF asserts that number_of_processors() is in the "constant expression"
category then this affects the F90 features which can currently rely on the
fact that bounds are statically determinable. Allowing number_of_processors()
in an initialisation-expression means that COMMON data objects and SAVE data 
objects are potentially no longer statically sized. This opens up "dynamic 
equivalence" relations since
  REAL   c(number_of_processors())
  COMMON /VARIABLE/a(number_of_processors()),b(10)
  EQUIVALENCE (b(1),c(1))
now becomes valid. This is, in my opinion, an unnecessary deviation from F90
since F90 is very specific about avoiding such a feature. 

cheers

  Dave Watson