Subject: hpff: Alignment of a single dimension

From: Adriaan Joubert <[email-masked]>
Date: Wed, 6 Sep 1995 11:34:49 +0100

---------------------------------------------------------------------------
[email-masked] is a mailing list for announcements related to High
Performance Fortran.  Instructions for adding or deleting yourself
from this list appear at the bottom of this message.
---------------------------------------------------------------------------

Hello,

	I am trying to align one dimension of two 2-dimensional arrays, and
cannot find a way of expressing this in HPF. The problem is the following

PROGRAM MAIN
   REAL, ALLOCATABLE :: A(:,:), Res(:,:)
   !HPF$ DISTRIBUTE A(BLOCK,*)
   !HPF$ DISTRIBUTE Res(BLOCK,*)

   ...
   ALLOCATE(A(N,M))
   ALLOCATE(Res(N,M*2))
   Res = SUB (A)
   ...
CONTAINS

   FUNCTION SUB (A) RESULT(B)
     REAL, INTENT(in) :: A(:,:)
     !HPF$ DISTRIBUTE *(BLOCK,*) :: A
     REAL :: B(SIZE(A,1),SIZE(A,2)*2)
     !HPF$ DISTRIBUTE (BLOCK,*) :: B

     ...
   END FUNCTION SUB
END PROGRAM MAIN

So the 1st dimension of B and A in the subroutine will be distributed in
the same way. It seems however that compilers can generate faster code, if
they know how arrays are aligned with one another. Among other things the
compiler would have to know that B is exactly aligned with Res. In other
words I would like to add to the main program something like

	!HPF$ TEMPLATE :: MYTemp(N,M*2)
        !HPF$ DISTRIBUTE MyTemp(BLOCK,*)
	!HPF$ ALIGN WITH MyTemp(:,I) :: A(:,I)
	!HPF$ ALIGN WITH MyTemp :: Res

and in the subroutine

	!HPF$ TEMPLATE :: MYTemp(SIZE(A,1),SIZE(A,2)*2)
	!HPF$ DISTRIBUTE MyTemp(BLOCK,*)
	!HPF$ ALIGN WITH *MyTemp(:,I) :: A(:,I)
	!HPF$ ALIGN WITH MyTemp :: B

and then the compiler should be able to figure out that everything is
nicely aligned. But I cannot do this, as I do not know N and M at compile
time. 

In this case I could probably get away with definitions

        !HPF$ ALIGN WITH A(:,*) :: Res(:,*)

in the main program and

	!HPF$ ALIGN WITH A(:,*) :: B(:,*)

in the subroutine. The replication of the second dimension would not
matter, as it is all on the same processor. 

But if I have a (BLOCK,BLOCK) distribution for both arrays, but I still
want to ensure that all elements in the first section of every column are
on the same row of processors, i.e.

	REAL A(20,20), B(20,40)

	P1: A(1:10,1:10)	P2: A(1:10,11:20)
            B(1:10,1:20)	    B(1:10,21:40)

        P3: A(11:20,1:10)       P4: A(11:20,11:20)
	    B(11:20,1:20)	    B(11:20,21:40)

there seems to be no way of telling the compiler about this with a
descriptive statement. 

Well, what about
	!HPF$ ALIGN WITH A(:,I) :: B(:,I*2-1)
	!HPF$ ALIGN WITH A(:,I) :: B(:,I*2)
But is this legal? And this can be harder to do if the second dimension is
(SIZE(A,2)-1)*SIZE(A,2)/2, as in my case.

I'd appreciate any help on this one. Understanding the distribution
directives seems to get harder the more you know, instead of easier ;-(

Adriaan

--------------------------------------+-------------------------------
 Adriaan Joubert                      | Phone:  +44-171-975 5168
 London Parallel Applications Centre  | Fax:    +44-181-981 9803
 QMW, Mile End Road		      | e-mail: [email-masked]
 London E1 4NS			      |
--------------------------------------+-------------------------------
---------------------------------------------------------------------------
To (un)subscribe to this list, send mail to [email-masked].  Leave
the subject line blank, and in the body put the line
(un)subscribe <email-address>
---------------------------------------------------------------------------

---

From: [email-masked] (Rob Schreiber)
Date: Wed, 6 Sep 1995 10:56:30 -0700

Adriaan Joubert asks:

        ------------------------------------------------------------------------
		I am trying to align one dimension of two 2-dimensional arrays, and
	cannot find a way of expressing this in HPF. The problem is the following

	PROGRAM MAIN

	   ...
	   ALLOCATE(A(N,M))
	   ALLOCATE(Res(N,M*2))
	   Res = SUB (A)
	   ...
	CONTAINS

	     ...
	   END FUNCTION SUB
	END PROGRAM MAIN

	and in the subroutine

	in the main program and

I see you want allocatable templates, a hole in HPF that we know about.
But in your case, no template is needed.
There is no reason to specify replicationin your alignment.   You can use

!HPF$ ALIGN A(I,J) WITH RES(I,J)

and drop the distribute directive for A in the main program.   In the function,
you can use a distribute on B and a descriptive alignment of A to B.

		REAL A(20,20), B(20,40)

	Well, what about
		!HPF$ ALIGN WITH A(:,I) :: B(:,I*2-1)
		!HPF$ ALIGN WITH A(:,I) :: B(:,I*2)
	But is this legal? And this can be harder to do if the second dimension is
	(SIZE(A,2)-1)*SIZE(A,2)/2, as in my case.
No, it's not legal.   Align is not a symmetric relation, and the alignment
map cannot be many to one unless it collapses a dimension; at least that's my
understanding.   In case this is not clear, your statement is
equivalent to

!HPF$ ALIGN B(:,2*I-1) WITH A(:,I)

which only explicitly aligns the odd numbered columns of B!
But why align the bigger of the two arrays (B) with the smaller (A)?    
My version of SUB would be:

	   FUNCTION SUB (A) RESULT(B)
	     REAL, INTENT(in) :: A(:,:)
	     REAL :: B(SIZE(A,1),SIZE(A,2)*2)
	     !HPF$ ALIGN *(I,J) WITH B(I,J) :: A
	     !HPF$ DISTRIBUTE B (BLOCK,*)

Yup.

Rob

---

From: "Dr A.C. Marshall" <[email-masked]>
Date: Fri, 8 Sep 1995 16:50:10 +0100 (BST)

Can I continue this discussion for one more iteration? Good then I'll
begin.

As I understand it Adriaan Joubert wants to tell HPF to set the data up as
follows:

                REAL A(20,20), B(20,40)

I would say (like Rob Schreiber) that you include, in Main, the line 

	!HPF$ ALIGN A(I,J) WITH B(I,J*2-1)

or  equivalently

	!HPF$ ALIGN A(:,J) WITH B(:,J*2-1)

PLUS (distribute them together)

	!HPF$ DISTRIBUTE B (BLOCK,BLOCK)

And I would claim that the procedure should be coded as follows:

	   FUNCTION SUB (A) RESULT(B)
	     REAL, INTENT(in) :: A(:,:)
	     REAL :: B(SIZE(A,1),SIZE(A,2)*2)
	     !HPF$ ALIGN *(I,J) WITH B(I,J*2-1) :: A
	     !HPF$ DISTRIBUTE *B (BLOCK,BLOCK)

If I am wrong - why?

Adam

In the last mail Rob Schreiber said:
|
|Adriaan Joubert asks:
|
|        ------------------------------------------------------------------------
|		I am trying to align one dimension of two 2-dimensional arrays, and
|	cannot find a way of expressing this in HPF. The problem is the following
|
|	PROGRAM MAIN
|	   REAL, ALLOCATABLE :: A(:,:), Res(:,:)
|	   !HPF$ DISTRIBUTE A(BLOCK,*)
|	   !HPF$ DISTRIBUTE Res(BLOCK,*)
|
|	   ...
|	   ALLOCATE(A(N,M))
|	   ALLOCATE(Res(N,M*2))
|	   Res = SUB (A)
|	   ...
|	CONTAINS
|
|	   FUNCTION SUB (A) RESULT(B)
|	     REAL, INTENT(in) :: A(:,:)
|	     !HPF$ DISTRIBUTE *(BLOCK,*) :: A
|	     REAL :: B(SIZE(A,1),SIZE(A,2)*2)
|	     !HPF$ DISTRIBUTE (BLOCK,*) :: B
|
|	     ...
|	   END FUNCTION SUB
|	END PROGRAM MAIN
|
|	So the 1st dimension of B and A in the subroutine will be distributed in
|	the same way. It seems however that compilers can generate faster code, if
|	they know how arrays are aligned with one another. Among other things the
|	compiler would have to know that B is exactly aligned with Res. In other
|	words I would like to add to the main program something like
|
|		
|		!HPF$ TEMPLATE :: MYTemp(N,M*2)
|	        !HPF$ DISTRIBUTE MyTemp(BLOCK,*)
|		!HPF$ ALIGN WITH MyTemp(:,I) :: A(:,I)
|		!HPF$ ALIGN WITH MyTemp :: Res
|
|	and in the subroutine
|
|		!HPF$ TEMPLATE :: MYTemp(SIZE(A,1),SIZE(A,2)*2)
|		!HPF$ DISTRIBUTE MyTemp(BLOCK,*)
|		!HPF$ ALIGN WITH *MyTemp(:,I) :: A(:,I)
|		!HPF$ ALIGN WITH MyTemp :: B
|
|	and then the compiler should be able to figure out that everything is
|	nicely aligned. But I cannot do this, as I do not know N and M at compile
|	time. 
|
|	In this case I could probably get away with definitions
|
|	        !HPF$ ALIGN WITH A(:,*) :: Res(:,*)
|
|	in the main program and
|
|		!HPF$ ALIGN WITH A(:,*) :: B(:,*)
|
|	in the subroutine. The replication of the second dimension would not
|	matter, as it is all on the same processor. 
|
|I see you want allocatable templates, a hole in HPF that we know about.
|But in your case, no template is needed.
|There is no reason to specify replicationin your alignment.   You can use
|
|!HPF$ ALIGN A(I,J) WITH RES(I,J)
|
|and drop the distribute directive for A in the main program.   In the function,
|you can use a distribute on B and a descriptive alignment of A to B.
|
|	But if I have a (BLOCK,BLOCK) distribution for both arrays, but I still
|	want to ensure that all elements in the first section of every column are
|	on the same row of processors, i.e.
|
|		REAL A(20,20), B(20,40)
|
|		P1: A(1:10,1:10)	P2: A(1:10,11:20)
|	            B(1:10,1:20)	    B(1:10,21:40)
|
|	        P3: A(11:20,1:10)       P4: A(11:20,11:20)
|		    B(11:20,1:20)	    B(11:20,21:40)
|
|	there seems to be no way of telling the compiler about this with a
|	descriptive statement. 
|
|	Well, what about
|		!HPF$ ALIGN WITH A(:,I) :: B(:,I*2-1)
|		!HPF$ ALIGN WITH A(:,I) :: B(:,I*2)
|	But is this legal? And this can be harder to do if the second dimension is
|	(SIZE(A,2)-1)*SIZE(A,2)/2, as in my case.
|No, it's not legal.   Align is not a symmetric relation, and the alignment
|map cannot be many to one unless it collapses a dimension; at least that's my
|understanding.   In case this is not clear, your statement is
|equivalent to
|
|!HPF$ ALIGN B(:,2*I-1) WITH A(:,I)
|
|which only explicitly aligns the odd numbered columns of B!
|But why align the bigger of the two arrays (B) with the smaller (A)?    
|My version of SUB would be:
|
|	   FUNCTION SUB (A) RESULT(B)
|	     REAL, INTENT(in) :: A(:,:)
|	     REAL :: B(SIZE(A,1),SIZE(A,2)*2)
|	     !HPF$ ALIGN *(I,J) WITH B(I,J) :: A
|	     !HPF$ DISTRIBUTE B (BLOCK,*)
|
|	I'd appreciate any help on this one. Understanding the distribution
|	directives seems to get harder the more you know, instead of easier ;-(
|Yup.
|
|Rob
|
|

-- 
                                       |
     AC Marshall                       |
     Computing Services Department,    |
     University of Liverpool,          |
     PO Box 147.                       |
     L69 3BX                           |
     UK                                |
     email: [email-masked]            |
     Tel:    0151-794 3722             |
     Fax:    0151-794 3759             |
---------------------------------------------------------------------------
To (un)subscribe to this list, send mail to [email-masked].  Leave
the subject line blank, and in the body put the line
(un)subscribe <email-address>
---------------------------------------------------------------------------