Application:
GYRO
Simulation:
B2-cy
Performance
Tool: TAU
Number of
Processors: 16
Platform:
Teragrid at NCSA (Mercury: Itanium Cluster)
OUTPUT
FROM PPROF:
Reading Profile files in
profile.*
NODE 0;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 9.134E+05
6.729E+08 1 2808
672917690 .TAU application
89.5
8.334E+06 6.023E+08 1000 17008
602330 DO_FULLADVANCE
85.5
2.149E+08 5.753E+08 1000 28000
575298 TIMESTEP_SSP_322
31.8
1.993E+08 2.142E+08 3000 6000 71385 GET_KINETIC_ADVANCE
14.8
9.98E+07 9.98E+07 24000 0 4158 MPI_Alltoall()
6.9
4.161E+07 4.645E+07 1001 2002 46401 GET_FIELD_EXPLICIT
4.4
2.851E+07 2.982E+07 1 770
29816758 MAKE_MAXWELL_MATRIX
3.6
2.444E+07 2.444E+07 10979 0 2226 MPI_Allreduce()
2.6
1.705E+07 1.737E+07 10 700
1737015 WRITE_BIG
2.4
1.617E+07 1.617E+07 1 0
16172266 MAKE_LAMBDA_GRID
2.1
1.381E+07 1.42E+07 1 130
14196821 MAKE_POISSON_MATRIX
0.3
6.581E+05 2.073E+06 1000 1000 2073 CATCH_HALT_SIGNAL
0.3
1.859E+06 1.859E+06 15555 0 119 MPI_Recv()
0.2
1.659E+06 1.659E+06 1 8
1659225 MPI_Init()
0.2
1.427E+05 1.603E+06 1000 1000 1603 GET_ERROR
0.2
1.601E+06 1.601E+06 1122 0 1427 MPI_Bcast()
0.1
6.877E+05 6.877E+05 1 0
687724 MAKE_THETA_OPERATORS
0.1
1.582E+05 5.59E+05 1 257
559006 MAKE_BLEND_ARRAYS
0.1
4.696E+05 4.696E+05 1 0
469577 MAKE_PROFILES
0.1
3.879E+05 3.879E+05 1 0
387932 MAKE_RADIAL_OPERATORS
0.1
3.365E+05 3.672E+05 1 6
367178 MAKE_IMPLICIT_ADVECT
0.0
3.096E+04 3.494E+04 1 17 34943 WRITE_INPUT
0.0
1.461E+04 1.461E+04 2 0 7303 MPI_Comm_split()
0.0
1.338E+04 1.35E+04 1 4 13495 MPI_Finalize()
0.0
5756 5756 12 0 480 MPI_Reduce()
0.0
113 113 4 0 28 MPI_Keyval_free()
0.0
103 103 4 0 26 MPI_Attr_put()
0.0
101 101 4 0 25 MPI_Attr_get()
0.0 72 72 4 0 18 MPI_Comm_rank()
0.0
21 21 2 0 10 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 0,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
1.556E+04
8192 8 111.5
761.7 Message size received from
all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 1;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.205E+05
6.585E+08 1 2808
658504246 .TAU application
89.6
7.071E+06 5.898E+08 1000 2008
589818 DO_FULLADVANCE
88.2
2.173E+08 5.807E+08 1000 28000
580747 TIMESTEP_SSP_322
32.8
2.005E+08 2.159E+08 3000 6000 71973 GET_KINETIC_ADVANCE
15.2
1.003E+08 1.003E+08 24000 0 4179 MPI_Alltoall()
7.2
4.221E+07 4.728E+07 1001 2002 47228 GET_FIELD_EXPLICIT
4.5
2.855E+07 2.982E+07 1 770
29821568 MAKE_MAXWELL_MATRIX
3.8
2.53E+07 2.53E+07 10979 0 2304 MPI_Allreduce()
2.5
1.621E+07 1.621E+07 1 0
16206357 MAKE_LAMBDA_GRID
2.2
1.385E+07 1.427E+07 1 130
14273814 MAKE_POISSON_MATRIX
0.6
4.193E+06 4.193E+06 1122 0 3737 MPI_Bcast()
0.2
1.41E+05 1.547E+06 1000 1000 1547 GET_ERROR
0.2
7.565E+04 1.529E+06 1000 1000 1529 CATCH_HALT_SIGNAL
0.1
7.025E+05 7.025E+05 1 0
702483 MAKE_THETA_OPERATORS
0.1
1.591E+05 5.933E+05 1 257
593308 MAKE_BLEND_ARRAYS
0.1
2.944E+05 5.667E+05 10 160 56666 WRITE_BIG
0.1
3.909E+05 3.909E+05 1 0
390862 MAKE_RADIAL_OPERATORS
0.1
3.441E+05 3.759E+05 1 6
375861 MAKE_IMPLICIT_ADVECT
0.0
6.816E+04 6.842E+04 1 8 68416 MPI_Init()
0.0
1.349E+04 1.349E+04 2 0 6745 MPI_Comm_split()
0.0
406 7372 1 2 7372 WRITE_INPUT
0.0
5402 5519 1 4 5519 MPI_Finalize()
0.0
2627 2627 12 0 219 MPI_Reduce()
0.0
1746 1746 1 0 1746 MAKE_PROFILES
0.0
128 128 4
0 32 MPI_Attr_put()
0.0
123 123 4 0 31 MPI_Attr_get()
0.0
117 117 4 0 29 MPI_Keyval_free()
0.0
82 82 4 0
20 MPI_Comm_rank()
0.0
23 23 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 1,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 2;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.06E+05
6.502E+08 1 2808
650222258 .TAU application
89.6
7.058E+06 5.826E+08 1000 2008
582636 DO_FULLADVANCE
88.2
2.136E+08 5.734E+08 1000 28000
573409 TIMESTEP_SSP_322
32.9
1.993E+08 2.142E+08 3000 6000 71402 GET_KINETIC_ADVANCE
15.2
9.887E+07 9.887E+07 24000 0 4120 MPI_Alltoall()
7.2
4.18E+07 4.673E+07 1001 2002 46685 GET_FIELD_EXPLICIT
4.6
2.851E+07 2.98E+07 1 770
29798700 MAKE_MAXWELL_MATRIX
3.8
2.477E+07 2.477E+07 10979 0 2256 MPI_Allreduce()
2.5
1.617E+07 1.617E+07 1 0
16173149 MAKE_LAMBDA_GRID
2.2
1.383E+07 1.418E+07 1 130
14182907 MAKE_POISSON_MATRIX
0.5
3.27E+06 3.27E+06 1122 0 2914 MPI_Bcast()
0.3
1.43E+05 1.727E+06 1000 1000 1727 GET_ERROR
0.2
7.211E+04 1.536E+06 1000 1000 1536 CATCH_HALT_SIGNAL
0.1
6.943E+05 6.943E+05 1 0
694294 MAKE_THETA_OPERATORS
0.1
1.565E+05 5.986E+05 1 257
598572 MAKE_BLEND_ARRAYS
0.1
2.827E+05 5.561E+05 10 160 55610 WRITE_BIG
0.1 3.882E+05
3.882E+05 1 0
388200 MAKE_RADIAL_OPERATORS
0.1
3.396E+05 3.706E+05 1 6
370578 MAKE_IMPLICIT_ADVECT
0.0
6.802E+04 6.823E+04 1 8 68234 MPI_Init()
0.0 3.139E+04
3.139E+04 2 0 15695 MPI_Comm_split()
0.0
5203 5311 1 4 5311 MPI_Finalize()
0.0
380 4187 1 2 4187 WRITE_INPUT
0.0
3194 3194 12 0 266 MPI_Reduce()
0.0
1672 1672 1 0 1672 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
108 108 4 0 27 MPI_Keyval_free()
0.0
104 104 4 0 26 MPI_Attr_get()
0.0
63 63 4 0 16 MPI_Comm_rank()
0.0
27 27 2
0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 2,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 3;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.013E+05
6.661E+08 1 2808
666084533 .TAU application
89.7
7.064E+06 5.973E+08 1000 2008
597290 DO_FULLADVANCE
88.1
2.229E+08 5.87E+08 1000 28000
586993 TIMESTEP_SSP_322
32.5
2.007E+08 2.163E+08 3000 6000 72098 GET_KINETIC_ADVANCE
15.1
1.006E+08 1.006E+08 24000 0 4191 MPI_Alltoall()
7.1
4.222E+07 4.728E+07 1001 2002 47235 GET_FIELD_EXPLICIT
4.5
2.857E+07 2.984E+07 1 770
29841822 MAKE_MAXWELL_MATRIX
4.0
2.68E+07 2.68E+07 10979 0 2441 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16200438 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.427E+07 1 130
14272815 MAKE_POISSON_MATRIX
0.6
4.252E+06 4.252E+06 1122 0 3790 MPI_Bcast()
0.3
1.396E+05 1.689E+06 1000 1000 1689 GET_ERROR
0.2
2.905E+05 1.662E+06 10 160
166234 WRITE_BIG
0.2
6.887E+04 1.504E+06 1000 1000 1504 CATCH_HALT_SIGNAL
0.1
7.008E+05 7.008E+05 1 0
700830 MAKE_THETA_OPERATORS
0.1
1.6E+05 6.319E+05 1 257
631945 MAKE_BLEND_ARRAYS
0.1
3.914E+05 3.914E+05 1 0
391401 MAKE_RADIAL_OPERATORS
0.1
3.394E+05 3.709E+05 1 6
370889 MAKE_IMPLICIT_ADVECT
0.0
7.054E+04 7.075E+04 1 8 70746 MPI_Init()
0.0
4.182E+04 4.182E+04 2 0 20908 MPI_Comm_split()
0.0
4992 5095 1 4 5095 MPI_Finalize()
0.0
402 4391 1 2 4391 WRITE_INPUT
0.0
2669 2669 12 0 222 MPI_Reduce()
0.0
1707 1707 1 0 1707 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
103 103 4 0 26 MPI_Keyval_free()
0.0 100 100 4 0 25 MPI_Attr_get()
0.0
62 62 4 0 16 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 3,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 4;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.061E+05
6.589E+08 1 2808
658865988 .TAU application
89.7
7.07E+06 5.911E+08 1000 2008
591142 DO_FULLADVANCE
88.3
2.224E+08 5.82E+08 1000 28000
582019 TIMESTEP_SSP_322
32.5
1.997E+08 2.142E+08 3000 6000 71407 GET_KINETIC_ADVANCE
15.0
9.902E+07 9.902E+07 24000 0 4126 MPI_Alltoall()
7.0
4.164E+07 4.64E+07 1001 2002 46349 GET_FIELD_EXPLICIT
4.5
2.851E+07 2.98E+07 1 770
29796103 MAKE_MAXWELL_MATRIX
3.7
2.413E+07 2.413E+07 10979 0 2198 MPI_Allreduce()
2.5
1.617E+07 1.617E+07 1 0
16172614 MAKE_LAMBDA_GRID
2.2
1.382E+07 1.419E+07 1 130
14188506 MAKE_POISSON_MATRIX
0.5
3.26E+06 3.26E+06 1122 0 2905 MPI_Bcast()
0.2 1.397E+05
1.63E+06 1000 1000 1630 GET_ERROR
0.2
7.65E+04 1.452E+06 1000 1000 1452 CATCH_HALT_SIGNAL
0.1
6.929E+05 6.929E+05 1 0
692904 MAKE_THETA_OPERATORS
0.1
1.576E+05 5.997E+05 1 257
599683 MAKE_BLEND_ARRAYS
0.1
2.875E+05 5.364E+05 10 160 53640 WRITE_BIG
0.1
4.408E+05 4.716E+05 1 6
471577 MAKE_IMPLICIT_ADVECT
0.1
4.232E+05 4.232E+05 1 0
423227 MAKE_RADIAL_OPERATORS
0.0
7.065E+04 7.086E+04 1 8 70860 MPI_Init()
0.0
3.096E+04 3.096E+04 2 0 15478 MPI_Comm_split()
0.0
4872 4982
1 4 4982 MPI_Finalize()
0.0
4506 4506 12 0 376 MPI_Reduce()
0.0
395 4131 1 2 4131 WRITE_INPUT
0.0
1729 1729 1
0 1729 MAKE_PROFILES
0.0
110 110 4 0 28 MPI_Attr_put()
0.0
110 110 4 0 28 MPI_Keyval_free()
0.0
103 103 4
0 26 MPI_Attr_get()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 4,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 5;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.062E+05
6.637E+08 1 2808
663710572 .TAU application
89.6
7.067E+06 5.949E+08 1000 2008
594873 DO_FULLADVANCE
88.3
2.228E+08 5.858E+08 1000 28000
585765 TIMESTEP_SSP_322
32.4
2.01E+08 2.154E+08 3000 6000 71789 GET_KINETIC_ADVANCE
15.2
1.007E+08 1.007E+08 24000 0 4198 MPI_Alltoall()
7.1
4.206E+07 4.694E+07 1001 2002 46892 GET_FIELD_EXPLICIT
4.5
2.855E+07 2.983E+07 1 770
29830203 MAKE_MAXWELL_MATRIX
3.6
2.413E+07 2.413E+07 10979 0 2198 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16200503 MAKE_LAMBDA_GRID
2.1
1.384E+07 1.423E+07 1 130
14228102 MAKE_POISSON_MATRIX
0.6
4.218E+06 4.218E+06 1122 0 3759 MPI_Bcast()
0.2
1.38E+05 1.603E+06 1000 1000 1603 GET_ERROR
0.2
7.681E+04 1.495E+06 1000 1000 1495 CATCH_HALT_SIGNAL
0.1
7.014E+05 7.014E+05 1 0
701393 MAKE_THETA_OPERATORS
0.1
1.581E+05 6.262E+05 1 257
626185 MAKE_BLEND_ARRAYS
0.1
2.976E+05 5.572E+05 10 160 55722 WRITE_BIG
0.1
4.408E+05 4.719E+05 1 6
471923 MAKE_IMPLICIT_ADVECT
0.1
4.268E+05 4.268E+05 1 0
426792 MAKE_RADIAL_OPERATORS
0.0
6.787E+04 6.807E+04 1 8 68073 MPI_Init()
0.0
4.009E+04 4.009E+04 2 0 20045 MPI_Comm_split()
0.0
5158 5270 1 4 5270 MPI_Finalize()
0.0
376 4350 1 2 4350 WRITE_INPUT
0.0
2566 2566 12 0 214 MPI_Reduce()
0.0
1691 1691 1 0 1691 MAKE_PROFILES
0.0
112 112 4 0 28 MPI_Keyval_free()
0.0
106 106 4 0 26 MPI_Attr_put()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
63 63 4
0 16 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 5,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 6;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.068E+05
6.674E+08 1
2808 667410026 .TAU
application
89.7
7.073E+06 5.985E+08 1000 2008
598528 DO_FULLADVANCE
88.3
2.232E+08 5.895E+08 1000 28000
589487 TIMESTEP_SSP_322
32.4
2.01E+08 2.161E+08 3000
6000 72045 GET_KINETIC_ADVANCE
15.4
1.031E+08 1.031E+08 24000 0 4295 MPI_Alltoall()
7.1
4.209E+07 4.715E+07 1001 2002 47103 GET_FIELD_EXPLICIT
4.5
2.855E+07 2.986E+07 1
770 29859682 MAKE_MAXWELL_MATRIX
3.7
2.497E+07 2.497E+07 10979 0 2274 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16201668 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.421E+07 1 130
14209130 MAKE_POISSON_MATRIX
0.6
4.229E+06 4.229E+06 1122 0 3769 MPI_Bcast()
0.2
1.391E+05 1.523E+06 1000 1000 1523 GET_ERROR
0.2
7.703E+04 1.496E+06 1000 1000 1496 CATCH_HALT_SIGNAL
0.1
7.014E+05 7.014E+05 1 0
701410 MAKE_THETA_OPERATORS
0.1
1.584E+05 6.255E+05 1 257
625474 MAKE_BLEND_ARRAYS
0.1
2.975E+05 5.643E+05 10 160 56431 WRITE_BIG
0.1
4.475E+05 4.787E+05 1 6
478666 MAKE_IMPLICIT_ADVECT
0.1
4.271E+05 4.271E+05 1 0
427096 MAKE_RADIAL_OPERATORS
0.0
6.794E+04 6.814E+04 1 8 68143 MPI_Init()
0.0
4.037E+04 4.037E+04 2 0 20184 MPI_Comm_split()
0.0
5109 5212 1 4 5212 MPI_Finalize()
0.0
403 4488 1 2 4488 WRITE_INPUT
0.0
3371 3371 12 0 281 MPI_Reduce()
0.0
1682 1682 1 0 1682 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
103 103 4 0 26 MPI_Keyval_free()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
62 62 4 0 16 MPI_Comm_rank()
0.0 29 29 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 6,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 7;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.056E+05
6.617E+08 1 2808
661675282 .TAU application
89.7
7.066E+06 5.938E+08 1000 2008
593772 DO_FULLADVANCE
88.4
2.225E+08 5.846E+08 1000 28000
584646 TIMESTEP_SSP_322
32.5
1.996E+08 2.15E+08 3000 6000 71655 GET_KINETIC_ADVANCE
15.2
1.005E+08 1.005E+08 24000 0 4189 MPI_Alltoall()
7.1
4.166E+07 4.673E+07 1001 2002 46686 GET_FIELD_EXPLICIT
4.5
2.853E+07 2.991E+07 1 770
29906524 MAKE_MAXWELL_MATRIX
3.8
2.545E+07 2.545E+07 10979 0 2318 MPI_Allreduce()
2.4
1.617E+07 1.617E+07 1 0
16173416 MAKE_LAMBDA_GRID
2.2
1.383E+07 1.425E+07 1 130
14248644 MAKE_POISSON_MATRIX
0.5
3.258E+06 3.258E+06 1122 0 2903 MPI_Bcast()
0.2
1.407E+05 1.635E+06 1000 1000 1635 GET_ERROR
0.2
7.555E+04 1.462E+06 1000 1000 1462 CATCH_HALT_SIGNAL
0.1
6.938E+05 6.938E+05 1 0
693754 MAKE_THETA_OPERATORS
0.1
1.581E+05 6.024E+05 1 257
602375 MAKE_BLEND_ARRAYS
0.1
2.894E+05 5.368E+05 10 160 53684 WRITE_BIG
0.1
4.404E+05 4.71E+05 1 6
471017 MAKE_IMPLICIT_ADVECT
0.1
4.24E+05 4.24E+05 1 0
423953 MAKE_RADIAL_OPERATORS
0.0
7.075E+04 7.096E+04 1 8 70965 MPI_Init()
0.0
3.029E+04 3.029E+04 2 0 15144 MPI_Comm_split()
0.0
4847 4950 1 4 4950 MPI_Finalize()
0.0
383 4233 1 2 4233 WRITE_INPUT
0.0
2555 2555 12
0 213 MPI_Reduce()
0.0
1758 1758 1 0 1758 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
103 103 4 0
26 MPI_Attr_get()
0.0
103 103 4 0 26 MPI_Keyval_free()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 7,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 8;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.096E+05
6.514E+08 1 2808
651438422 .TAU application
89.6
7.064E+06 5.838E+08 1000 2008
583796 DO_FULLADVANCE
88.2
2.14E+08 5.747E+08 1000 28000
574687 TIMESTEP_SSP_322
32.8
1.995E+08 2.138E+08 3000 6000 71280 GET_KINETIC_ADVANCE
15.4
1.004E+08 1.004E+08 24000 0 4184 MPI_Alltoall()
7.1
4.181E+07 4.65E+07 1001 2002 46457 GET_FIELD_EXPLICIT
4.6
2.851E+07 2.985E+07 1 770
29851824 MAKE_MAXWELL_MATRIX
3.7
2.393E+07 2.393E+07 10979 0 2180 MPI_Allreduce()
2.5
1.617E+07 1.617E+07 1 0
16172549 MAKE_LAMBDA_GRID
2.2
1.382E+07 1.42E+07 1 130
14202728 MAKE_POISSON_MATRIX
0.5
3.228E+06 3.228E+06 1122 0 2877 MPI_Bcast()
0.2
1.421E+05 1.604E+06 1000 1000 1604 GET_ERROR
0.2
7.083E+04 1.455E+06 1000 1000 1455 CATCH_HALT_SIGNAL
0.1
6.929E+05 6.929E+05 1 0
692860 MAKE_THETA_OPERATORS
0.1
1.585E+05 6.077E+05 1 257
607738 MAKE_BLEND_ARRAYS
0.1
2.831E+05 5.553E+05 10 160 55530 WRITE_BIG
0.1
3.882E+05 3.882E+05 1 0
388243 MAKE_RADIAL_OPERATORS
0.1
3.472E+05 3.781E+05 1 6
378134 MAKE_IMPLICIT_ADVECT
0.0
6.844E+04 6.866E+04 1 8 68655 MPI_Init()
0.0
3.011E+04 3.011E+04 2 0 15054 MPI_Comm_split()
0.0 6545 6545 12 0 545 MPI_Reduce()
0.0
5109 5225 1 4 5225 MPI_Finalize()
0.0
396 4041 1 2 4041 WRITE_INPUT
0.0
1753 1753 1 0 1753 MAKE_PROFILES
0.0
116 116 4 0 29 MPI_Keyval_free()
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 8,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 9;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.098E+05
6.619E+08 1 2808
661891024 .TAU application
89.6
7.071E+06 5.931E+08 1000 2008
593069 DO_FULLADVANCE
88.2
2.188E+08 5.839E+08 1000 28000
583944 TIMESTEP_SSP_322
32.7
2.008E+08 2.166E+08 3000 6000 72185 GET_KINETIC_ADVANCE
15.3
1.014E+08 1.014E+08 24000 0 4226 MPI_Alltoall()
7.1
4.221E+07 4.724E+07 1001
2002 47195 GET_FIELD_EXPLICIT
4.5
2.856E+07 2.99E+07 1 770
29896056 MAKE_MAXWELL_MATRIX
3.9
2.571E+07 2.571E+07 10979 0 2342 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1
0 16199637 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.422E+07 1 130
14223379 MAKE_POISSON_MATRIX
0.6
4.234E+06 4.234E+06 1122 0 3774 MPI_Bcast()
0.2
1.437E+05 1.613E+06 1000 1000 1613 GET_ERROR
0.2
7.473E+04 1.511E+06 1000 1000 1511 CATCH_HALT_SIGNAL
0.1
7.021E+05 7.021E+05 1 0
702077 MAKE_THETA_OPERATORS
0.1
1.587E+05 6.366E+05 1 257
636627 MAKE_BLEND_ARRAYS
0.1
2.932E+05 5.588E+05 10 160 55878 WRITE_BIG
0.1
3.914E+05 3.914E+05 1 0
391418 MAKE_RADIAL_OPERATORS
0.1
3.47E+05 3.789E+05 1 6
378915 MAKE_IMPLICIT_ADVECT
0.0
7.02E+04 7.041E+04 1 8 70410 MPI_Init()
0.0
4.039E+04 4.039E+04 2 0 20196 MPI_Comm_split()
0.0
4912 5012 1 4 5012 MPI_Finalize()
0.0
385 4438 1 2 4438 WRITE_INPUT
0.0
2660 2660 12 0 222 MPI_Reduce()
0.0
1703 1703 1 0 1703 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Attr_get()
0.0
100 100 4 0 25 MPI_Keyval_free()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE 9,
CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 10;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.077E+05
6.631E+08 1 2808
663108041 .TAU application
89.5
7.066E+06 5.937E+08 1000 2008
593706 DO_FULLADVANCE
88.2
2.187E+08 5.846E+08 1000 28000
584573 TIMESTEP_SSP_322
32.8
2.004E+08 2.174E+08 3000 6000 72457 GET_KINETIC_ADVANCE
15.2
1.007E+08 1.007E+08 24000 0 4195 MPI_Alltoall()
7.2
4.219E+07 4.784E+07 1001 2002 47796 GET_FIELD_EXPLICIT
4.5
2.858E+07 2.992E+07 1 770
29921344 MAKE_MAXWELL_MATRIX
4.2
2.754E+07 2.754E+07 10979 0 2508 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16199928 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.421E+07 1 130
14208194 MAKE_POISSON_MATRIX
0.7
4.818E+06 4.818E+06 1122 0 4294 MPI_Bcast()
0.3
7.392E+04 2.098E+06 1000 1000 2098 CATCH_HALT_SIGNAL
0.2 1.435E+05
1.624E+06 1000 1000 1624 GET_ERROR
0.1
6.999E+05 6.999E+05 1 0
699926 MAKE_THETA_OPERATORS
0.1
1.576E+05 6.374E+05 1 257
637410 MAKE_BLEND_ARRAYS
0.1
2.904E+05 5.614E+05 10 160 56143 WRITE_BIG
0.1
3.913E+05 3.913E+05 1 0
391346 MAKE_RADIAL_OPERATORS
0.1
3.419E+05 3.737E+05 1 6
373682 MAKE_IMPLICIT_ADVECT
0.0
7.153E+04 7.174E+04 1 8 71740 MPI_Init()
0.0
4.005E+04 4.005E+04 2 0 20024 MPI_Comm_split()
0.0
4806 4930 1 4 4930 MPI_Finalize()
0.0
392 4488 1 2 4488 WRITE_INPUT
0.0
3698 3698 12 0 308 MPI_Reduce()
0.0
1701 1701 1 0 1701 MAKE_PROFILES
0.0
124 124 4
0 31 MPI_Keyval_free()
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
65 65 4 0
16 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
10, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 11;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.034E+05
6.605E+08 1 2808
660484560 .TAU application
89.8
7.07E+06 5.928E+08 1000 2008
592841 DO_FULLADVANCE
88.4
2.225E+08 5.837E+08 1000 28000
583674 TIMESTEP_SSP_322
32.4
1.993E+08 2.142E+08 3000 6000 71410 GET_KINETIC_ADVANCE
15.2
1.002E+08 1.002E+08 24000 0 4174 MPI_Alltoall()
7.1
4.182E+07 4.677E+07 1001 2002 46722 GET_FIELD_EXPLICIT
4.5
2.853E+07 2.978E+07 1 770
29783053 MAKE_MAXWELL_MATRIX
3.7
2.477E+07 2.477E+07 10979 0 2256 MPI_Allreduce()
2.4
1.617E+07 1.617E+07 1 0
16173864 MAKE_LAMBDA_GRID
2.2
1.382E+07 1.423E+07 1 130
14227240 MAKE_POISSON_MATRIX
0.5
3.295E+06 3.295E+06 1122 0 2936 MPI_Bcast()
0.3
1.423E+05 1.675E+06 1000 1000 1675 GET_ERROR
0.2
6.839E+04 1.463E+06 1000 1000 1463 CATCH_HALT_SIGNAL
0.1
6.925E+05 6.925E+05 1 0
692499 MAKE_THETA_OPERATORS
0.1
1.567E+05 6.066E+05 1 257
606591 MAKE_BLEND_ARRAYS
0.1
2.875E+05 5.356E+05 10 160 53562 WRITE_BIG
0.1
3.887E+05 3.887E+05 1 0
388707 MAKE_RADIAL_OPERATORS
0.1
3.409E+05 3.716E+05 1 6
371568 MAKE_IMPLICIT_ADVECT
0.0
6.854E+04 6.876E+04 1 8 68755 MPI_Init()
0.0
3.07E+04 3.07E+04 2 0 15348 MPI_Comm_split()
0.0
5101 5197 1 4 5197 MPI_Finalize()
0.0
398 4066 1 2 4066 WRITE_INPUT
0.0
2681 2681 12 0 223 MPI_Reduce()
0.0
1720 1720 1 0 1720 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
103 103
4 0 26 MPI_Attr_get()
0.0
96 96 4 0 24 MPI_Keyval_free()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
11, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 12;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.002E+05
6.636E+08 1 2808
663590455 .TAU application
89.6
7.068E+06 5.945E+08 1000 2008
594467 DO_FULLADVANCE
88.2
2.23E+08 5.853E+08 1000 28000
585311 TIMESTEP_SSP_322
32.4
1.994E+08 2.151E+08 3000 6000 71716 GET_KINETIC_ADVANCE
15.1
1.004E+08 1.004E+08 24000 0 4185 MPI_Alltoall()
7.1
4.169E+07 4.68E+07 1001 2002 46756 GET_FIELD_EXPLICIT
4.5
2.85E+07 2.987E+07 1 770
29871704 MAKE_MAXWELL_MATRIX
4.1
2.71E+07 2.71E+07 10979 0 2468 MPI_Allreduce()
2.4
1.617E+07 1.617E+07 1 0
16171651 MAKE_LAMBDA_GRID
2.1
1.382E+07 1.42E+07 1 130
14200228 MAKE_POISSON_MATRIX
0.5
3.291E+06 3.291E+06 1122 0 2933 MPI_Bcast()
0.2
1.414E+05 1.657E+06 1000 1000 1657 GET_ERROR
0.2
7.361E+04 1.458E+06 1000 1000 1458 CATCH_HALT_SIGNAL
0.1
6.944E+05 6.944E+05 1 0
694412 MAKE_THETA_OPERATORS
0.1
1.564E+05 6.084E+05 1 257
608435 MAKE_BLEND_ARRAYS
0.1
2.868E+05 5.459E+05 10 160 54587 WRITE_BIG
0.1
4.36E+05 4.667E+05 1 6
466654 MAKE_IMPLICIT_ADVECT
0.1
4.236E+05 4.236E+05 1 0
423629 MAKE_RADIAL_OPERATORS
0.0
7.09E+04 7.111E+04 1 8 71112 MPI_Init()
0.0
3.04E+04 3.04E+04 2 0 15200 MPI_Comm_split()
0.0
7760 7760 12 0 647 MPI_Reduce()
0.0
4825 4944 1 4 4944 MPI_Finalize()
0.0
362 4065 1 2 4065 WRITE_INPUT
0.0
1740 1740 1
0 1740 MAKE_PROFILES
0.0
119 119 4 0 30 MPI_Keyval_free()
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
102 102 4 0 26 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
12, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 13;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.068E+05
6.648E+08 1 2808
664804205 .TAU application
89.6
7.07E+06 5.959E+08 1000 2008
595864 DO_FULLADVANCE
88.3
2.232E+08 5.867E+08 1000 28000
586732 TIMESTEP_SSP_322
32.5
2.006E+08 2.158E+08 3000 6000 71938 GET_KINETIC_ADVANCE
15.1
1.006E+08 1.006E+08 24000 0 4192 MPI_Alltoall()
7.1
4.204E+07 4.714E+07 1001 2002 47094 GET_FIELD_EXPLICIT
4.5
2.855E+07 2.988E+07 1 770
29877760 MAKE_MAXWELL_MATRIX
3.8
2.528E+07 2.528E+07 10979 0 2303 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16201956 MAKE_LAMBDA_GRID
2.1
1.384E+07 1.423E+07 1 130
14226087 MAKE_POISSON_MATRIX
0.6
4.208E+06 4.208E+06 1122 0 3750 MPI_Bcast()
0.2 1.4E+05
1.623E+06 1000 1000 1623 GET_ERROR
0.2
7.412E+04 1.49E+06 1000 1000 1490 CATCH_HALT_SIGNAL
0.1
7.018E+05 7.018E+05 1 0
701750 MAKE_THETA_OPERATORS
0.1
1.582E+05 6.292E+05 1 257
629237 MAKE_BLEND_ARRAYS
0.1
2.954E+05 5.575E+05 10 160 55746 WRITE_BIG
0.1
4.488E+05 4.801E+05 1 6
480122 MAKE_IMPLICIT_ADVECT
0.1
4.275E+05 4.275E+05 1 0
427490 MAKE_RADIAL_OPERATORS
0.0
6.838E+04 6.859E+04 1 8 68589 MPI_Init()
0.0
3.983E+04 3.983E+04 2 0 19916 MPI_Comm_split()
0.0
5193 5293
1 4 5293 MPI_Finalize()
0.0
404 4492 1 2 4492 WRITE_INPUT
0.0
2605 2605 12 0 217 MPI_Reduce()
0.0
1720 1720 1
0 1720 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
100 100 4 0 25 MPI_Keyval_free()
0.0
98 98 4
0 24 MPI_Attr_get()
0.0
66 66 4 0 16 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
13, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 14;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.065E+05
6.62E+08 1 2808
661956940 .TAU application
89.8
7.068E+06 5.941E+08 1000 2008
594114 DO_FULLADVANCE
88.2
2.231E+08 5.839E+08 1000 28000
583907 TIMESTEP_SSP_322
32.5
1.996E+08 2.149E+08 3000 6000 71621 GET_KINETIC_ADVANCE
15.0
9.935E+07 9.935E+07 24000 0 4140 MPI_Alltoall()
7.1
4.165E+07 4.667E+07 1001 2002 46624 GET_FIELD_EXPLICIT
4.5
2.853E+07 2.983E+07 1 770
29831277 MAKE_MAXWELL_MATRIX
4.0
2.622E+07 2.622E+07 10979 0 2388 MPI_Allreduce()
2.4
1.617E+07 1.617E+07 1 0
16173722 MAKE_LAMBDA_GRID
2.1
1.383E+07 1.419E+07 1 130
14188874 MAKE_POISSON_MATRIX
0.5
3.315E+06 3.315E+06 1122 0 2954 MPI_Bcast()
0.2
2.887E+05 1.638E+06 10 160
163792 WRITE_BIG
0.2
1.407E+05 1.615E+06 1000 1000 1615 GET_ERROR
0.2
7.499E+04 1.47E+06 1000 1000 1470 CATCH_HALT_SIGNAL
0.1
6.923E+05 6.923E+05 1 0
692265 MAKE_THETA_OPERATORS
0.1
1.592E+05 6.066E+05 1
257 606611
MAKE_BLEND_ARRAYS
0.1
4.419E+05 4.723E+05 1 6
472338 MAKE_IMPLICIT_ADVECT
0.1
4.242E+05 4.242E+05 1 0
424226 MAKE_RADIAL_OPERATORS
0.0
6.893E+04 6.914E+04 1 8 69141 MPI_Init()
0.0
3.047E+04 3.047E+04 2 0 15236 MPI_Comm_split()
0.0
5428 5530 1 4 5530 MPI_Finalize()
0.0
377 4170 1 2 4170 WRITE_INPUT
0.0
3388 3388 12 0 282 MPI_Reduce()
0.0
1709 1709 1 0 1709 MAKE_PROFILES
0.0
112 112 4 0 28 MPI_Attr_put()
0.0
102 102 4 0 26 MPI_Keyval_free()
0.0
100 100 4 0 25 MPI_Attr_get()
0.0
62 62 4
0 16 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
14, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 15;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.079E+05
6.66E+08 1 2808
665989266 .TAU application
89.7
7.066E+06 5.972E+08 1000 2008
597154 DO_FULLADVANCE
88.1
2.229E+08 5.866E+08 1000 28000
586572 TIMESTEP_SSP_322
32.5
2.008E+08 2.163E+08 3000 6000 72097 GET_KINETIC_ADVANCE
15.1
1.003E+08 1.003E+08 24000 0 4180 MPI_Alltoall()
7.1
4.207E+07 4.712E+07 1001 2002 47075 GET_FIELD_EXPLICIT
4.5
2.855E+07 2.985E+07 1 770
29849707 MAKE_MAXWELL_MATRIX
4.0
2.683E+07 2.683E+07 10979 0 2444 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16202264 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.421E+07 1 130
14207535 MAKE_POISSON_MATRIX
0.6 4.214E+06
4.214E+06 1122 0 3756 MPI_Bcast()
0.3
2.965E+05 1.967E+06 10 160
196737 WRITE_BIG
0.3
1.4E+05 1.667E+06 1000 1000 1667 GET_ERROR
0.2
7.324E+04 1.506E+06 1000 1000 1506 CATCH_HALT_SIGNAL
0.1
7.004E+05 7.004E+05 1 0
700445 MAKE_THETA_OPERATORS
0.1
1.592E+05 6.268E+05 1 257
626836 MAKE_BLEND_ARRAYS
0.1
4.394E+05 4.707E+05 1 6
470680 MAKE_IMPLICIT_ADVECT
0.1
4.278E+05 4.278E+05 1 0
427814 MAKE_RADIAL_OPERATORS
0.0
7.049E+04 7.07E+04 1 8 70700 MPI_Init()
0.0
3.973E+04 3.973E+04 2 0 19863 MPI_Comm_split()
0.0
4776 4883 1 4 4883 MPI_Finalize()
0.0
385 4361 1 2 4361 WRITE_INPUT
0.0
2471 2471 12 0 206 MPI_Reduce()
0.0
1732 1732 1 0 1732 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
107 107 4 0 27 MPI_Keyval_free()
0.0
101 101 4 0 25 MPI_Attr_get()
0.0
64 64 4 0 16 MPI_Comm_rank()
0.0
22 22 2 0 11 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
15, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096
0 Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 16;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.14E+05
6.55E+08 1 2808
654977686 .TAU application
89.5
7.075E+06 5.86E+08 1000 2008
585983 DO_FULLADVANCE
88.1
2.185E+08 5.768E+08 1000 28000
576828 TIMESTEP_SSP_322
32.7
1.995E+08 2.139E+08 3000 6000 71302 GET_KINETIC_ADVANCE
14.9
9.776E+07 9.776E+07 24000 0 4073 MPI_Alltoall()
7.1
4.187E+07 4.674E+07 1001 2002 46695 GET_FIELD_EXPLICIT
4.6
2.855E+07 2.982E+07 1 770
29824996 MAKE_MAXWELL_MATRIX
3.9
2.546E+07 2.546E+07 10979 0 2319 MPI_Allreduce()
2.5
1.617E+07 1.617E+07 1 0
16172384 MAKE_LAMBDA_GRID
2.2
1.384E+07 1.424E+07 1 130
14237753 MAKE_POISSON_MATRIX
0.5
3.291E+06 3.291E+06 1122 0 2933 MPI_Bcast()
0.3
1.426E+05 1.654E+06 1000 1000 1654 GET_ERROR
0.2
7.478E+04 1.474E+06 1000 1000 1474 CATCH_HALT_SIGNAL
0.1
6.926E+05 6.926E+05 1 0
692607 MAKE_THETA_OPERATORS
0.1
1.587E+05 6.075E+05 1 257
607479 MAKE_BLEND_ARRAYS
0.1
2.857E+05 5.386E+05 10 160 53862 WRITE_BIG
0.1
3.879E+05 3.879E+05 1 0
387944 MAKE_RADIAL_OPERATORS
0.1
3.428E+05 3.736E+05 1 6
373572 MAKE_IMPLICIT_ADVECT
0.0
6.818E+04 6.84E+04 1 8 68398 MPI_Init()
0.0
3.076E+04 3.076E+04 2 0 15382 MPI_Comm_split()
0.0
5698 5698 12 0 475 MPI_Reduce()
0.0
5132 5246 1 4 5246 MPI_Finalize()
0.0
397 4278 1 2 4278 WRITE_INPUT
0.0
1644 1644 1 0 1644 MAKE_PROFILES
0.0
114 114 4 0 28 MPI_Keyval_free()
0.0
113 113 4
0 28 MPI_Attr_put()
0.0
103 103 4 0 26 MPI_Attr_get()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
32 32 2 0 16 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
16, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 17;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.028E+05
6.666E+08 1 2808
666617149 .TAU application
89.4
7.07E+06 5.959E+08 1000 2008
595887 DO_FULLADVANCE
88.0
2.23E+08 5.867E+08 1000 28000
586712 TIMESTEP_SSP_322
32.3
2.005E+08 2.153E+08 3000 6000 71766 GET_KINETIC_ADVANCE
15.2
1.013E+08 1.013E+08 24000 0 4220 MPI_Alltoall()
7.1
4.221E+07 4.719E+07 1001 2002 47141 GET_FIELD_EXPLICIT
4.5
2.857E+07 2.984E+07 1 770
29842761 MAKE_MAXWELL_MATRIX
3.7
2.478E+07 2.478E+07 10979 0 2257 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16199612 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.422E+07 1 130
14224838 MAKE_POISSON_MATRIX
0.9
6.109E+06 6.109E+06 1122 0 5445 MPI_Bcast()
0.5 6.969E+04 3.395E+06 1000 1000 3395 CATCH_HALT_SIGNAL
0.2
1.426E+05 1.629E+06 1000 1000 1629 GET_ERROR
0.1
7.004E+05 7.004E+05 1 0
700351 MAKE_THETA_OPERATORS
0.1
1.58E+05 6.332E+05 1 257
633221 MAKE_BLEND_ARRAYS
0.1
2.947E+05 5.956E+05 10 160 59555 WRITE_BIG
0.1
3.915E+05 3.915E+05 1 0
391547 MAKE_RADIAL_OPERATORS
0.1
3.394E+05 3.713E+05 1 6
371272 MAKE_IMPLICIT_ADVECT
0.0
7.016E+04 7.037E+04 1 8 70369 MPI_Init()
0.0
4.053E+04 4.053E+04 2 0 20266 MPI_Comm_split()
0.0
4743 4854 1 4 4854 MPI_Finalize()
0.0
416 4311 1 2 4311 WRITE_INPUT
0.0
2947 2947 12 0 246 MPI_Reduce()
0.0
1729 1729 1 0 1729 MAKE_PROFILES
0.0
111 111 4 0 28 MPI_Keyval_free()
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
99 99
4 0 25 MPI_Attr_get()
0.0
65 65 4 0 16 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
17, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 18;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.101E+05
6.581E+08 1 2808
658087759 .TAU application
89.7
7.074E+06 5.903E+08 1000 2008
590268 DO_FULLADVANCE
88.3
2.187E+08 5.811E+08 1000 28000
581064 TIMESTEP_SSP_322
32.7 1.995E+08
2.155E+08 3000 6000 71835 GET_KINETIC_ADVANCE
15.2
9.974E+07 9.974E+07 24000 0 4156 MPI_Alltoall()
7.2
4.179E+07 4.716E+07 1001 2002 47112 GET_FIELD_EXPLICIT
4.5 2.854E+07
2.987E+07 1 770
29873188 MAKE_MAXWELL_MATRIX
4.0
2.639E+07 2.639E+07 10979 0 2404 MPI_Allreduce()
2.5
1.617E+07 1.617E+07 1 0
16171852 MAKE_LAMBDA_GRID
2.2 1.383E+07 1.425E+07 1 130
14254846 MAKE_POISSON_MATRIX
0.5
3.289E+06 3.289E+06 1122 0 2932 MPI_Bcast()
0.3
1.409E+05 1.702E+06 1000 1000 1702 GET_ERROR
0.2
7.522E+04 1.524E+06 1000 1000 1524 CATCH_HALT_SIGNAL
0.1
6.924E+05 6.924E+05 1 0
692395 MAKE_THETA_OPERATORS
0.1
1.587E+05 6.103E+05 1 257
610281 MAKE_BLEND_ARRAYS
0.1
2.895E+05 5.392E+05 10 160 53916 WRITE_BIG
0.1
3.887E+05 3.887E+05 1 0
388710 MAKE_RADIAL_OPERATORS
0.1
3.438E+05 3.748E+05 1 6
374750 MAKE_IMPLICIT_ADVECT
0.0
6.775E+04 6.796E+04 1 8 67965 MPI_Init()
0.0
3.014E+04 3.014E+04 2 0 15070 MPI_Comm_split()
0.0
5087 5189 1 4 5189 MPI_Finalize()
0.0
381 4069 1 2 4069 WRITE_INPUT
0.0
3540 3540 12 0 295 MPI_Reduce()
0.0
1742 1742 1 0 1742 MAKE_PROFILES
0.0
111 111 4 0
28 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Attr_get()
0.0
102 102 4 0 26 MPI_Keyval_free()
0.0
67 67 4 0
17 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
18, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 19;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.056E+05
6.614E+08 1 2808
661397145 .TAU application
89.6
7.072E+06 5.924E+08 1000 2008
592414 DO_FULLADVANCE
88.2
2.208E+08 5.832E+08 1000 28000
583239 TIMESTEP_SSP_322
32.7
2.005E+08 2.16E+08 3000 6000 71993 GET_KINETIC_ADVANCE
15.0
9.897E+07 9.897E+07 24000 0 4124 MPI_Alltoall()
7.2
4.225E+07 4.755E+07 1001 2002 47498 GET_FIELD_EXPLICIT
4.5
2.857E+07 2.993E+07 1 770
29934953 MAKE_MAXWELL_MATRIX
3.9
2.589E+07 2.589E+07 10979 0 2358 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16200181 MAKE_LAMBDA_GRID
2.2
1.385E+07 1.425E+07 1 130
14245960 MAKE_POISSON_MATRIX
0.6
4.259E+06 4.259E+06 1122 0 3796 MPI_Bcast()
0.3
1.415E+05 1.67E+06 1000 1000 1670 GET_ERROR
0.2
7.032E+04 1.553E+06 1000 1000 1553 CATCH_HALT_SIGNAL
0.1
7.014E+05 7.014E+05 1 0
701449 MAKE_THETA_OPERATORS
0.1
1.589E+05 6.254E+05 1 257
625394 MAKE_BLEND_ARRAYS
0.1
2.868E+05 5.501E+05 10 160 55006 WRITE_BIG
0.1
3.929E+05 3.929E+05 1 0
392916 MAKE_RADIAL_OPERATORS
0.1
3.502E+05 3.816E+05 1 6
381633 MAKE_IMPLICIT_ADVECT
0.0
7.055E+04 7.076E+04 1 8 70755 MPI_Init()
0.0
4.01E+04 4.01E+04 2 0 20048 MPI_Comm_split()
0.0
4837 4952 1 4 4952 MPI_Finalize()
0.0
410 4402 1 2 4402 WRITE_INPUT
0.0
2659 2659 12 0 222 MPI_Reduce()
0.0
1708 1708 1 0 1708 MAKE_PROFILES
0.0
115 115 4 0 29 MPI_Keyval_free()
0.0
110 110 4
0 28 MPI_Attr_put()
0.0
97 97 4 0 24 MPI_Attr_get()
0.0
64 64 4 0 16 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
19, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 20;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.042E+05
6.66E+08 1 2808
665990061 .TAU application
89.6
7.072E+06 5.97E+08 1000 2008
596962 DO_FULLADVANCE
88.3
2.236E+08 5.878E+08 1000 28000
587768 TIMESTEP_SSP_322
32.4
2.006E+08 2.156E+08 3000 6000 71869 GET_KINETIC_ADVANCE
15.2
1.015E+08 1.015E+08 24000 0 4229 MPI_Alltoall()
7.1
4.208E+07 4.71E+07 1001 2002 47049 GET_FIELD_EXPLICIT
4.5
2.852E+07 2.981E+07 1 770
29812346 MAKE_MAXWELL_MATRIX
3.8
2.508E+07 2.508E+07 10979 0 2284 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16200798 MAKE_LAMBDA_GRID
2.1
1.384E+07 1.425E+07 1 130
14253571 MAKE_POISSON_MATRIX
0.6
4.311E+06 4.311E+06 1122 0 3843 MPI_Bcast()
0.3 1.399E+05 1.668E+06 1000 1000 1668 GET_ERROR
0.2
7.336E+04 1.583E+06 1000 1000 1583 CATCH_HALT_SIGNAL
0.1
7.011E+05 7.011E+05 1 0
701085 MAKE_THETA_OPERATORS
0.1
1.593E+05 6.326E+05 1 257
632630 MAKE_BLEND_ARRAYS
0.1
2.951E+05 5.769E+05 10 160 57687 WRITE_BIG
0.1
4.34E+05 4.651E+05 1 6
465082 MAKE_IMPLICIT_ADVECT
0.1
4.28E+05 4.28E+05 1 0
428015 MAKE_RADIAL_OPERATORS
0.0
6.824E+04 6.845E+04 1 8 68447 MPI_Init()
0.0
4.002E+04 4.002E+04 2 0 20008 MPI_Comm_split()
0.0
2.126E+04 2.126E+04 12 0 1772 MPI_Reduce()
0.0
5057 5158 1 4 5158 MPI_Finalize()
0.0
389 4602 1 2 4602 WRITE_INPUT
0.0
1753 1753 1 0 1753 MAKE_PROFILES
0.0
106 106 4 0 26 MPI_Attr_put()
0.0
102 102 4 0 26 MPI_Attr_get()
0.0
101 101 4 0 25 MPI_Keyval_free()
0.0
63 63 4 0 16 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
20, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 21;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.113E+05
6.607E+08 1 2808
660709838 .TAU application
89.7
7.073E+06 5.929E+08 1000 2008
592870 DO_FULLADVANCE
88.3
2.23E+08 5.837E+08 1000 28000
583718 TIMESTEP_SSP_322
32.4 1.994E+08
2.138E+08 3000 6000 71281 GET_KINETIC_ADVANCE
15.2
1.004E+08 1.004E+08 24000 0 4183 MPI_Alltoall()
7.0
4.164E+07 4.657E+07 1001 2002 46519 GET_FIELD_EXPLICIT
4.5 2.852E+07
2.981E+07 1 770
29807542 MAKE_MAXWELL_MATRIX
3.7
2.429E+07 2.429E+07 10979 0 2212 MPI_Allreduce()
2.4
1.617E+07 1.617E+07 1 0
16172632 MAKE_LAMBDA_GRID
2.2 1.382E+07 1.422E+07 1 130
14220638 MAKE_POISSON_MATRIX
0.5
3.312E+06 3.312E+06 1122 0 2952 MPI_Bcast()
0.3
1.411E+05 1.658E+06 1000 1000 1658 GET_ERROR
0.2
7.651E+04 1.509E+06 1000 1000 1509 CATCH_HALT_SIGNAL
0.1
6.94E+05 6.94E+05 1 0
694039 MAKE_THETA_OPERATORS
0.1
1.572E+05 6.044E+05 1 257
604398 MAKE_BLEND_ARRAYS
0.1
2.867E+05 5.32E+05 10 160 53204 WRITE_BIG
0.1
4.434E+05 4.737E+05 1 6
473696 MAKE_IMPLICIT_ADVECT
0.1
4.241E+05 4.241E+05 1 0
424101 MAKE_RADIAL_OPERATORS
0.0
7.06E+04 7.081E+04 1 8 70806 MPI_Init()
0.0
2.982E+04 2.982E+04 2 0 14909 MPI_Comm_split()
0.0
4779 4887 1 4 4887 MPI_Finalize()
0.0
382 4096 1 2 4096 WRITE_INPUT
0.0
2462 2462 12 0 205 MPI_Reduce()
0.0
1729 1729 1 0 1729 MAKE_PROFILES
0.0
109 109 4 0
27 MPI_Attr_put()
0.0
108 108 4 0 27 MPI_Keyval_free()
0.0
100 100 4 0 25 MPI_Attr_get()
0.0
67 67 4 0
17 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
21, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 22;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.064E+05
6.667E+08 1 2808
666697171 .TAU application
89.7
7.07E+06 5.977E+08 1000 2008
597695 DO_FULLADVANCE
88.3
2.231E+08 5.886E+08 1000 28000
588601 TIMESTEP_SSP_322
32.4
2.008E+08 2.163E+08 3000 6000 72090 GET_KINETIC_ADVANCE
15.3
1.019E+08 1.019E+08 24000 0 4245 MPI_Alltoall()
7.1
4.208E+07 4.738E+07 1001 2002 47334 GET_FIELD_EXPLICIT
4.5
2.854E+07 2.986E+07 1 770
29855394 MAKE_MAXWELL_MATRIX
3.9
2.578E+07 2.578E+07 10979 0 2348 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16200886 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.426E+07 1 130
14260930 MAKE_POISSON_MATRIX
0.6
4.244E+06 4.244E+06 1122 0 3783 MPI_Bcast()
0.2
1.412E+05 1.581E+06 1000 1000 1581 GET_ERROR
0.2
7.285E+04 1.545E+06 1000 1000 1545 CATCH_HALT_SIGNAL
0.1
6.991E+05 6.991E+05 1 0
699061 MAKE_THETA_OPERATORS
0.1
1.583E+05 6.307E+05 1 257
630688 MAKE_BLEND_ARRAYS
0.1
2.935E+05 5.626E+05 10 160 56264 WRITE_BIG
0.1
4.381E+05 4.69E+05 1 6
469047 MAKE_IMPLICIT_ADVECT
0.1
4.28E+05 4.28E+05 1 0
427962 MAKE_RADIAL_OPERATORS
0.0
6.807E+04 6.828E+04 1 8 68280 MPI_Init()
0.0
3.966E+04 3.966E+04 2 0 19832 MPI_Comm_split()
0.0
5057 5175 1 4 5175 MPI_Finalize()
0.0
4558 4558 12 0 380 MPI_Reduce()
0.0
370 4358 1 2 4358 WRITE_INPUT
0.0
1663 1663 1 0 1663 MAKE_PROFILES
0.0
118 118 4 0 30 MPI_Keyval_free()
0.0
109 109 4
0 27 MPI_Attr_put()
0.0
100 100 4 0 25 MPI_Attr_get()
0.0
62 62 4 0 16 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
22, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 23;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.089E+05
6.6E+08 1 2808
659993587 .TAU application
89.7
7.074E+06 5.921E+08 1000 2008
592115 DO_FULLADVANCE
88.3
2.23E+08 5.83E+08 1000 28000
583002 TIMESTEP_SSP_322
32.4
1.996E+08 2.139E+08 3000 6000 71305 GET_KINETIC_ADVANCE
15.1
9.968E+07 9.968E+07 24000 0 4153 MPI_Alltoall()
7.0
4.168E+07 4.649E+07 1001 2002 46442 GET_FIELD_EXPLICIT
4.5
2.854E+07 2.981E+07 1 770
29810363 MAKE_MAXWELL_MATRIX
3.6
2.389E+07 2.389E+07 10979 0 2176 MPI_Allreduce()
2.5
1.617E+07 1.617E+07 1 0
16172916 MAKE_LAMBDA_GRID
2.2
1.383E+07 1.421E+07 1 130
14209151 MAKE_POISSON_MATRIX
0.5
3.38E+06 3.38E+06 1122 0 3012 MPI_Bcast()
0.2 1.427E+05 1.62E+06 1000 1000 1620 GET_ERROR
0.2
7.558E+04 1.542E+06 1000 1000 1542 CATCH_HALT_SIGNAL
0.1
6.936E+05 6.936E+05 1 0
693555 MAKE_THETA_OPERATORS
0.1
1.572E+05 6.022E+05 1 257
602222 MAKE_BLEND_ARRAYS
0.1
2.876E+05 5.318E+05 10 160 53179 WRITE_BIG
0.1
4.394E+05 4.695E+05 1 6
469536 MAKE_IMPLICIT_ADVECT
0.1
4.246E+05 4.246E+05 1 0
424622 MAKE_RADIAL_OPERATORS
0.0
7.03E+04 7.051E+04 1 8 70506 MPI_Init()
0.0
3.077E+04 3.077E+04 2 0 15387 MPI_Comm_split()
0.0
4871 4966 1 4 4966 MPI_Finalize()
0.0
404 4012 1 2 4012 WRITE_INPUT
0.0
2651 2651 12 0 221 MPI_Reduce()
0.0
1763 1763 1 0 1763 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
95 95
4 0 24 MPI_Keyval_free()
0.0
62 62 4 0 16 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
23, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 24;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.08E+05
6.571E+08 1 2808
657109612 .TAU application
89.7
7.059E+06 5.894E+08 1000 2008
589354 DO_FULLADVANCE
88.3
2.184E+08 5.802E+08 1000 28000
580190 TIMESTEP_SSP_322
32.6
1.995E+08 2.14E+08 3000 6000 71335 GET_KINETIC_ADVANCE
15.4
1.011E+08 1.011E+08 24000 0 4214 MPI_Alltoall()
7.1
4.181E+07 4.669E+07 1001 2002 46639 GET_FIELD_EXPLICIT
4.5
2.851E+07 2.99E+07 1 770
29895124 MAKE_MAXWELL_MATRIX
3.7
2.439E+07 2.439E+07 10979 0 2222 MPI_Allreduce()
2.5
1.617E+07 1.617E+07 1 0
16172485 MAKE_LAMBDA_GRID
2.2
1.383E+07 1.419E+07 1 130
14191598 MAKE_POISSON_MATRIX
0.5
3.301E+06 3.301E+06 1122 0 2942 MPI_Bcast()
0.3
1.428E+05 1.681E+06 1000 1000 1681 GET_ERROR
0.2
7.138E+04 1.495E+06 1000 1000 1495 CATCH_HALT_SIGNAL
0.1
6.927E+05 6.927E+05 1 0
692718 MAKE_THETA_OPERATORS
0.1
1.582E+05 6.097E+05 1 257
609691 MAKE_BLEND_ARRAYS
0.1
2.832E+05 5.358E+05 10 160 53580 WRITE_BIG
0.1
3.883E+05 3.883E+05 1 0
388282 MAKE_RADIAL_OPERATORS
0.1
3.449E+05 3.762E+05 1 6
376205 MAKE_IMPLICIT_ADVECT
0.0
6.86E+04 6.881E+04 1 8 68810 MPI_Init()
0.0
3.079E+04 3.079E+04 2 0 15396 MPI_Comm_split()
0.0
6310 6310 12 0 526 MPI_Reduce()
0.0
5242 5334 1 4 5334 MPI_Finalize()
0.0
379 4136 1 2 4136 WRITE_INPUT
0.0
1745 1745 1 0 1745 MAKE_PROFILES
0.0
109 109 4
0 27 MPI_Attr_put()
0.0
102 102 4 0 26 MPI_Attr_get()
0.0
92 92 4 0 23 MPI_Keyval_free()
0.0
58 58 4 0
14 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
24, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 25;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.19E+05
6.596E+08 1 2808
659553198 .TAU application
89.6
7.072E+06 5.907E+08 1000 2008
590653 DO_FULLADVANCE
88.2
2.179E+08 5.815E+08 1000 28000
581466 TIMESTEP_SSP_322
32.7
2.005E+08 2.154E+08 3000 6000 71799 GET_KINETIC_ADVANCE
15.3
1.01E+08 1.01E+08 24000 0 4210 MPI_Alltoall()
7.2
4.222E+07 4.72E+07 1001 2002 47149 GET_FIELD_EXPLICIT
4.5
2.855E+07 2.987E+07 1 770
29871330 MAKE_MAXWELL_MATRIX
3.8
2.494E+07 2.494E+07 10979 0 2272 MPI_Allreduce()
2.5
1.62E+07 1.62E+07 1 0
16202268 MAKE_LAMBDA_GRID
2.2
1.385E+07 1.423E+07 1 130
14226450 MAKE_POISSON_MATRIX
0.6
4.245E+06 4.245E+06 1122 0 3783 MPI_Bcast()
0.3
1.418E+05 1.669E+06 1000 1000 1669 GET_ERROR
0.2
7.661E+04 1.546E+06 1000 1000 1546 CATCH_HALT_SIGNAL
0.1
6.991E+05 6.991E+05 1 0
699109 MAKE_THETA_OPERATORS
0.1
1.585E+05 6.249E+05 1 257
624862 MAKE_BLEND_ARRAYS
0.1
2.948E+05 5.638E+05 10 160 56377 WRITE_BIG
0.1 3.913E+05
3.913E+05 1 0
391318 MAKE_RADIAL_OPERATORS
0.1
3.398E+05 3.716E+05 1 6
371605 MAKE_IMPLICIT_ADVECT
0.0
7.07E+04 7.09E+04 1 8 70902 MPI_Init()
0.0 4.043E+04
4.043E+04 2 0 20213 MPI_Comm_split()
0.0
4820 4929 1 4 4929 MPI_Finalize()
0.0
381 4467 1 2 4467 WRITE_INPUT
0.0
2816 2816 12 0 235 MPI_Reduce()
0.0
1673 1673 1 0 1673 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Keyval_free()
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
59 59 4 0 15 MPI_Comm_rank()
0.0
25 25 2
0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
25, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 26;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.181E+05
6.522E+08 1 2808
652213376 .TAU application
89.6
7.066E+06 5.846E+08 1000 2008
584591 DO_FULLADVANCE
88.2
2.142E+08 5.754E+08 1000 28000
575374 TIMESTEP_SSP_322
32.9
1.994E+08 2.144E+08 3000 6000 71473 GET_KINETIC_ADVANCE
15.3
9.99E+07 9.99E+07 24000 0 4162 MPI_Alltoall()
7.2
4.183E+07 4.688E+07 1001 2002 46829 GET_FIELD_EXPLICIT
4.6
2.852E+07 2.979E+07 1 770
29785102 MAKE_MAXWELL_MATRIX
3.8
2.503E+07 2.503E+07 10979 0 2280 MPI_Allreduce()
2.5
1.617E+07 1.617E+07 1 0
16173172 MAKE_LAMBDA_GRID
2.2
1.383E+07 1.42E+07 1 130
14204875 MAKE_POISSON_MATRIX
0.5
3.265E+06 3.265E+06 1122 0 2910 MPI_Bcast()
0.3
1.452E+05 1.729E+06 1000 1000 1729 GET_ERROR
0.2
7.313E+04 1.493E+06 1000 1000 1493 CATCH_HALT_SIGNAL
0.1
6.945E+05 6.945E+05 1 0
694494 MAKE_THETA_OPERATORS
0.1
1.584E+05 6.032E+05 1 257
603202 MAKE_BLEND_ARRAYS
0.1
2.83E+05 5.348E+05 10 160 53479 WRITE_BIG
0.1
3.892E+05 3.892E+05 1 0
389189 MAKE_RADIAL_OPERATORS
0.1
3.423E+05 3.731E+05 1 6
373116 MAKE_IMPLICIT_ADVECT
0.0
6.846E+04 6.866E+04 1 8 68663 MPI_Init()
0.0
3.043E+04 3.043E+04 2 0 15214 MPI_Comm_split()
0.0
5033 5140 1
4 5140 MPI_Finalize()
0.0
405 4054 1 2 4054 WRITE_INPUT
0.0
3425 3425 12 0 285 MPI_Reduce()
0.0
1663 1663 1 0 1663 MAKE_PROFILES
0.0
107 107 4 0 27 MPI_Keyval_free()
0.0
106 106 4 0 26 MPI_Attr_put()
0.0 102 102 4 0 26 MPI_Attr_get()
0.0
58 58 4 0 14 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
26, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 27;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.046E+05
6.638E+08 1 2808
663833457 .TAU application
89.5
7.07E+06 5.938E+08 1000 2008
593850 DO_FULLADVANCE
88.1
2.23E+08 5.846E+08 1000 28000
584628 TIMESTEP_SSP_322
32.4
2.007E+08 2.154E+08 3000 6000 71801 GET_KINETIC_ADVANCE
14.9
9.894E+07 9.894E+07 24000 0 4122 MPI_Alltoall()
7.1
4.222E+07 4.733E+07 1001 2002 47285 GET_FIELD_EXPLICIT
4.5
2.856E+07 2.982E+07 1 770
29816177 MAKE_MAXWELL_MATRIX
3.9
2.604E+07 2.604E+07 10979 0 2372 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16201780 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.423E+07 1 130
14232064 MAKE_POISSON_MATRIX
0.6
4.23E+06 4.23E+06 1122 0 3770 MPI_Bcast()
0.3 1.45E+05 1.706E+06 1000 1000 1706 GET_ERROR
0.2
7.022E+04 1.528E+06 1000 1000 1528 CATCH_HALT_SIGNAL
0.1
6.994E+05 6.994E+05 1 0
699444 MAKE_THETA_OPERATORS
0.1
1.588E+05 6.315E+05 1 257
631452 MAKE_BLEND_ARRAYS
0.1
2.932E+05 5.655E+05 10 160 56550 WRITE_BIG
0.1
3.927E+05 3.927E+05 1 0
392720 MAKE_RADIAL_OPERATORS
0.1
3.386E+05 3.702E+05 1 6
370229 MAKE_IMPLICIT_ADVECT
0.0
7.051E+04 7.072E+04 1 8 70716 MPI_Init()
0.0
3.996E+04 3.996E+04 2 0 19981 MPI_Comm_split()
0.0
4835 4961 1 4 4961 MPI_Finalize()
0.0
416 4370 1 2 4370 WRITE_INPUT
0.0
2746 2746 12 0 229 MPI_Reduce()
0.0
1723 1723 1 0 1723 MAKE_PROFILES
0.0
126 126 4 0 32 MPI_Keyval_free()
0.0
110 110 4 0 28 MPI_Attr_put()
0.0
98 98 4 0
24 MPI_Attr_get()
0.0
66 66 4 0 16 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
27, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 28;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.031E+05
6.608E+08 1 2808
660755746 .TAU application
89.7
7.072E+06 5.93E+08 1000 2008
592992 DO_FULLADVANCE
88.2
2.224E+08 5.827E+08 1000 28000
582740 TIMESTEP_SSP_322
32.4
1.995E+08 2.144E+08 3000 6000 71465 GET_KINETIC_ADVANCE
15.0
9.928E+07 9.928E+07 24000 0 4137 MPI_Alltoall()
7.1
4.167E+07 4.675E+07 1001 2002 46699 GET_FIELD_EXPLICIT
4.5
2.851E+07 2.981E+07 1 770
29812500 MAKE_MAXWELL_MATRIX
3.9
2.592E+07 2.592E+07 10979 0 2361 MPI_Allreduce()
2.4
1.617E+07 1.617E+07 1 0
16172942 MAKE_LAMBDA_GRID
2.1
1.382E+07 1.418E+07 1 130
14180977 MAKE_POISSON_MATRIX
0.5
3.302E+06 3.302E+06 1122 0 2943 MPI_Bcast()
0.2
2.869E+05 1.651E+06 10 160
165148 WRITE_BIG
0.2
1.441E+05 1.64E+06 1000 1000 1640 GET_ERROR
0.2
7.456E+04 1.492E+06 1000 1000 1492 CATCH_HALT_SIGNAL
0.1
6.932E+05 6.932E+05 1 0
693166 MAKE_THETA_OPERATORS
0.1 1.581E+05
6.048E+05 1 257
604796 MAKE_BLEND_ARRAYS
0.1
4.323E+05 4.63E+05 1 6
462995 MAKE_IMPLICIT_ADVECT
0.1
4.242E+05 4.242E+05 1 0
424202 MAKE_RADIAL_OPERATORS
0.0
6.868E+04 6.889E+04 1 8 68886 MPI_Init()
0.0
3.015E+04 3.015E+04 2 0 15076 MPI_Comm_split()
0.0
7777 7777 12 0 648 MPI_Reduce()
0.0
5188 5292 1 4 5292 MPI_Finalize()
0.0
412 4141 1 2 4141 WRITE_INPUT
0.0
1714 1714 1 0 1714 MAKE_PROFILES
0.0
113 113 4 0 28 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Keyval_free()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
61 61 4
0 15 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
28, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 29;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.042E+05
6.618E+08 1 2808
661810536 .TAU application
89.6 7.077E+06
5.93E+08 1000 2008
593022 DO_FULLADVANCE
88.2
2.213E+08 5.839E+08 1000 28000
583886 TIMESTEP_SSP_322
32.5
2.008E+08 2.15E+08 3000 6000 71668 GET_KINETIC_ADVANCE
15.2 1.006E+08 1.006E+08 24000 0 4193 MPI_Alltoall()
7.1
4.203E+07 4.698E+07 1001 2002 46931 GET_FIELD_EXPLICIT
4.5
2.854E+07 2.98E+07 1 770
29797024 MAKE_MAXWELL_MATRIX
3.6
2.401E+07 2.401E+07 10979 0 2187 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16200258 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.422E+07 1 130
14222106 MAKE_POISSON_MATRIX
0.6
4.219E+06 4.219E+06 1122 0 3760 MPI_Bcast()
0.2
1.413E+05 1.622E+06 1000 1000 1622 GET_ERROR
0.2
7.532E+04 1.528E+06 1000 1000 1528 CATCH_HALT_SIGNAL
0.1
7.009E+05 7.009E+05 1 0
700924 MAKE_THETA_OPERATORS
0.1
1.579E+05 6.15E+05 1 257
615002 MAKE_BLEND_ARRAYS
0.1
2.98E+05 5.573E+05 10 160 55725 WRITE_BIG
0.1
4.411E+05 4.717E+05 1 6
471712 MAKE_IMPLICIT_ADVECT
0.1
4.285E+05 4.285E+05 1 0
428527 MAKE_RADIAL_OPERATORS
0.0
7.069E+04 7.09E+04 1 8 70903 MPI_Init()
0.0
3.938E+04 3.938E+04 2 0 19688 MPI_Comm_split()
0.0
4756 4862 1 4 4862 MPI_Finalize()
0.0
384 4543 1 2 4543 WRITE_INPUT
0.0
2764 2764 12 0 230 MPI_Reduce()
0.0
1785 1785 1 0 1785 MAKE_PROFILES
0.0
110 110 4 0 28 MPI_Attr_put()
0.0
106 106 4 0 26 MPI_Keyval_free()
0.0
103 103 4 0 26 MPI_Attr_get()
0.0
65 65 4 0 16 MPI_Comm_rank()
0.0
31 31 2 0 16 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
29, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 30;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.109E+05
6.644E+08 1 2808
664437601 .TAU application
89.6
7.075E+06 5.955E+08 1000 2008
595542 DO_FULLADVANCE
88.2
2.232E+08 5.863E+08 1000 28000
586289 TIMESTEP_SSP_322
32.6
2.009E+08 2.165E+08 3000 6000 72156 GET_KINETIC_ADVANCE
14.9
9.916E+07 9.916E+07 24000 0 4132 MPI_Alltoall()
7.1
4.204E+07 4.749E+07 1001 2002 47442 GET_FIELD_EXPLICIT
4.5
2.855E+07 2.984E+07 1 770
29843999 MAKE_MAXWELL_MATRIX
3.9
2.602E+07 2.602E+07 10979 0 2370 MPI_Allreduce()
2.4
1.62E+07 1.62E+07 1 0
16200553 MAKE_LAMBDA_GRID
2.1
1.385E+07 1.422E+07 1 130
14218962 MAKE_POISSON_MATRIX
0.6
4.257E+06 4.257E+06 1122 0 3794 MPI_Bcast()
0.3
1.4E+05 1.727E+06 1000 1000 1727 GET_ERROR
0.2
7.463E+04 1.557E+06 1000 1000 1557 CATCH_HALT_SIGNAL
0.1
7.013E+05 7.013E+05 1 0
701276 MAKE_THETA_OPERATORS
0.1
1.597E+05 6.266E+05 1 257
626619 MAKE_BLEND_ARRAYS
0.1
2.973E+05 5.702E+05 10 160 57020 WRITE_BIG
0.1
4.429E+05 4.735E+05 1 6
473549 MAKE_IMPLICIT_ADVECT
0.1
4.285E+05 4.285E+05 1 0
428480 MAKE_RADIAL_OPERATORS
0.0
7.098E+04 7.118E+04 1 8 71184 MPI_Init()
0.0
3.979E+04 3.979E+04 2 0 19897 MPI_Comm_split()
0.0
4829 4925 1 4 4925 MPI_Finalize()
0.0
395 4696 1 2 4696 WRITE_INPUT
0.0
4657 4657 12
0 388 MPI_Reduce()
0.0
1718 1718 1 0 1718 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
100 100 4 0
25 MPI_Attr_get()
0.0
96 96 4 0 24 MPI_Keyval_free()
0.0
66 66 4 0 16 MPI_Comm_rank()
0.0
23 23 2 0
12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
30, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0
0 0 Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 31;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.051E+05
6.619E+08 1 2808
661916747 .TAU application
89.8
7.072E+06 5.942E+08 1000 2008
594160 DO_FULLADVANCE
88.2
2.228E+08 5.839E+08 1000 28000
583909 TIMESTEP_SSP_322
32.5
1.994E+08 2.153E+08 3000 6000 71779 GET_KINETIC_ADVANCE
14.9
9.868E+07 9.868E+07 24000 0 4112 MPI_Alltoall()
7.1
4.168E+07 4.719E+07 1001
2002 47142
GET_FIELD_EXPLICIT
4.5
2.852E+07 2.981E+07 1 770
29810957 MAKE_MAXWELL_MATRIX
4.1
2.736E+07 2.736E+07 10979 0 2492 MPI_Allreduce()
2.4
1.617E+07 1.617E+07 1 0
16173250 MAKE_LAMBDA_GRID
2.1
1.383E+07 1.416E+07 1 130
14161230 MAKE_POISSON_MATRIX
0.5
3.304E+06 3.304E+06 1122 0 2945 MPI_Bcast()
0.3
1.428E+05 1.679E+06 1000 1000 1679 GET_ERROR
0.2
2.87E+05 1.613E+06 10 160
161279 WRITE_BIG
0.2
7.55E+04 1.504E+06 1000 1000 1504 CATCH_HALT_SIGNAL
0.1
6.934E+05 6.934E+05 1 0
693391 MAKE_THETA_OPERATORS
0.1
1.577E+05 6.046E+05 1 257
604600 MAKE_BLEND_ARRAYS
0.1
4.346E+05 4.648E+05 1 6
464822 MAKE_IMPLICIT_ADVECT
0.1
4.246E+05 4.246E+05 1 0
424597 MAKE_RADIAL_OPERATORS
0.0
6.826E+04 6.847E+04 1 8 68470 MPI_Init()
0.0
3.031E+04 3.031E+04 2 0 15157 MPI_Comm_split()
0.0
5066 5178 1 4 5178 MPI_Finalize()
0.0
401 4003 1 2 4003 WRITE_INPUT
0.0
2675 2675 12 0 223 MPI_Reduce()
0.0
1719 1719 1 0 1719 MAKE_PROFILES
0.0
112 112 4 0 28 MPI_Keyval_free()
0.0
106 106 4 0 26 MPI_Attr_put()
0.0
100 100
4 0 25 MPI_Attr_get()
0.0
59 59 4 0 15 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
31, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 32;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.883E+05
6.112E+08 1 2808
611221534 .TAU application
93.9
7.184E+06 5.739E+08 1000 3008
573926 DO_FULLADVANCE
92.3
2.206E+08 5.641E+08 1000 28000
564136 TIMESTEP_SSP_322
32.7
1.851E+08 1.997E+08 3000 6000 66577 GET_KINETIC_ADVANCE
16.8
1.024E+08 1.024E+08 24000 0 4268 MPI_Alltoall()
6.8
3.662E+07 4.144E+07 1001
2002 41400 GET_FIELD_EXPLICIT
4.1
2.485E+07 2.485E+07 10979 0 2263 MPI_Allreduce()
2.7
1.62E+07 1.62E+07 1 0
16200468 MAKE_LAMBDA_GRID
1.4
6.974E+06 8.271E+06 1 770
8271232 MAKE_MAXWELL_MATRIX
0.7
3.956E+06 4.368E+06 1 130
4367953 MAKE_POISSON_MATRIX
0.7
4.171E+06 4.171E+06 1122 0 3717 MPI_Bcast()
0.3
1.455E+05 1.999E+06 1000 1000 1999 GET_ERROR
0.2
6.953E+04 1.462E+06 1000 1000 1462 CATCH_HALT_SIGNAL
0.1
7.011E+05 7.011E+05 1 0
701083 MAKE_THETA_OPERATORS
0.1
1.579E+05 5.664E+05 1 257
566378 MAKE_BLEND_ARRAYS
0.1
2.383E+05 5.208E+05 10 196 52077 WRITE_BIG
0.1
4.008E+05 4.008E+05 1 0
400844 MAKE_RADIAL_OPERATORS
0.1
3.418E+05 3.737E+05 1 6
373675 MAKE_IMPLICIT_ADVECT
0.0
2.026E+05 2.026E+05 1037 0 195 MPI_Send()
0.0
645 1.375E+05 1 3
137477 WRITE_INPUT
0.0
7.004E+04 7.024E+04 1 8 70243 MPI_Init()
0.0
3.988E+04 3.988E+04 2 0 19940 MPI_Comm_split()
0.0
1.173E+04 1.173E+04 12 0 977 MPI_Reduce()
0.0
4821 4930 1 4 4930 MPI_Finalize()
0.0
1663 1663 1 0 1663 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
109 109 4 0 27 MPI_Keyval_free()
0.0 99 99 4 0 25 MPI_Attr_get()
0.0
66 66 4 0 16 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
32, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
1037
8192 8 111.5
761.7 Message size sent to all
nodes
---------------------------------------------------------------------------------------
NODE 33;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.069E+05
6.097E+08 1 2808
609692222 .TAU application
94.0
7.071E+06 5.733E+08 1000 2008
573337 DO_FULLADVANCE
92.5
2.201E+08 5.637E+08 1000 28000
563741 TIMESTEP_SSP_322
32.9
1.843E+08 2.003E+08 3000 6000 66779 GET_KINETIC_ADVANCE
16.7
1.016E+08 1.016E+08 24000 0 4235 MPI_Alltoall()
6.8
3.638E+07 4.168E+07 1001 2002 41636 GET_FIELD_EXPLICIT
4.4
2.679E+07 2.679E+07 10979 0 2441 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16172156 MAKE_LAMBDA_GRID
1.4
6.965E+06 8.291E+06 1 770
8291048 MAKE_MAXWELL_MATRIX
0.7
3.942E+06 4.341E+06 1 130
4341391 MAKE_POISSON_MATRIX
0.5
3.307E+06 3.307E+06 1122 0 2947 MPI_Bcast()
0.3 1.474E+05
2.121E+06 1000 1000 2121 GET_ERROR
0.3
7.239E+04 1.528E+06 1000 1000 1528 CATCH_HALT_SIGNAL
0.1
6.925E+05 6.925E+05 1 0
692467 MAKE_THETA_OPERATORS
0.1
1.574E+05 5.737E+05 1 257
573691 MAKE_BLEND_ARRAYS
0.1
2.364E+05 5.046E+05 10 160 50463 WRITE_BIG
0.1
3.983E+05 3.983E+05 1 0
398310 MAKE_RADIAL_OPERATORS
0.1
3.398E+05 3.71E+05 1 6
371049 MAKE_IMPLICIT_ADVECT
0.0
384 8.9E+04 1 2 89001 WRITE_INPUT
0.0
6.817E+04 6.838E+04 1 8 68378 MPI_Init()
0.0
2.875E+04 2.875E+04 2 0 14373 MPI_Comm_split()
0.0
5031 5138 1 4 5138 MPI_Finalize()
0.0
2730 2730 12 0 228 MPI_Reduce()
0.0
1737 1737 1
0 1737 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
107 107 4 0 27 MPI_Keyval_free()
0.0
99 99 4
0 25 MPI_Attr_get()
0.0
62 62 4 0 16 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
33, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 34;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.947E+05
6.101E+08 1 2808
610147986 .TAU application
93.9
7.066E+06 5.727E+08 1000 2008
572746 DO_FULLADVANCE
92.3
2.198E+08 5.631E+08 1000 28000
563145 TIMESTEP_SSP_322
32.8
1.849E+08 2.001E+08 3000 6000 66696 GET_KINETIC_ADVANCE
16.6
1.016E+08 1.016E+08 24000 0 4232 MPI_Alltoall()
6.8
3.659E+07 4.175E+07 1001 2002 41711 GET_FIELD_EXPLICIT
4.2
2.582E+07 2.582E+07 10979 0 2352 MPI_Allreduce()
2.7
1.62E+07 1.62E+07 1 0
16201110 MAKE_LAMBDA_GRID
1.4
6.975E+06 8.296E+06 1 770
8296387 MAKE_MAXWELL_MATRIX
0.7
3.953E+06 4.318E+06 1 130
4318443 MAKE_POISSON_MATRIX
0.7
4.304E+06 4.304E+06 1122 0 3836 MPI_Bcast()
0.3
1.444E+05 2.132E+06 1000 1000 2132 GET_ERROR
0.3
7.296E+04 1.598E+06 1000 1000 1598 CATCH_HALT_SIGNAL
0.1
7.015E+05 7.015E+05 1 0
701549 MAKE_THETA_OPERATORS
0.1
1.591E+05 5.708E+05 1 257
570761 MAKE_BLEND_ARRAYS
0.1
2.333E+05 5.084E+05 10 160 50838 WRITE_BIG
0.1
4.015E+05 4.015E+05 1 0
401512 MAKE_RADIAL_OPERATORS
0.1
3.415E+05 3.731E+05 1 6
373100 MAKE_IMPLICIT_ADVECT
0.0
401 1.375E+05 1 2
137504 WRITE_INPUT
0.0
7.114E+04 7.135E+04 1 8 71354 MPI_Init()
0.0
4.003E+04 4.003E+04 2 0 20016 MPI_Comm_split()
0.0
5123 5226 1 4 5226 MPI_Finalize()
0.0
3475 3475 12 0 290 MPI_Reduce()
0.0
1713 1713 1 0 1713 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
103 103 4 0 26 MPI_Keyval_free()
0.0
102 102 4 0 26 MPI_Attr_get()
0.0
62 62 4
0 16 MPI_Comm_rank()
0.0
29 29 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
34, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04 4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 35;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.913E+05
6.096E+08 1
2808 609611611 .TAU
application
94.1
7.07E+06 5.734E+08 1000 2008
573360 DO_FULLADVANCE
92.5
2.224E+08 5.637E+08 1000 28000
563729 TIMESTEP_SSP_322
32.8
1.845E+08 1.998E+08 3000
6000 66586 GET_KINETIC_ADVANCE
16.4
1.003E+08 1.003E+08 24000 0 4177 MPI_Alltoall()
6.8
3.636E+07 4.132E+07 1001 2002 41279 GET_FIELD_EXPLICIT
4.2
2.571E+07 2.571E+07 10979
0 2342 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16171288 MAKE_LAMBDA_GRID
1.4
6.956E+06 8.273E+06 1 770
8273125 MAKE_MAXWELL_MATRIX
0.7
3.939E+06 4.321E+06 1 130
4320728 MAKE_POISSON_MATRIX
0.5
3.307E+06 3.307E+06 1122 0 2947 MPI_Bcast()
0.4
1.441E+05 2.173E+06 1000 1000 2173 GET_ERROR
0.3
7.275E+04 1.541E+06 1000 1000 1541 CATCH_HALT_SIGNAL
0.1
6.931E+05 6.931E+05 1 0
693075 MAKE_THETA_OPERATORS
0.1
1.59E+05 5.67E+05 1 257
567039 MAKE_BLEND_ARRAYS
0.1
2.358E+05 4.864E+05 10 160 48640 WRITE_BIG
0.1
3.982E+05 3.982E+05 1 0
398194 MAKE_RADIAL_OPERATORS
0.1
3.421E+05 3.731E+05 1 6
373070 MAKE_IMPLICIT_ADVECT
0.0
400 8.712E+04 1 2 87119 WRITE_INPUT
0.0
6.782E+04 6.803E+04 1 8 68029 MPI_Init()
0.0
3.006E+04 3.006E+04 2 0 15028 MPI_Comm_split()
0.0
5073 5179 1 4 5179 MPI_Finalize()
0.0
2681 2681 12 0 223 MPI_Reduce()
0.0
1742 1742 1 0 1742 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
106 106 4 0 26 MPI_Keyval_free()
0.0
102 102 4 0 26 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0 26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
35, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 36;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.073E+05
6.136E+08 1 2808
613621364 .TAU application
94.1
7.071E+06 5.772E+08 1000 2008
577209 DO_FULLADVANCE
92.5
2.231E+08 5.677E+08 1000 28000
567715 TIMESTEP_SSP_322
32.7
1.847E+08 2.006E+08 3000 6000 66853 GET_KINETIC_ADVANCE
16.6
1.018E+08 1.018E+08 24000 0 4240 MPI_Alltoall()
6.9
3.639E+07 4.229E+07 1001 2002 42248 GET_FIELD_EXPLICIT
4.4
2.711E+07 2.711E+07 10979 0 2469 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16173400 MAKE_LAMBDA_GRID
1.3
6.957E+06 8.242E+06 1 770
8242449 MAKE_MAXWELL_MATRIX
0.7
3.927E+06 4.312E+06 1 130
4312378 MAKE_POISSON_MATRIX
0.5
3.318E+06 3.318E+06 1122 0 2957 MPI_Bcast()
0.3
1.425E+05 2.02E+06 1000 1000 2020 GET_ERROR
0.2
7.473E+04 1.508E+06 1000 1000 1508 CATCH_HALT_SIGNAL
0.1
6.927E+05 6.927E+05 1 0
692677 MAKE_THETA_OPERATORS
0.1
1.57E+05 5.697E+05 1 257
569694 MAKE_BLEND_ARRAYS
0.1
2.329E+05 5.051E+05 10 160 50509 WRITE_BIG
0.1
4.444E+05 4.755E+05 1 6
475515 MAKE_IMPLICIT_ADVECT
0.1
4.345E+05 4.345E+05 1 0
434478 MAKE_RADIAL_OPERATORS
0.0
386 9.162E+04 1 2 91615 WRITE_INPUT
0.0
6.83E+04 6.851E+04 1 8 68511 MPI_Init()
0.0
2.981E+04 2.981E+04 2 0 14903 MPI_Comm_split()
0.0
5131 5237 1 4 5237 MPI_Finalize()
0.0
4474 4474 12
0 373 MPI_Reduce()
0.0
1718 1718 1 0 1718 MAKE_PROFILES
0.0
106 106 4 0 26 MPI_Attr_put()
0.0
106 106 4 0
26 MPI_Keyval_free()
0.0
102 102 4 0 26 MPI_Attr_get()
0.0
56 56 4 0 14 MPI_Comm_rank()
0.0
24 24 2 0
12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
36, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0
0 0 Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 37;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.061E+05
6.156E+08 1 2808
615638965 .TAU application
93.9
7.069E+06 5.783E+08 1000 2008
578268 DO_FULLADVANCE
91.8
2.209E+08 5.653E+08 1000 28000
565322 TIMESTEP_SSP_322
32.4
1.853E+08 1.996E+08 3000 6000 66550 GET_KINETIC_ADVANCE
16.8
1.034E+08 1.034E+08 24000 0 4308 MPI_Alltoall()
6.7
3.666E+07 4.143E+07 1001
2002 41387
GET_FIELD_EXPLICIT
4.5
2.782E+07 2.782E+07 10979 0 2534 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16199982 MAKE_LAMBDA_GRID
1.3
6.973E+06 8.247E+06 1
770 8247179
MAKE_MAXWELL_MATRIX
0.9
1.426E+05 5.478E+06 1000 1000 5478 GET_ERROR
0.7
3.949E+06 4.3E+06 1 130
4299910 MAKE_POISSON_MATRIX
0.7
4.213E+06 4.213E+06 1122 0 3755 MPI_Bcast()
0.2
7.856E+04 1.52E+06 1000 1000 1520 CATCH_HALT_SIGNAL
0.1
7.012E+05 7.012E+05 1 0
701208 MAKE_THETA_OPERATORS
0.1
1.591E+05 5.693E+05 1 257
569343 MAKE_BLEND_ARRAYS
0.1
2.427E+05 5.049E+05 10 160 50490 WRITE_BIG
0.1
4.39E+05 4.701E+05 1 6
470059 MAKE_IMPLICIT_ADVECT
0.1 4.38E+05
4.38E+05 1 0
437967 MAKE_RADIAL_OPERATORS
0.0
396 1.369E+05 1 2
136939 WRITE_INPUT
0.0
7.029E+04 7.05E+04 1 8 70497 MPI_Init()
0.0
3.976E+04 3.976E+04 2 0 19880 MPI_Comm_split()
0.0
4911 5015 1 4 5015 MPI_Finalize()
0.0
2488 2488 12 0 207 MPI_Reduce()
0.0
1669 1669 1 0 1669 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Keyval_free()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
67 67 4 0 17 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
37, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 38;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.086E+05
6.142E+08 1 2808
614217011 .TAU application
94.1
7.07E+06 5.778E+08 1000 2008
577776 DO_FULLADVANCE
92.5
2.233E+08 5.68E+08 1000 28000
568038 TIMESTEP_SSP_322
32.6
1.846E+08 2.002E+08 3000 6000 66725 GET_KINETIC_ADVANCE
16.8
1.03E+08 1.03E+08 24000 0 4291 MPI_Alltoall()
6.8
3.641E+07 4.158E+07 1001
2002 41541 GET_FIELD_EXPLICIT
4.3
2.632E+07 2.632E+07 10979 0 2398 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16172948 MAKE_LAMBDA_GRID
1.3
6.965E+06 8.271E+06 1 770
8271377 MAKE_MAXWELL_MATRIX
0.7
3.938E+06 4.359E+06 1 130
4358527 MAKE_POISSON_MATRIX
0.5
3.265E+06 3.265E+06 1122 0 2910 MPI_Bcast()
0.4
1.432E+05 2.278E+06 1000 1000 2278 GET_ERROR
0.2
7.475E+04 1.501E+06 1000 1000 1501 CATCH_HALT_SIGNAL
0.1
6.933E+05 6.933E+05 1 0
693273 MAKE_THETA_OPERATORS
0.1
1.573E+05 5.645E+05 1 257
564463 MAKE_BLEND_ARRAYS
0.1
2.331E+05 4.91E+05 10 160 49100 WRITE_BIG
0.1
4.468E+05 4.772E+05 1 6
477186 MAKE_IMPLICIT_ADVECT
0.1
4.343E+05 4.343E+05 1 0
434254 MAKE_RADIAL_OPERATORS
0.0
382 9.114E+04 1 2 91140 WRITE_INPUT
0.0
7.055E+04 7.077E+04 1 8 70766 MPI_Init()
0.0
2.988E+04 2.988E+04 2 0 14942 MPI_Comm_split()
0.0
4867 4968 1 4 4968 MPI_Finalize()
0.0
3394 3394 12 0 283 MPI_Reduce()
0.0
1735 1735 1 0 1735 MAKE_PROFILES
0.0
110 110 4 0 28 MPI_Attr_put()
0.0
103 103 4 0 26 MPI_Attr_get()
0.0
101 101 4 0 25 MPI_Keyval_free()
0.0 60 60 4 0 15 MPI_Comm_rank()
0.0
29 29 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
38, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 39;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.061E+05
6.15E+08 1 2808
614973111 .TAU application
93.9
7.067E+06 5.776E+08 1000 2008
577578 DO_FULLADVANCE
92.4
2.209E+08 5.681E+08 1000 28000
568058 TIMESTEP_SSP_322
32.7
1.854E+08 2.01E+08 3000 6000 67002 GET_KINETIC_ADVANCE
17.0
1.047E+08 1.047E+08 24000 0 4362 MPI_Alltoall()
6.7
3.665E+07 4.151E+07 1001 2002 41466 GET_FIELD_EXPLICIT
4.2
2.574E+07 2.574E+07 10979 0 2344 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16199432 MAKE_LAMBDA_GRID
1.3
6.978E+06 8.238E+06 1 770
8237546 MAKE_MAXWELL_MATRIX
0.7
3.953E+06 4.307E+06 1 130
4307159 MAKE_POISSON_MATRIX
0.7
4.244E+06 4.244E+06 1122 0 3782 MPI_Bcast()
0.3
1.444E+05 2.052E+06 1000 1000 2052 GET_ERROR
0.2
7.776E+04 1.525E+06 1000 1000 1525 CATCH_HALT_SIGNAL
0.1
7.012E+05 7.012E+05 1 0
701171 MAKE_THETA_OPERATORS
0.1
1.585E+05 5.691E+05 1 257
569148 MAKE_BLEND_ARRAYS
0.1
2.445E+05 5.065E+05 10 160 50647 WRITE_BIG
0.1
4.361E+05 4.675E+05 1 6
467525 MAKE_IMPLICIT_ADVECT
0.1
4.384E+05 4.384E+05 1 0
438442 MAKE_RADIAL_OPERATORS
0.0
395 1.377E+05 1 2
137710 WRITE_INPUT
0.0
6.839E+04 6.86E+04 1 8 68600 MPI_Init()
0.0
3.987E+04 3.987E+04 2 0 19933 MPI_Comm_split()
0.0
4995 5105 1 4 5105 MPI_Finalize()
0.0
2439 2439 12 0 203 MPI_Reduce()
0.0
1757 1757 1 0 1757 MAKE_PROFILES
0.0
110 110 4
0 28 MPI_Keyval_free()
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
61 61 4 0
15 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
39, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 40;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.976E+05
6.122E+08 1 2808
612216433 .TAU application
93.9
7.064E+06 5.749E+08 1000 2008
574877 DO_FULLADVANCE
92.3
2.198E+08 5.652E+08 1000 28000
565205 TIMESTEP_SSP_322
32.7
1.851E+08 2.001E+08 3000 6000 66704 GET_KINETIC_ADVANCE
17.0
1.038E+08 1.038E+08 24000 0 4327 MPI_Alltoall()
6.8
3.662E+07 4.153E+07 1001 2002 41485 GET_FIELD_EXPLICIT
4.2
2.544E+07 2.544E+07 10979 0 2317 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201137 MAKE_LAMBDA_GRID
1.4
6.974E+06 8.304E+06 1 770
8303970 MAKE_MAXWELL_MATRIX
0.7
3.954E+06 4.309E+06 1 130
4309260 MAKE_POISSON_MATRIX
0.7
4.239E+06 4.239E+06 1122 0 3778 MPI_Bcast()
0.4
1.42E+05 2.2E+06 1000 1000 2200 GET_ERROR
0.3
7.226E+04 1.532E+06 1000 1000 1532 CATCH_HALT_SIGNAL
0.1
7.015E+05 7.015E+05 1 0
701486 MAKE_THETA_OPERATORS
0.1
1.579E+05 5.687E+05 1 257
568654 MAKE_BLEND_ARRAYS
0.1
2.335E+05 5.116E+05 10 160 51159 WRITE_BIG
0.1
4.013E+05 4.013E+05 1 0
401328 MAKE_RADIAL_OPERATORS
0.1
3.456E+05 3.777E+05 1 6
377731 MAKE_IMPLICIT_ADVECT
0.0
385 1.361E+05 1 2
136056 WRITE_INPUT
0.0
6.78E+04 6.8E+04 1 8 68003 MPI_Init()
0.0
4.031E+04 4.031E+04 2 0 20156 MPI_Comm_split()
0.0
9281 9281 12 0 773 MPI_Reduce()
0.0
5154 5263 1 4 5263 MPI_Finalize()
0.0
1662 1662 1 0 1662 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Keyval_free()
0.0
106 106 4 0 26 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
59 59 4 0 15 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
40, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 41;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.909E+05
6.123E+08 1 2808
612323869 .TAU application
94.1
7.069E+06 5.761E+08 1000 2008
576117 DO_FULLADVANCE
92.5
2.228E+08 5.665E+08 1000 28000
566517 TIMESTEP_SSP_322
32.5
1.844E+08 1.992E+08 3000 6000 66413 GET_KINETIC_ADVANCE
16.9
1.033E+08 1.033E+08 24000 0 4306 MPI_Alltoall()
6.7
3.638E+07 4.116E+07 1001 2002 41116 GET_FIELD_EXPLICIT
4.1
2.5E+07 2.5E+07 10979 0 2277 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16171326 MAKE_LAMBDA_GRID
1.4 6.961E+06 8.279E+06 1 770
8279232 MAKE_MAXWELL_MATRIX
0.7
3.944E+06 4.33E+06 1 130
4329979 MAKE_POISSON_MATRIX
0.5
3.251E+06 3.251E+06 1122 0 2898 MPI_Bcast()
0.4
1.434E+05 2.15E+06 1000 1000 2150 GET_ERROR
0.2
7.267E+04 1.523E+06 1000 1000 1523 CATCH_HALT_SIGNAL
0.1
6.934E+05 6.934E+05 1 0
693379 MAKE_THETA_OPERATORS
0.1
1.571E+05 5.676E+05 1 257
567552 MAKE_BLEND_ARRAYS
0.1
2.359E+05 4.822E+05 10 160 48220 WRITE_BIG
0.1
3.983E+05 3.983E+05 1 0
398266 MAKE_RADIAL_OPERATORS
0.1
3.398E+05 3.708E+05 1 6
370754 MAKE_IMPLICIT_ADVECT
0.0
400 8.815E+04 1 2 88146 WRITE_INPUT
0.0
7.093E+04 7.114E+04 1 8 71141 MPI_Init()
0.0
2.899E+04 2.899E+04 2 0 14493 MPI_Comm_split()
0.0
4720 4822 1 4 4822 MPI_Finalize()
0.0
2608 2608 12 0 217 MPI_Reduce()
0.0
1731 1731 1
0 1731 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
102 102 4 0 26 MPI_Keyval_free()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
23 23 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
41, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 42;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.036E+05
6.11E+08 1 2808
610987330 .TAU application
93.9
7.069E+06 5.737E+08 1000 2008
573679 DO_FULLADVANCE
92.3
2.199E+08 5.641E+08 1000 28000
564059 TIMESTEP_SSP_322
32.8
1.852E+08 2.003E+08 3000 6000 66780 GET_KINETIC_ADVANCE
16.7
1.021E+08 1.021E+08 24000 0 4253 MPI_Alltoall()
6.8
3.658E+07 4.181E+07 1001 2002 41765 GET_FIELD_EXPLICIT
4.2
2.588E+07 2.588E+07 10979 0 2357 MPI_Allreduce()
2.7
1.62E+07 1.62E+07 1 0
16200152 MAKE_LAMBDA_GRID
1.4
6.976E+06 8.256E+06 1 770
8256117 MAKE_MAXWELL_MATRIX
0.7
3.957E+06 4.339E+06 1 130
4338832 MAKE_POISSON_MATRIX
0.7
4.234E+06 4.234E+06 1122 0 3773 MPI_Bcast()
0.4
1.43E+05 2.151E+06 1000 1000 2151 GET_ERROR
0.3
7.092E+04 1.53E+06 1000 1000 1530 CATCH_HALT_SIGNAL
0.1
7.012E+05 7.012E+05 1 0
701198 MAKE_THETA_OPERATORS
0.1 1.568E+05 5.672E+05 1 257
567153 MAKE_BLEND_ARRAYS
0.1
2.381E+05 5.051E+05 10 160 50512 WRITE_BIG
0.1
4.018E+05 4.018E+05 1 0
401798 MAKE_RADIAL_OPERATORS
0.1
3.429E+05 3.746E+05 1 6
374603 MAKE_IMPLICIT_ADVECT
0.0
407 1.381E+05 1 2
138094 WRITE_INPUT
0.0
7.05E+04 7.071E+04 1 8 70713 MPI_Init()
0.0
3.998E+04 3.998E+04 2 0 19988 MPI_Comm_split()
0.0
4815 4920 1 4 4920 MPI_Finalize()
0.0
3749 3749 12 0 312 MPI_Reduce()
0.0
1736 1736 1
0 1736 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
105 105 4 0 26 MPI_Keyval_free()
0.0
103 103 4
0 26 MPI_Attr_get()
0.0
59 59 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
42, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 43;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.909E+05
6.116E+08 1 2808
611578796 .TAU application
94.1
7.073E+06 5.754E+08 1000 2008
575415 DO_FULLADVANCE
92.5
2.222E+08 5.658E+08 1000 28000
565761 TIMESTEP_SSP_322
32.7
1.845E+08 2E+08 3000 6000 66660 GET_KINETIC_ADVANCE
16.6
1.018E+08 1.018E+08 24000 0 4240 MPI_Alltoall()
6.8
3.64E+07 4.186E+07 1001 2002 41820 GET_FIELD_EXPLICIT
4.3
2.637E+07 2.637E+07 10979 0 2402 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16172916 MAKE_LAMBDA_GRID
1.3
6.953E+06 8.242E+06 1 770
8242469 MAKE_MAXWELL_MATRIX
0.7
3.939E+06 4.275E+06 1 130
4274859 MAKE_POISSON_MATRIX
0.5
3.31E+06 3.31E+06 1122 0 2950 MPI_Bcast()
0.4
1.459E+05 2.187E+06 1000 1000 2187 GET_ERROR
0.3
7.273E+04 1.547E+06 1000 1000 1547 CATCH_HALT_SIGNAL
0.1
6.941E+05 6.941E+05 1 0
694097 MAKE_THETA_OPERATORS
0.1
1.569E+05 5.634E+05 1 257
563434 MAKE_BLEND_ARRAYS
0.1
2.316E+05 4.938E+05 10 160 49384 WRITE_BIG
0.1
3.987E+05 3.987E+05 1 0
398740 MAKE_RADIAL_OPERATORS
0.1
3.442E+05 3.751E+05 1 6
375121 MAKE_IMPLICIT_ADVECT
0.0
386 8.778E+04 1 2 87785 WRITE_INPUT
0.0
6.777E+04 6.797E+04 1 8 67974 MPI_Init()
0.0
2.92E+04 2.92E+04 2 0 14598 MPI_Comm_split()
0.0
5064 5167 1 4 5167 MPI_Finalize()
0.0
2546 2546 12 0 212 MPI_Reduce()
0.0
1748 1748 1 0 1748 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
103 103 4 0 26 MPI_Keyval_free()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
62 62 4
0 16 MPI_Comm_rank()
0.0
23 23 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
43, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 44;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.033E+05
6.141E+08 1
2808 614118511 .TAU
application
93.9
7.068E+06 5.767E+08 1000 2008
576706 DO_FULLADVANCE
92.4
2.232E+08 5.672E+08 1000 28000
567196 TIMESTEP_SSP_322
32.6
1.855E+08 2.001E+08 3000
6000 66716 GET_KINETIC_ADVANCE
16.6
1.022E+08 1.022E+08 24000 0 4257 MPI_Alltoall()
6.8
3.661E+07 4.171E+07 1001 2002 41668 GET_FIELD_EXPLICIT
4.1
2.512E+07 2.512E+07 10979
0 2288 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16202211 MAKE_LAMBDA_GRID
1.3
6.975E+06 8.272E+06 1 770
8271848 MAKE_MAXWELL_MATRIX
0.7
3.944E+06 4.305E+06 1 130
4304614 MAKE_POISSON_MATRIX
0.7
4.217E+06 4.217E+06 1122 0 3758 MPI_Bcast()
0.3
1.436E+05 2.037E+06 1000 1000 2037 GET_ERROR
0.2
7.713E+04 1.525E+06 1000 1000 1525 CATCH_HALT_SIGNAL
0.1
7.01E+05 7.01E+05 1 0
700961 MAKE_THETA_OPERATORS
0.1
1.581E+05 5.65E+05 1 257
565035 MAKE_BLEND_ARRAYS
0.1
2.415E+05 5.121E+05 10 160 51212 WRITE_BIG
0.1
4.439E+05 4.752E+05 1 6
475241 MAKE_IMPLICIT_ADVECT
0.1
4.383E+05 4.383E+05 1 0
438315 MAKE_RADIAL_OPERATORS
0.0
425 1.381E+05 1 2
138069 WRITE_INPUT
0.0
7.138E+04 7.158E+04 1 8 71578 MPI_Init()
0.0
4.029E+04 4.029E+04 2 0 20146 MPI_Comm_split()
0.0
7224 7224 12 0 602 MPI_Reduce()
0.0
4945 5047 1 4 5047 MPI_Finalize()
0.0
1702 1702 1 0 1702 MAKE_PROFILES
0.0
106 106 4 0 26 MPI_Attr_put()
0.0 102 102 4 0 26 MPI_Keyval_free()
0.0
97 97 4 0 24 MPI_Attr_get()
0.0
67 67 4 0 17 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
44, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096
0 Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 45;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.077E+05
6.103E+08 1 2808
610276608 .TAU application
94.0
7.068E+06 5.74E+08 1000 2008
573956 DO_FULLADVANCE
92.5
2.229E+08 5.646E+08 1000 28000
564587 TIMESTEP_SSP_322
32.7
1.846E+08 1.994E+08 3000 6000 66454 GET_KINETIC_ADVANCE
16.5
1.01E+08 1.01E+08 24000 0 4208 MPI_Alltoall()
6.8
3.642E+07 4.133E+07 1001 2002 41288 GET_FIELD_EXPLICIT
4.1
2.48E+07 2.48E+07 10979 0 2259 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16172758 MAKE_LAMBDA_GRID
1.3
6.961E+06 8.233E+06 1 770
8233105 MAKE_MAXWELL_MATRIX
0.7
3.933E+06 4.324E+06 1 130
4323681 MAKE_POISSON_MATRIX
0.5
3.254E+06 3.254E+06 1122 0 2901 MPI_Bcast()
0.3
1.423E+05 1.917E+06 1000 1000 1917 GET_ERROR
0.2
7.657E+04 1.501E+06 1000 1000 1501 CATCH_HALT_SIGNAL
0.1
6.931E+05 6.931E+05 1 0
693135 MAKE_THETA_OPERATORS
0.1
1.57E+05 5.609E+05 1 257
560893 MAKE_BLEND_ARRAYS
0.1
2.339E+05 4.839E+05 10 160 48390 WRITE_BIG
0.1
4.462E+05 4.768E+05 1 6
476826 MAKE_IMPLICIT_ADVECT
0.1
4.345E+05 4.345E+05 1 0
434475 MAKE_RADIAL_OPERATORS
0.0
387 8.991E+04 1 2 89912 WRITE_INPUT
0.0
6.821E+04 6.842E+04 1 8 68419 MPI_Init()
0.0
2.975E+04 2.975E+04 2 0 14876 MPI_Comm_split()
0.0
5032 5139 1 4 5139 MPI_Finalize()
0.0
2604 2604 12 0 217 MPI_Reduce()
0.0
1728 1728 1 0 1728 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
107 107 4
0 27 MPI_Keyval_free()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
57 57 4 0 14 MPI_Comm_rank()
0.0
26 26 2 0
13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
45, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 46;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.023E+05
6.095E+08 1 2808
609520997 .TAU application
94.0
7.071E+06 5.732E+08 1000 2008
573155 DO_FULLADVANCE
92.5
2.204E+08 5.636E+08 1000 28000
563570 TIMESTEP_SSP_322
32.7
1.844E+08 1.993E+08 3000 6000 66450 GET_KINETIC_ADVANCE
16.8
1.026E+08 1.026E+08 24000 0 4274 MPI_Alltoall()
6.8
3.639E+07 4.13E+07 1001 2002 41261 GET_FIELD_EXPLICIT
4.2
2.533E+07 2.533E+07 10979 0 2307 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16171805 MAKE_LAMBDA_GRID
1.4
6.961E+06 8.255E+06 1 770
8255071 MAKE_MAXWELL_MATRIX
0.7
3.936E+06 4.351E+06 1 130
4351305 MAKE_POISSON_MATRIX
0.5
3.262E+06 3.262E+06 1122 0 2907 MPI_Bcast()
0.3
1.421E+05 2.116E+06 1000 1000 2116 GET_ERROR
0.2
7.751E+04 1.499E+06 1000 1000 1499 CATCH_HALT_SIGNAL
0.1
6.933E+05 6.933E+05 1 0
693343 MAKE_THETA_OPERATORS
0.1
1.559E+05 5.619E+05 1 257
561938 MAKE_BLEND_ARRAYS
0.1
2.308E+05 4.971E+05 10 160 49711 WRITE_BIG
0.1
4.358E+05 4.664E+05 1 6
466358 MAKE_IMPLICIT_ADVECT
0.1
4.35E+05 4.35E+05 1 0
435040 MAKE_RADIAL_OPERATORS
0.0
397 8.943E+04 1 2 89431 WRITE_INPUT
0.0
7.124E+04 7.145E+04 1 8 71451 MPI_Init()
0.0
2.984E+04 2.984E+04 2 0 14918 MPI_Comm_split()
0.0
4911 5020 1 4 5020 MPI_Finalize()
0.0
3564 3564 12 0 297 MPI_Reduce()
0.0
1652 1652 1 0 1652 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
109 109 4 0 27 MPI_Keyval_free()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
58 58 4 0 14 MPI_Comm_rank()
0.0
34 34 2 0 17 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
46, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 47;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.108E+05
6.153E+08 1 2808
615316055 .TAU application
93.9
7.07E+06 5.778E+08 1000 2008
577843 DO_FULLADVANCE
92.2
2.21E+08 5.67E+08 1000 28000
567049 TIMESTEP_SSP_322
32.5 1.853E+08
2.002E+08 3000 6000 66732 GET_KINETIC_ADVANCE
16.9
1.042E+08 1.042E+08 24000 0 4340 MPI_Alltoall()
6.8
3.665E+07 4.169E+07 1001 2002 41646 GET_FIELD_EXPLICIT
4.3 2.66E+07
2.66E+07 10979 0 2423 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200778 MAKE_LAMBDA_GRID
1.4
6.98E+06 8.344E+06 1 770
8343646 MAKE_MAXWELL_MATRIX
0.7 3.959E+06 4.313E+06 1 130
4313492 MAKE_POISSON_MATRIX
0.7
4.201E+06 4.201E+06 1122 0 3745 MPI_Bcast()
0.3
1.441E+05 2.025E+06 1000 1000 2025 GET_ERROR
0.3
2.439E+05 1.805E+06 10 160
180524 WRITE_BIG
0.2
7.892E+04 1.531E+06 1000 1000 1531 CATCH_HALT_SIGNAL
0.1
6.991E+05 6.991E+05 1 0
699089 MAKE_THETA_OPERATORS
0.1
1.598E+05 5.684E+05 1 257
568448 MAKE_BLEND_ARRAYS
0.1
4.381E+05 4.694E+05 1 6
469374 MAKE_IMPLICIT_ADVECT
0.1
4.388E+05 4.388E+05 1 0
438792 MAKE_RADIAL_OPERATORS
0.0
400 1.358E+05 1 2
135820 WRITE_INPUT
0.0
7.081E+04 7.102E+04 1 8 71019 MPI_Init()
0.0
3.989E+04 3.989E+04 2 0 19945 MPI_Comm_split()
0.0
5022 5127 1
4 5127 MPI_Finalize()
0.0
2501 2501 12 0 208 MPI_Reduce()
0.0
1708 1708 1 0 1708 MAKE_PROFILES
0.0
107 107 4 0
27 MPI_Attr_put()
0.0
105 105 4 0 26 MPI_Keyval_free()
0.0
100 100 4 0 25 MPI_Attr_get()
0.0
59 59 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
47, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 48;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.975E+05
6.088E+08 1 2808
608767556 .TAU application
94.0
7.067E+06 5.724E+08 1000 2008
572438 DO_FULLADVANCE
92.5
2.205E+08 5.63E+08 1000 28000
562962 TIMESTEP_SSP_322
32.9
1.842E+08 2E+08 3000 6000 66665 GET_KINETIC_ADVANCE
16.5
1.006E+08 1.006E+08 24000 0 4190 MPI_Alltoall()
6.9
3.639E+07 4.19E+07 1001 2002 41861 GET_FIELD_EXPLICIT
4.4
2.668E+07 2.668E+07 10979 0 2430 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16172045 MAKE_LAMBDA_GRID
1.4
6.958E+06 8.34E+06 1 770
8339996 MAKE_MAXWELL_MATRIX
0.7
3.937E+06 4.309E+06 1 130
4309096 MAKE_POISSON_MATRIX
0.5
3.312E+06 3.312E+06 1122 0 2952 MPI_Bcast()
0.3
1.464E+05 2.023E+06 1000 1000 2023 GET_ERROR
0.2
7.182E+04 1.505E+06 1000 1000 1505 CATCH_HALT_SIGNAL
0.1
6.93E+05 6.93E+05 1 0
693008 MAKE_THETA_OPERATORS
0.1
1.575E+05 5.711E+05 1 257
571071 MAKE_BLEND_ARRAYS
0.1
2.278E+05 4.875E+05 10 160 48752 WRITE_BIG
0.1
3.973E+05 3.973E+05 1 0
397306 MAKE_RADIAL_OPERATORS
0.1
3.381E+05 3.696E+05 1 6
369594 MAKE_IMPLICIT_ADVECT
0.0
439 9.005E+04 1 2 90054 WRITE_INPUT
0.0
6.761E+04 6.782E+04 1 8 67817 MPI_Init()
0.0
2.992E+04 2.992E+04 2 0 14958 MPI_Comm_split()
0.0
5028 5135 1 4 5135 MPI_Finalize()
0.0
4977 4977 12 0 415 MPI_Reduce()
0.0
1700 1700 1 0 1700 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
107 107 4 0 27 MPI_Keyval_free()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
62 62 4 0
16 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
48, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 49;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.073E+05
6.119E+08 1 2808
611880884 .TAU application
93.9
7.069E+06 5.745E+08 1000 2008
574534 DO_FULLADVANCE
92.3
2.201E+08 5.65E+08 1000 28000
565031 TIMESTEP_SSP_322
32.8
1.851E+08 2.007E+08 3000 6000 66885 GET_KINETIC_ADVANCE
16.8
1.025E+08 1.025E+08 24000 0 4272 MPI_Alltoall()
6.8
3.658E+07 4.18E+07 1001 2002 41758 GET_FIELD_EXPLICIT
4.3
2.61E+07 2.61E+07 10979 0 2378 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16199591 MAKE_LAMBDA_GRID
1.4
6.973E+06 8.273E+06 1 770
8273013 MAKE_MAXWELL_MATRIX
0.7
3.96E+06 4.33E+06 1 130
4330260 MAKE_POISSON_MATRIX
0.7 4.253E+06 4.253E+06 1122 0 3791 MPI_Bcast()
0.3
1.454E+05 2.033E+06 1000 1000 2033 GET_ERROR
0.3
7.334E+04 1.571E+06 1000 1000 1571 CATCH_HALT_SIGNAL
0.1
7.006E+05 7.006E+05 1 0
700610 MAKE_THETA_OPERATORS
0.1
1.576E+05 5.685E+05 1 257
568499 MAKE_BLEND_ARRAYS
0.1
2.388E+05 5.062E+05 10 160 50616 WRITE_BIG
0.1
4.019E+05 4.019E+05 1 0
401906 MAKE_RADIAL_OPERATORS
0.1
3.416E+05 3.74E+05 1 6
374001 MAKE_IMPLICIT_ADVECT
0.0
400 1.365E+05 1 2
136531 WRITE_INPUT
0.0
7.039E+04 7.06E+04 1 8 70598 MPI_Init()
0.0
3.948E+04 3.948E+04 2 0 19738 MPI_Comm_split()
0.0
4808 4903 1 4 4903 MPI_Finalize()
0.0
2920 2920 12 0 243 MPI_Reduce()
0.0
1730 1730 1 0 1730 MAKE_PROFILES
0.0
107 107 4 0 27 MPI_Attr_put()
0.0
98 98
4 0 24 MPI_Attr_get()
0.0
95 95 4 0 24 MPI_Keyval_free()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
49, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 50;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.943E+05
6.064E+08 1 2808
606398297 .TAU application
94.0
7.068E+06 5.702E+08 1000 2008
570166 DO_FULLADVANCE
92.5
2.195E+08 5.607E+08 1000 28000
560724 TIMESTEP_SSP_322
32.9
1.844E+08 1.994E+08 3000 6000 66451 GET_KINETIC_ADVANCE
16.6
1.004E+08 1.004E+08 24000 0 4184 MPI_Alltoall()
6.8
3.638E+07 4.148E+07 1001 2002 41437 GET_FIELD_EXPLICIT
4.2
2.535E+07 2.535E+07 10979 0 2309 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16172410 MAKE_LAMBDA_GRID
1.4 6.958E+06 8.29E+06 1 770
8289850 MAKE_MAXWELL_MATRIX
0.7
3.937E+06 4.342E+06 1 130
4342034 MAKE_POISSON_MATRIX
0.5
3.24E+06 3.24E+06 1122 0 2888 MPI_Bcast()
0.3
1.46E+05 1.997E+06 1000 1000 1997 GET_ERROR
0.2
7.321E+04 1.508E+06 1000 1000 1508 CATCH_HALT_SIGNAL
0.1
6.93E+05 6.93E+05 1 0
693044 MAKE_THETA_OPERATORS
0.1
1.573E+05 5.631E+05 1 257
563110 MAKE_BLEND_ARRAYS
0.1
2.271E+05 4.758E+05 10 160 47579 WRITE_BIG
0.1
3.989E+05 3.989E+05 1 0
398936 MAKE_RADIAL_OPERATORS
0.1
3.44E+05 3.75E+05 1 6
374961 MAKE_IMPLICIT_ADVECT
0.0
397 8.829E+04 1 2 88290 WRITE_INPUT
0.0
7.04E+04 7.061E+04 1 8 70612 MPI_Init()
0.0
2.929E+04 2.929E+04 2 0 14644 MPI_Comm_split()
0.0
4755 4860 1 4 4860 MPI_Finalize()
0.0
3773 3773 12 0 314 MPI_Reduce()
0.0
1719 1719 1
0 1719 MAKE_PROFILES
0.0
110 110 4 0 28 MPI_Attr_put()
0.0
105 105 4 0 26 MPI_Keyval_free()
0.0
102 102 4 0
26 MPI_Attr_get()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
50, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345
Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 51;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.925E+05
6.15E+08 1 2808
614983597 .TAU application
93.9
7.072E+06 5.777E+08 1000 2008
577663 DO_FULLADVANCE
92.3
2.225E+08 5.679E+08 1000 28000
567929 TIMESTEP_SSP_322
32.7
1.852E+08 2.012E+08 3000 6000 67074 GET_KINETIC_ADVANCE
16.6
1.023E+08 1.023E+08 24000 0 4261 MPI_Alltoall()
6.8
3.661E+07 4.199E+07 1001 2002 41948 GET_FIELD_EXPLICIT
4.4
2.697E+07 2.697E+07 10979 0 2456 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200302 MAKE_LAMBDA_GRID
1.3
6.973E+06 8.286E+06 1 770
8286308 MAKE_MAXWELL_MATRIX
0.7
3.956E+06 4.324E+06 1 130
4323655 MAKE_POISSON_MATRIX
0.7
4.255E+06 4.255E+06 1122 0 3792 MPI_Bcast()
0.4
1.421E+05 2.255E+06 1000 1000 2255 GET_ERROR
0.3
7.18E+04 1.574E+06 1000 1000 1574 CATCH_HALT_SIGNAL
0.1
7.011E+05 7.011E+05 1 0
701055 MAKE_THETA_OPERATORS
0.1 1.582E+05
5.706E+05 1 257
570590 MAKE_BLEND_ARRAYS
0.1
2.374E+05 5.115E+05 10 160 51147 WRITE_BIG
0.1
4.027E+05 4.027E+05 1 0
402744 MAKE_RADIAL_OPERATORS
0.1
3.439E+05 3.759E+05 1 6
375870 MAKE_IMPLICIT_ADVECT
0.0
415 1.357E+05 1 2
135744 WRITE_INPUT
0.0
6.824E+04 6.845E+04 1 8 68447 MPI_Init()
0.0
3.961E+04 3.961E+04 2 0 19806 MPI_Comm_split()
0.0
5121 5233 1 4 5233 MPI_Finalize()
0.0
2611 2611 12 0 218 MPI_Reduce()
0.0
1691 1691 1 0 1691 MAKE_PROFILES
0.0
112 112 4 0 28 MPI_Keyval_free()
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
51, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04 4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 52;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.061E+05
6.134E+08 1 2808
613388414 .TAU application
93.9
7.067E+06 5.76E+08 1000 2008
575964 DO_FULLADVANCE
92.3
2.204E+08 5.664E+08 1000 28000
566407 TIMESTEP_SSP_322
32.6 1.854E+08 1.999E+08 3000 6000 66623 GET_KINETIC_ADVANCE
17.1
1.047E+08 1.047E+08 24000 0 4362 MPI_Alltoall()
6.8
3.666E+07 4.153E+07 1001 2002 41493 GET_FIELD_EXPLICIT
4.0
2.477E+07 2.477E+07 10979 0 2256 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200562 MAKE_LAMBDA_GRID
1.4
6.967E+06 8.288E+06 1 770
8288162 MAKE_MAXWELL_MATRIX
0.7
3.946E+06 4.322E+06 1 130
4322458 MAKE_POISSON_MATRIX
0.7
4.197E+06 4.197E+06 1122 0 3740 MPI_Bcast()
0.3
1.426E+05 2.059E+06 1000 1000 2059 GET_ERROR
0.2
7.765E+04 1.525E+06 1000 1000 1525 CATCH_HALT_SIGNAL
0.1
7E+05 7E+05 1 0
699991 MAKE_THETA_OPERATORS
0.1
1.586E+05 5.684E+05 1 257
568400 MAKE_BLEND_ARRAYS
0.1
2.415E+05 5.376E+05 10 160 53756 WRITE_BIG
0.1
4.376E+05 4.686E+05 1 6
468570 MAKE_IMPLICIT_ADVECT
0.1
4.39E+05 4.39E+05 1 0
439046 MAKE_RADIAL_OPERATORS
0.0
400 1.366E+05 1 2
136581 WRITE_INPUT
0.0
7.072E+04 7.093E+04 1 8 70929 MPI_Init()
0.0
3.98E+04 3.98E+04 2 0 19902 MPI_Comm_split()
0.0
1.854E+04 1.854E+04 12 0 1545 MPI_Reduce()
0.0
4827 4932 1 4 4932 MPI_Finalize()
0.0
1768 1768 1 0 1768 MAKE_PROFILES
0.0
106 106 4 0 26 MPI_Attr_put()
0.0
105 105 4 0 26 MPI_Keyval_free()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
66 66 4 0 16 MPI_Comm_rank()
0.0
29 29 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
52, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 53;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.051E+05
6.133E+08 1 2808
613263447 .TAU application
94.1
7.07E+06 5.769E+08 1000 2008
576897 DO_FULLADVANCE
92.5
2.238E+08 5.674E+08 1000 28000
567375 TIMESTEP_SSP_322
32.5 1.847E+08
1.994E+08 3000 6000 66483 GET_KINETIC_ADVANCE
16.8
1.027E+08 1.027E+08 24000 0 4281 MPI_Alltoall()
6.7
3.641E+07 4.138E+07 1001 2002 41342 GET_FIELD_EXPLICIT
4.1 2.499E+07
2.499E+07 10979 0 2276 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16172690 MAKE_LAMBDA_GRID
1.3
6.963E+06 8.257E+06 1 770
8256955 MAKE_MAXWELL_MATRIX
0.7 3.931E+06 4.302E+06 1 130
4302147 MAKE_POISSON_MATRIX
0.5
3.317E+06 3.317E+06 1122 0 2956 MPI_Bcast()
0.3
1.403E+05 2.067E+06 1000 1000 2067 GET_ERROR
0.2
7.654E+04 1.527E+06 1000 1000 1527 CATCH_HALT_SIGNAL
0.1
6.922E+05 6.922E+05 1 0
692244 MAKE_THETA_OPERATORS
0.1
1.572E+05 5.607E+05 1 257
560736 MAKE_BLEND_ARRAYS
0.1
2.33E+05 4.856E+05 10 160 48563 WRITE_BIG
0.1
4.379E+05 4.685E+05 1 6
468499 MAKE_IMPLICIT_ADVECT
0.1
4.356E+05 4.356E+05 1 0
435578 MAKE_RADIAL_OPERATORS
0.0
381 8.806E+04 1 2 88063 WRITE_INPUT
0.0
6.826E+04 6.847E+04 1 8 68471 MPI_Init()
0.0
2.967E+04 2.967E+04 2 0 14833 MPI_Comm_split()
0.0
5099 5212 1
4 5212 MPI_Finalize()
0.0
2498 2498 12 0 208 MPI_Reduce()
0.0
1700 1700 1 0 1700 MAKE_PROFILES
0.0
113 113 4 0
28 MPI_Keyval_free()
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
61 61 4 0
15 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
53, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 54;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.087E+05
6.154E+08 1 2808
615411633 .TAU application
94.1
7.07E+06 5.791E+08 1000 2008
579068 DO_FULLADVANCE
92.0
2.229E+08 5.663E+08 1000 28000
566288 TIMESTEP_SSP_322
32.4
1.846E+08 1.993E+08 3000 6000 66441 GET_KINETIC_ADVANCE
16.7
1.028E+08 1.028E+08 24000 0 4282 MPI_Alltoall()
6.7
3.641E+07 4.132E+07 1001 2002 41284 GET_FIELD_EXPLICIT
4.6
2.818E+07 2.818E+07 10979 0 2567 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16172840 MAKE_LAMBDA_GRID
1.3
6.972E+06 8.263E+06 1 770
8262523 MAKE_MAXWELL_MATRIX
0.9
1.402E+05 5.324E+06 1000 1000 5324 GET_ERROR
0.7
3.938E+06 4.329E+06 1 130
4328655 MAKE_POISSON_MATRIX
0.5
3.243E+06 3.243E+06 1122 0 2890 MPI_Bcast()
0.2
7.613E+04 1.512E+06 1000 1000 1512 CATCH_HALT_SIGNAL
0.1
6.929E+05 6.929E+05 1 0
692917 MAKE_THETA_OPERATORS
0.1
1.584E+05 5.616E+05 1 257
561605 MAKE_BLEND_ARRAYS
0.1
2.344E+05 4.868E+05 10 160 48680 WRITE_BIG
0.1
4.405E+05 4.712E+05 1 6
471243 MAKE_IMPLICIT_ADVECT
0.1
4.357E+05 4.357E+05 1 0
435666 MAKE_RADIAL_OPERATORS
0.0
393 8.9E+04 1 2 88998 WRITE_INPUT
0.0
6.813E+04 6.833E+04 1 8 68332 MPI_Init()
0.0
2.948E+04 2.948E+04 2 0 14742 MPI_Comm_split()
0.0
5065 5166 1 4 5166 MPI_Finalize()
0.0
3563 3563
12 0 297 MPI_Reduce()
0.0
1719 1719 1 0 1719 MAKE_PROFILES
0.0
107 107 4 0 27 MPI_Attr_put()
0.0
101 101 4
0 25
MPI_Keyval_free()
0.0
95 95 4 0 24 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
29 29 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
54, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4 4.232
0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 55;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.047E+05
6.126E+08 1 2808
612556062 .TAU application
93.9
7.07E+06 5.75E+08 1000 2008
575041 DO_FULLADVANCE
92.3
2.209E+08 5.654E+08 1000 28000
565434 TIMESTEP_SSP_322
32.6
1.854E+08 1.997E+08 3000 6000 66566 GET_KINETIC_ADVANCE
16.9
1.034E+08 1.034E+08 24000 0 4307 MPI_Alltoall()
6.8
3.666E+07 4.148E+07 1001 2002 41436 GET_FIELD_EXPLICIT
4.0
2.464E+07 2.464E+07 10979 0 2244 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200179 MAKE_LAMBDA_GRID
1.3 6.982E+06
8.268E+06 1 770
8268234 MAKE_MAXWELL_MATRIX
0.7
3.955E+06 4.379E+06 1 130
4378983 MAKE_POISSON_MATRIX
0.7
4.255E+06 4.255E+06 1122 0 3793 MPI_Bcast()
0.3 1.436E+05 2.137E+06 1000 1000 2137 GET_ERROR
0.3
7.754E+04 1.546E+06 1000 1000 1546 CATCH_HALT_SIGNAL
0.1
7.017E+05 7.017E+05 1 0
701713 MAKE_THETA_OPERATORS
0.1
1.571E+05 5.676E+05 1 257
567629 MAKE_BLEND_ARRAYS
0.1
2.438E+05 5.049E+05 10 160 50488 WRITE_BIG
0.1
4.366E+05 4.677E+05 1 6
467691 MAKE_IMPLICIT_ADVECT
0.1
4.398E+05 4.398E+05 1 0
439847 MAKE_RADIAL_OPERATORS
0.0
394 1.381E+05 1 2
138068 WRITE_INPUT
0.0
7.094E+04 7.115E+04 1 8 71146 MPI_Init()
0.0
4.05E+04 4.05E+04 2 0 20252 MPI_Comm_split()
0.0
4861 4966 1 4 4966 MPI_Finalize()
0.0
2671 2671 12 0 223 MPI_Reduce()
0.0
1707 1707 1 0 1707 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
105 105 4 0 26 MPI_Keyval_free()
0.0
101 101 4 0 25 MPI_Attr_get()
0.0
71 71 4 0 18 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
55, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev. Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 56;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.066E+05
6.094E+08 1 2808
609376142 .TAU application
94.1
7.068E+06 5.731E+08 1000 2008
573128 DO_FULLADVANCE
92.5
2.197E+08 5.635E+08 1000 28000
563520 TIMESTEP_SSP_322
32.7
1.845E+08 1.995E+08 3000 6000 66496 GET_KINETIC_ADVANCE
16.9
1.029E+08 1.029E+08 24000 0 4288 MPI_Alltoall()
6.8
3.636E+07 4.142E+07 1001 2002 41374 GET_FIELD_EXPLICIT
4.2
2.549E+07 2.549E+07 10979 0 2322 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16171366 MAKE_LAMBDA_GRID
1.4
6.959E+06 8.262E+06 1 770
8261816 MAKE_MAXWELL_MATRIX
0.7
3.942E+06 4.306E+06 1 130
4306210 MAKE_POISSON_MATRIX
0.5
3.297E+06 3.297E+06 1122 0 2939 MPI_Bcast()
0.4
1.411E+05 2.151E+06 1000 1000 2151 GET_ERROR
0.3
7.234E+04 1.529E+06 1000 1000 1529 CATCH_HALT_SIGNAL
0.1
6.924E+05 6.924E+05 1 0
692383 MAKE_THETA_OPERATORS
0.1
1.576E+05 5.687E+05 1 257
568661 MAKE_BLEND_ARRAYS
0.1
2.33E+05 4.898E+05 10 160 48985 WRITE_BIG
0.1
3.974E+05 3.974E+05 1 0
397447 MAKE_RADIAL_OPERATORS
0.1
3.4E+05 3.715E+05 1 6
371490 MAKE_IMPLICIT_ADVECT
0.0
384 9.095E+04 1 2 90947 WRITE_INPUT
0.0
6.786E+04 6.807E+04 1 8 68067 MPI_Init()
0.0
2.957E+04 2.957E+04 2 0 14784 MPI_Comm_split()
0.0
5065 5169 1 4 5169 MPI_Finalize()
0.0
4687 4687 12 0 391 MPI_Reduce()
0.0
1663 1663 1 0 1663 MAKE_PROFILES
0.0
109 109 4
0 27 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Keyval_free()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
57 57 4 0
14 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
56, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 57;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.945E+05
6.153E+08 1 2808
615292554 .TAU application
93.9
7.068E+06 5.78E+08 1000 2008
577991 DO_FULLADVANCE
92.4
2.221E+08 5.685E+08 1000 28000
568457 TIMESTEP_SSP_322
32.5
1.851E+08 1.998E+08 3000 6000 66606 GET_KINETIC_ADVANCE
17.1
1.051E+08 1.051E+08 24000 0 4381 MPI_Alltoall()
6.7
3.658E+07 4.148E+07 1001 2002 41434 GET_FIELD_EXPLICIT
4.1
2.494E+07 2.494E+07 10979 0 2271 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200559 MAKE_LAMBDA_GRID
1.3
6.976E+06 8.248E+06 1 770
8247711 MAKE_MAXWELL_MATRIX
0.7
3.958E+06 4.33E+06 1 130
4329837 MAKE_POISSON_MATRIX
0.7
4.252E+06 4.252E+06 1122 0 3789 MPI_Bcast()
0.3
1.419E+05 2.073E+06 1000 1000 2073 GET_ERROR
0.3
7.316E+04 1.541E+06 1000 1000 1541 CATCH_HALT_SIGNAL
0.1
7.007E+05 7.007E+05 1 0
700719 MAKE_THETA_OPERATORS
0.1
1.59E+05 5.663E+05 1 257
566299 MAKE_BLEND_ARRAYS
0.1
2.365E+05 4.994E+05 10 160 49939 WRITE_BIG
0.1 4.026E+05
4.026E+05 1 0
402582 MAKE_RADIAL_OPERATORS
0.1
3.393E+05 3.712E+05 1 6
371212 MAKE_IMPLICIT_ADVECT
0.0
399 1.388E+05 1 2
138842 WRITE_INPUT
0.0 7.128E+04
7.149E+04 1 8 71493 MPI_Init()
0.0
4.034E+04 4.034E+04 2 0 20170 MPI_Comm_split()
0.0
4880 4990 1 4 4990 MPI_Finalize()
0.0
2973 2973 12 0 248 MPI_Reduce()
0.0
1699 1699 1 0 1699 MAKE_PROFILES
0.0
110 110 4 0 28 MPI_Keyval_free()
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
64 64 4 0 16 MPI_Comm_rank()
0.0
28 28 2
0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
57, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 58;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.07E+05
6.095E+08 1 2808
609511625 .TAU application
94.0
7.067E+06 5.732E+08 1000 2008
573220 DO_FULLADVANCE
92.5
2.197E+08 5.637E+08 1000 28000
563669 TIMESTEP_SSP_322
32.8
1.844E+08 2.002E+08 3000 6000 66740 GET_KINETIC_ADVANCE
16.8
1.022E+08 1.022E+08 24000 0 4260 MPI_Alltoall()
6.8
3.638E+07 4.154E+07 1001 2002 41497 GET_FIELD_EXPLICIT
4.3
2.632E+07 2.632E+07 10979 0 2398 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16171477 MAKE_LAMBDA_GRID
1.4
6.956E+06 8.252E+06 1 770
8251732 MAKE_MAXWELL_MATRIX
0.7
3.941E+06 4.344E+06 1 130
4344277 MAKE_POISSON_MATRIX
0.5
3.322E+06 3.322E+06 1122 0 2961 MPI_Bcast()
0.3
1.396E+05 2.103E+06 1000 1000 2103 GET_ERROR
0.2
7.137E+04 1.524E+06 1000 1000 1524 CATCH_HALT_SIGNAL
0.1
6.928E+05 6.928E+05 1 0
692844 MAKE_THETA_OPERATORS
0.1
1.572E+05 5.627E+05 1 257
562734 MAKE_BLEND_ARRAYS
0.1
2.34E+05 4.829E+05 10 160 48286 WRITE_BIG
0.1
3.993E+05 3.993E+05 1 0
399281 MAKE_RADIAL_OPERATORS
0.1
3.379E+05 3.69E+05 1 6
368961 MAKE_IMPLICIT_ADVECT
0.0
408 9.062E+04 1 2 90620 WRITE_INPUT
0.0
7.096E+04 7.117E+04 1 8 71171 MPI_Init()
0.0
3.079E+04 3.079E+04 2
0 15396 MPI_Comm_split()
0.0
4685 4783 1 4 4783 MPI_Finalize()
0.0
3553 3553 12 0 296 MPI_Reduce()
0.0
1714 1714 1 0 1714 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Attr_get()
0.0
98 98 4 0 24 MPI_Keyval_free()
0.0
59 59 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
58, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 59;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.995E+05
6.114E+08 1 2808
611406776 .TAU application
93.9
7.064E+06 5.741E+08 1000 2008
574064 DO_FULLADVANCE
92.4
2.206E+08 5.646E+08 1000 28000
564635 TIMESTEP_SSP_322
32.8
1.852E+08 2.007E+08 3000 6000 66898 GET_KINETIC_ADVANCE
16.5
1.007E+08 1.007E+08 24000 0 4195 MPI_Alltoall()
7.0
3.663E+07 4.275E+07 1001 2002 42706 GET_FIELD_EXPLICIT
4.4
2.693E+07 2.693E+07 10979 0 2453 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200598 MAKE_LAMBDA_GRID
1.4 6.973E+06
8.282E+06 1 770
8281786 MAKE_MAXWELL_MATRIX
0.7
3.952E+06 4.351E+06 1 130
4350633 MAKE_POISSON_MATRIX
0.7
4.239E+06 4.239E+06 1122 0 3778 MPI_Bcast()
0.3 1.408E+05 1.967E+06 1000 1000 1967 GET_ERROR
0.2
7.067E+04 1.526E+06 1000 1000 1526 CATCH_HALT_SIGNAL
0.1
7.008E+05 7.008E+05 1 0
700803 MAKE_THETA_OPERATORS
0.1
1.589E+05 5.667E+05 1 257
566704 MAKE_BLEND_ARRAYS
0.1
2.344E+05 5.026E+05 10 160 50262 WRITE_BIG
0.1
4.029E+05 4.029E+05 1 0
402891 MAKE_RADIAL_OPERATORS
0.1
3.468E+05 3.785E+05 1 6
378497 MAKE_IMPLICIT_ADVECT
0.0
397 1.376E+05 1 2
137608 WRITE_INPUT
0.0
6.848E+04 6.868E+04 1 8 68682 MPI_Init()
0.0
4.037E+04 4.037E+04 2 0 20184 MPI_Comm_split()
0.0
5037 5140 1 4 5140 MPI_Finalize()
0.0
2709 2709 12 0 226 MPI_Reduce()
0.0
1701 1701 1 0 1701 MAKE_PROFILES
0.0
105 105 4 0 26 MPI_Attr_put()
0.0
103 103 4 0 26 MPI_Keyval_free()
0.0
99 99 4 0
25 MPI_Attr_get()
0.0
66 66 4 0 16 MPI_Comm_rank()
0.0
30 30 2 0 15 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
59, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 60;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.035E+05
6.076E+08 1 2808
607585401 .TAU application
94.0
7.072E+06 5.712E+08 1000 2008
571205 DO_FULLADVANCE
92.4
2.204E+08 5.617E+08 1000 28000
561661 TIMESTEP_SSP_322
32.8
1.846E+08 1.992E+08 3000 6000 66398 GET_KINETIC_ADVANCE
16.6
1.006E+08 1.006E+08 24000 0 4193 MPI_Alltoall()
6.8
3.644E+07 4.148E+07 1001 2002 41442 GET_FIELD_EXPLICIT
4.1
2.499E+07 2.499E+07 10979 0 2276 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16172584 MAKE_LAMBDA_GRID
1.4
6.948E+06 8.234E+06 1 770
8234286 MAKE_MAXWELL_MATRIX
0.7
3.926E+06 4.302E+06 1 130
4301796 MAKE_POISSON_MATRIX
0.6
3.345E+06 3.345E+06 1122 0 2982 MPI_Bcast()
0.3
1.455E+05 2.09E+06 1000 1000 2090 GET_ERROR
0.2
7.632E+04 1.509E+06 1000 1000 1509 CATCH_HALT_SIGNAL
0.1
6.926E+05 6.926E+05 1 0
692648 MAKE_THETA_OPERATORS
0.1
1.575E+05 5.645E+05 1 257
564485 MAKE_BLEND_ARRAYS
0.1
2.328E+05 4.835E+05 10 160 48355 WRITE_BIG
0.1
4.337E+05 4.645E+05 1 6
464462 MAKE_IMPLICIT_ADVECT
0.1
4.356E+05 4.356E+05 1 0
435557 MAKE_RADIAL_OPERATORS
0.0
368 9.311E+04 1 2 93108 WRITE_INPUT
0.0
7.054E+04 7.075E+04 1 8 70752 MPI_Init()
0.0
2.936E+04 2.936E+04 2 0 14678 MPI_Comm_split()
0.0
4792 4912 1 4 4912 MPI_Finalize()
0.0
4245 4245 12 0 354 MPI_Reduce()
0.0
1647 1647 1 0 1647 MAKE_PROFILES
0.0
120 120 4 0 30 MPI_Keyval_free()
0.0
110 110 4 0 28 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Attr_get()
0.0
60 60 4
0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
60, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 61;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.035E+05
6.13E+08 1 2808
613010245 .TAU application
93.9 7.063E+06
5.756E+08 1000 2008
575640 DO_FULLADVANCE
92.4
2.211E+08 5.662E+08 1000 28000
566184 TIMESTEP_SSP_322
32.6
1.854E+08 1.998E+08 3000 6000 66584 GET_KINETIC_ADVANCE
16.9 1.037E+08 1.037E+08 24000 0 4322 MPI_Alltoall()
6.8
3.662E+07 4.16E+07 1001 2002 41562 GET_FIELD_EXPLICIT
4.0
2.454E+07 2.454E+07 10979 0 2235 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200913 MAKE_LAMBDA_GRID
1.3
6.973E+06 8.229E+06 1 770
8228557 MAKE_MAXWELL_MATRIX
0.7
3.952E+06 4.314E+06 1 130
4313785 MAKE_POISSON_MATRIX
0.7
4.241E+06 4.241E+06 1122 0 3780 MPI_Bcast()
0.3
1.416E+05 1.999E+06 1000 1000 1999 GET_ERROR
0.2
7.802E+04 1.531E+06 1000 1000 1531 CATCH_HALT_SIGNAL
0.1
7.01E+05 7.01E+05 1 0
701028 MAKE_THETA_OPERATORS
0.1
1.586E+05 5.59E+05 1 257
559017 MAKE_BLEND_ARRAYS
0.1
2.41E+05 5.008E+05 10 160 50077 WRITE_BIG
0.1
4.381E+05 4.693E+05 1 6
469323 MAKE_IMPLICIT_ADVECT
0.1
4.394E+05 4.394E+05 1 0
439353 MAKE_RADIAL_OPERATORS
0.0
417 1.38E+05 1 2
138034 WRITE_INPUT
0.0
6.776E+04 6.8E+04 1 8 67997 MPI_Init()
0.0
3.937E+04 3.937E+04 2 0 19683 MPI_Comm_split()
0.0
5052 5155 1 4 5155 MPI_Finalize()
0.0
2767 2767 12 0 231 MPI_Reduce()
0.0
1805 1805 1 0 1805 MAKE_PROFILES
0.0
127 127 4 0 32 MPI_Attr_put()
0.0
110 110 4 0 28 MPI_Attr_get()
0.0
103 103 4 0 26 MPI_Keyval_free()
0.0
62 62 4 0 16 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
61, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 62;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.064E+05
6.122E+08 1 2808
612172875 .TAU application
93.9
7.072E+06 5.747E+08 1000 2008
574700 DO_FULLADVANCE
92.3
2.201E+08 5.65E+08 1000 28000
564990 TIMESTEP_SSP_322
32.8
1.852E+08 2.008E+08 3000 6000 66918 GET_KINETIC_ADVANCE
16.7
1.02E+08 1.02E+08 24000 0 4249 MPI_Alltoall()
6.9
3.666E+07 4.222E+07 1001 2002 42174 GET_FIELD_EXPLICIT
4.4
2.67E+07 2.67E+07 10979 0 2432 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200187 MAKE_LAMBDA_GRID
1.4
6.977E+06 8.265E+06 1 770
8265249 MAKE_MAXWELL_MATRIX
0.7
3.953E+06 4.339E+06 1 130
4339491 MAKE_POISSON_MATRIX
0.7
4.259E+06 4.259E+06 1122 0 3796 MPI_Bcast()
0.4
1.418E+05 2.237E+06 1000 1000 2237 GET_ERROR
0.3
7.815E+04 1.547E+06 1000 1000 1547 CATCH_HALT_SIGNAL
0.1
7.014E+05 7.014E+05 1 0
701445 MAKE_THETA_OPERATORS
0.1
1.581E+05 5.682E+05 1 257
568188 MAKE_BLEND_ARRAYS
0.1
2.401E+05 5.071E+05 10 160 50706 WRITE_BIG
0.1
4.352E+05 4.666E+05 1 6
466587 MAKE_IMPLICIT_ADVECT
0.1
4.401E+05 4.401E+05 1 0
440112 MAKE_RADIAL_OPERATORS
0.0
384 1.386E+05 1 2
138568 WRITE_INPUT
0.0
7.073E+04 7.093E+04 1 8 70934 MPI_Init()
0.0
4.037E+04 4.037E+04 2 0 20184 MPI_Comm_split()
0.0
4753 4865 1 4 4865 MPI_Finalize()
0.0
3690 3690 12
0 308 MPI_Reduce()
0.0
1713 1713 1 0 1713 MAKE_PROFILES
0.0
112 112 4 0 28 MPI_Keyval_free()
0.0
109 109 4
0 27 MPI_Attr_put()
0.0
96 96 4 0 24 MPI_Attr_get()
0.0
63 63 4 0 16 MPI_Comm_rank()
0.0
28 28 2 0
14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
62, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0
0 0 Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 63;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.049E+05
6.078E+08 1 2808
607808974 .TAU application
94.0
7.07E+06 5.715E+08 1000 2008
571461 DO_FULLADVANCE
92.5
2.208E+08 5.62E+08 1000 28000
561967 TIMESTEP_SSP_322
32.7
1.846E+08 1.989E+08 3000 6000 66315 GET_KINETIC_ADVANCE
16.6
1.009E+08 1.009E+08 24000 0 4203 MPI_Alltoall()
6.8
3.642E+07 4.144E+07 1001
2002 41394
GET_FIELD_EXPLICIT
4.1
2.467E+07 2.467E+07 10979 0 2247 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16171915 MAKE_LAMBDA_GRID
1.4
6.959E+06 8.255E+06 1
770 8255070
MAKE_MAXWELL_MATRIX
0.7
3.939E+06 4.329E+06 1 130
4329135 MAKE_POISSON_MATRIX
0.5
3.261E+06 3.261E+06 1122 0 2906 MPI_Bcast()
0.3
1.459E+05 2.028E+06 1000 1000 2028 GET_ERROR
0.2
7.705E+04 1.498E+06 1000 1000 1498 CATCH_HALT_SIGNAL
0.1
6.938E+05 6.938E+05 1 0
693796 MAKE_THETA_OPERATORS
0.1
1.582E+05 5.694E+05 1 257
569429 MAKE_BLEND_ARRAYS
0.1
2.336E+05 4.94E+05 10 160 49395 WRITE_BIG
0.1
4.368E+05 4.673E+05 1 6
467303 MAKE_IMPLICIT_ADVECT
0.1 4.359E+05
4.359E+05 1 0
435912 MAKE_RADIAL_OPERATORS
0.0
404 8.953E+04 1 2 89531 WRITE_INPUT
0.0
6.774E+04 6.795E+04 1 8 67951 MPI_Init()
0.0
2.947E+04 2.947E+04 2 0 14734 MPI_Comm_split()
0.0
4987 5100 1 4 5100 MPI_Finalize()
0.0
2772 2772 12 0 231 MPI_Reduce()
0.0
1733 1733 1 0 1733 MAKE_PROFILES
0.0
113 113 4 0 28 MPI_Keyval_free()
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
103 103 4 0 26 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
63, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 64;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.971E+05
6.142E+08 1 2808
614191265 .TAU application
93.9
7.184E+06 5.768E+08 1000 3008
576786 DO_FULLADVANCE
92.3
2.21E+08 5.667E+08 1000 28000
566729 TIMESTEP_SSP_322
32.5
1.851E+08 1.998E+08 3000 6000 66608 GET_KINETIC_ADVANCE
17.0
1.043E+08 1.043E+08 24000 0 4348 MPI_Alltoall()
6.8
3.661E+07 4.156E+07 1001
2002 41520 GET_FIELD_EXPLICIT
4.1
2.532E+07 2.532E+07 10979 0 2306 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201128 MAKE_LAMBDA_GRID
1.4
6.975E+06 8.309E+06 1 770
8308829 MAKE_MAXWELL_MATRIX
0.7
3.957E+06 4.36E+06 1 130
4360247 MAKE_POISSON_MATRIX
0.7
4.252E+06 4.252E+06 1122 0 3790 MPI_Bcast()
0.4
1.409E+05 2.263E+06 1000 1000 2263 GET_ERROR
0.3
7.027E+04 1.538E+06 1000 1000 1538 CATCH_HALT_SIGNAL
0.1
7.02E+05 7.02E+05 1 0
701990 MAKE_THETA_OPERATORS
0.1
1.575E+05 5.68E+05 1 257
568005 MAKE_BLEND_ARRAYS
0.1
2.37E+05 5.356E+05 10 196 53556 WRITE_BIG
0.1
4.017E+05 4.017E+05 1 0
401687 MAKE_RADIAL_OPERATORS
0.1
3.448E+05 3.767E+05 1 6
376664 MAKE_IMPLICIT_ADVECT
0.0
1.926E+05 1.926E+05 1037 0 186 MPI_Send()
0.0
606 1.357E+05 1 3
135736 WRITE_INPUT
0.0
6.814E+04 6.839E+04 1 8 68390 MPI_Init()
0.0
4.107E+04 4.107E+04 2 0 20537 MPI_Comm_split()
0.0
3.102E+04 3.102E+04 12 0 2585 MPI_Reduce()
0.0
5037 5139 1 4 5139 MPI_Finalize()
0.0
1739 1739 1 0 1739 MAKE_PROFILES
0.0
129 129 4 0 32 MPI_Attr_put()
0.0
116 116 4 0 29 MPI_Attr_get()
0.0
102 102 4 0 26 MPI_Keyval_free()
0.0
82 82 4 0 20 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
64, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
1037
8192 8 111.5
761.7 Message size sent to all
nodes
---------------------------------------------------------------------------------------
NODE 65;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.991E+05
6.106E+08 1 2808
610598396 .TAU application
93.8
7.066E+06 5.729E+08 1000 2008
572890 DO_FULLADVANCE
92.3
2.196E+08 5.634E+08 1000 28000
563418 TIMESTEP_SSP_322
32.7
1.844E+08 1.996E+08 3000 6000 66538 GET_KINETIC_ADVANCE
16.8
1.028E+08 1.028E+08 24000 0 4283 MPI_Alltoall()
6.8
3.637E+07 4.143E+07 1001 2002 41392 GET_FIELD_EXPLICIT
4.2
2.567E+07 2.567E+07 10979 0 2339 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16172781 MAKE_LAMBDA_GRID
1.4
6.961E+06 8.332E+06 1 770
8332353 MAKE_MAXWELL_MATRIX
0.8
4.657E+06 4.657E+06 1122 0 4150 MPI_Bcast()
0.7
3.937E+06 4.33E+06 1 130
4330188 MAKE_POISSON_MATRIX
0.5 7.212E+04
2.847E+06 1000 1000 2847 CATCH_HALT_SIGNAL
0.3
1.465E+05 2.003E+06 1000 1000 2003 GET_ERROR
0.1
6.935E+05 6.935E+05 1 0
693527 MAKE_THETA_OPERATORS
0.1
1.572E+05 5.705E+05 1 257
570518 MAKE_BLEND_ARRAYS
0.1
2.302E+05 5.043E+05 10 160 50434 WRITE_BIG
0.1
3.987E+05 3.987E+05 1 0
398673 MAKE_RADIAL_OPERATORS
0.1
3.414E+05 3.726E+05 1 6
372635 MAKE_IMPLICIT_ADVECT
0.0
413 9.009E+04 1 2 90090 WRITE_INPUT
0.0
7.075E+04 7.096E+04 1 8 70964 MPI_Init()
0.0
3.136E+04 3.136E+04 2 0 15680 MPI_Comm_split()
0.0
4861 4968 1 4 4968 MPI_Finalize()
0.0
4384 4384 12 0 365 MPI_Reduce()
0.0
1703 1703 1
0 1703 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
107 107 4 0 27 MPI_Keyval_free()
0.0
105 105 4
0 26 MPI_Attr_get()
0.0
58 58 4 0 14 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
65, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 66;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.972E+05
6.123E+08 1 2808
612304716 .TAU application
93.9
7.07E+06 5.749E+08 1000 2008
574864 DO_FULLADVANCE
92.3
2.201E+08 5.654E+08 1000 28000
565390 TIMESTEP_SSP_322
32.9
1.85E+08 2.012E+08 3000 6000 67050 GET_KINETIC_ADVANCE
16.7
1.023E+08 1.023E+08 24000 0 4262 MPI_Alltoall()
6.8
3.66E+07 4.185E+07 1001 2002 41807 GET_FIELD_EXPLICIT
4.4
2.683E+07 2.683E+07 10979 0 2444 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201032 MAKE_LAMBDA_GRID
1.4
6.97E+06 8.342E+06 1 770
8341937 MAKE_MAXWELL_MATRIX
0.7
3.956E+06 4.352E+06 1 130
4351635 MAKE_POISSON_MATRIX
0.7
4.261E+06 4.261E+06 1122 0 3797 MPI_Bcast()
0.3
1.442E+05 2.004E+06 1000 1000 2004 GET_ERROR
0.3
7.209E+04 1.542E+06 1000 1000 1542 CATCH_HALT_SIGNAL
0.1
7.013E+05 7.013E+05 1 0
701338 MAKE_THETA_OPERATORS
0.1
1.582E+05 5.677E+05 1 257
567708 MAKE_BLEND_ARRAYS
0.1
2.313E+05 5.051E+05 10 160 50512 WRITE_BIG
0.1
4.014E+05 4.014E+05 1 0
401421 MAKE_RADIAL_OPERATORS
0.1
3.424E+05 3.743E+05 1 6
374299 MAKE_IMPLICIT_ADVECT
0.0
394 1.347E+05 1 2
134733 WRITE_INPUT
0.0
6.878E+04 6.898E+04 1 8 68983 MPI_Init()
0.0
4.085E+04 4.085E+04 2 0 20424 MPI_Comm_split()
0.0
6382 6382 12 0 532 MPI_Reduce()
0.0
4994 5104 1 4 5104 MPI_Finalize()
0.0
1713 1713 1 0 1713 MAKE_PROFILES
0.0
110 110 4 0 28 MPI_Keyval_free()
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
95 95 4 0 24 MPI_Attr_get()
0.0
70 70 4
0 18 MPI_Comm_rank()
0.0
29 29 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
66, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04 4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 67;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.898E+05
6.111E+08 1
2808 611071593 .TAU
application
94.1
7.069E+06 5.748E+08 1000 2008
574836 DO_FULLADVANCE
92.5
2.219E+08 5.653E+08 1000 28000
565306 TIMESTEP_SSP_322
32.7
1.846E+08 1.997E+08 3000
6000 66569 GET_KINETIC_ADVANCE
16.8
1.025E+08 1.025E+08 24000 0 4270 MPI_Alltoall()
6.7
3.636E+07 4.122E+07 1001 2002 41177 GET_FIELD_EXPLICIT
4.2
2.538E+07 2.538E+07 10979
0 2311 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16171891 MAKE_LAMBDA_GRID
1.4
6.956E+06 8.282E+06 1 770
8282347 MAKE_MAXWELL_MATRIX
0.7
3.937E+06 4.325E+06 1 130
4325070 MAKE_POISSON_MATRIX
0.5
3.285E+06 3.285E+06 1122 0 2928 MPI_Bcast()
0.3
1.421E+05 2.077E+06 1000 1000 2077 GET_ERROR
0.2
6.633E+04 1.499E+06 1000 1000 1499 CATCH_HALT_SIGNAL
0.1
6.928E+05 6.928E+05 1 0
692756 MAKE_THETA_OPERATORS
0.1
1.569E+05 5.668E+05 1 257
566776 MAKE_BLEND_ARRAYS
0.1
2.33E+05 4.822E+05 10 160 48220 WRITE_BIG
0.1
3.983E+05 3.983E+05 1 0
398326 MAKE_RADIAL_OPERATORS
0.1
3.461E+05 3.771E+05 1 6
377100 MAKE_IMPLICIT_ADVECT
0.0
406 8.726E+04 1 2 87265 WRITE_INPUT
0.0
7.093E+04 7.114E+04 1 8 71136 MPI_Init()
0.0
2.936E+04 2.936E+04 2 0 14680 MPI_Comm_split()
0.0
4854 4947 1 4 4947 MPI_Finalize()
0.0
4139 4139 12 0 345 MPI_Reduce()
0.0
1731 1731 1 0 1731 MAKE_PROFILES
0.0
105 105 4 0 26 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
93 93 4 0 23 MPI_Keyval_free()
0.0
58 58 4 0 14 MPI_Comm_rank()
0.0 26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
67, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 68;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.09E+05
6.147E+08 1 2808
614685344 .TAU application
93.9
7.069E+06 5.772E+08 1000 2008
577200 DO_FULLADVANCE
92.3
2.213E+08 5.676E+08 1000 28000
567593 TIMESTEP_SSP_322
32.6
1.853E+08 2.005E+08 3000 6000 66847 GET_KINETIC_ADVANCE
16.9
1.041E+08 1.041E+08 24000 0 4337 MPI_Alltoall()
6.8
3.664E+07 4.173E+07 1001 2002 41688 GET_FIELD_EXPLICIT
4.2
2.586E+07 2.586E+07 10979 0 2356 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200159 MAKE_LAMBDA_GRID
1.3
6.968E+06 8.289E+06 1 770
8289029 MAKE_MAXWELL_MATRIX
0.7
3.946E+06 4.345E+06 1 130
4344908 MAKE_POISSON_MATRIX
0.7
4.215E+06 4.215E+06 1122 0 3757 MPI_Bcast()
0.3
1.44E+05 2.137E+06 1000 1000 2137 GET_ERROR
0.2
7.834E+04 1.511E+06 1000 1000 1511 CATCH_HALT_SIGNAL
0.1
7.022E+05 7.022E+05 1 0
702207 MAKE_THETA_OPERATORS
0.1
1.579E+05 5.704E+05 1 257
570424 MAKE_BLEND_ARRAYS
0.1
2.387E+05 5.058E+05 10 160 50583 WRITE_BIG
0.1
4.366E+05 4.68E+05 1 6
467973 MAKE_IMPLICIT_ADVECT
0.1
4.376E+05 4.376E+05 1 0
437573 MAKE_RADIAL_OPERATORS
0.0
402 1.357E+05 1 2
135671 WRITE_INPUT
0.0
6.86E+04 6.881E+04 1 8 68807 MPI_Init()
0.0
4.09E+04 4.09E+04 2 0 20452 MPI_Comm_split()
0.0
7492 7492 12 0 624 MPI_Reduce()
0.0
5112 5216 1
4 5216 MPI_Finalize()
0.0
1696 1696 1 0 1696 MAKE_PROFILES
0.0
106 106 4 0 26 MPI_Attr_put()
0.0
104 104 4 0
26 MPI_Keyval_free()
0.0
100 100 4 0 25 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
26 26 2 0
13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
68, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0
0 0 Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 69;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.094E+05
6.134E+08 1 2808
613391061 .TAU application
94.1
7.07E+06 5.771E+08 1000 2008
577070 DO_FULLADVANCE
92.5
2.23E+08 5.676E+08 1000 28000
567596 TIMESTEP_SSP_322
32.5
1.846E+08 1.993E+08 3000 6000 66417 GET_KINETIC_ADVANCE
16.9
1.039E+08 1.039E+08 24000 0 4331 MPI_Alltoall()
6.7
3.639E+07 4.14E+07 1001
2002 41358
GET_FIELD_EXPLICIT
4.1
2.49E+07 2.49E+07 10979 0 2268 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16172748 MAKE_LAMBDA_GRID
1.3
6.959E+06 8.222E+06 1
770 8222307
MAKE_MAXWELL_MATRIX
0.7
3.93E+06 4.313E+06 1 130
4312916 MAKE_POISSON_MATRIX
0.5
3.275E+06 3.275E+06 1122 0 2918 MPI_Bcast()
0.3
1.451E+05 2.024E+06 1000 1000 2024 GET_ERROR
0.2
7.685E+04 1.515E+06 1000 1000 1515 CATCH_HALT_SIGNAL
0.1
6.936E+05 6.936E+05 1 0
693612 MAKE_THETA_OPERATORS
0.1
1.572E+05 5.674E+05 1 257
567449 MAKE_BLEND_ARRAYS
0.1
2.344E+05 4.807E+05 10 160 48075 WRITE_BIG
0.1
4.399E+05 4.703E+05 1 6
470290 MAKE_IMPLICIT_ADVECT
0.1 4.342E+05
4.342E+05 1 0
434213 MAKE_RADIAL_OPERATORS
0.0
394 8.818E+04 1 2 88183 WRITE_INPUT
0.0
7.053E+04 7.074E+04 1 8 70735 MPI_Init()
0.0
2.99E+04 2.99E+04 2 0 14948 MPI_Comm_split()
0.0
4762 4869 1 4 4869 MPI_Finalize()
0.0
4160 4160 12 0 347 MPI_Reduce()
0.0
1734 1734 1 0 1734 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
107 107 4 0 27 MPI_Keyval_free()
0.0
100 100 4 0 25 MPI_Attr_get()
0.0
57 57 4 0 14 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
69, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 70;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.097E+05
6.15E+08 1 2808
614984751 .TAU application
93.9
7.069E+06 5.775E+08 1000 2008
577500 DO_FULLADVANCE
92.3
2.211E+08 5.679E+08 1000 28000
567914 TIMESTEP_SSP_322
32.6
1.854E+08 2.005E+08 3000 6000 66818 GET_KINETIC_ADVANCE
17.0
1.047E+08 1.047E+08 24000 0 4363 MPI_Alltoall()
6.8
3.665E+07 4.166E+07 1001
2002 41623 GET_FIELD_EXPLICIT
4.2
2.554E+07 2.554E+07 10979 0 2326 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16199164 MAKE_LAMBDA_GRID
1.3
6.973E+06 8.281E+06 1 770
8281023 MAKE_MAXWELL_MATRIX
0.7
3.957E+06 4.365E+06 1 130
4364962 MAKE_POISSON_MATRIX
0.7
4.218E+06 4.218E+06 1122 0 3760 MPI_Bcast()
0.3
1.436E+05 2.113E+06 1000 1000 2113 GET_ERROR
0.2
7.85E+04 1.519E+06 1000 1000 1519 CATCH_HALT_SIGNAL
0.1
7.001E+05 7.001E+05 1 0
700135 MAKE_THETA_OPERATORS
0.1
1.58E+05 5.683E+05 1 257
568301 MAKE_BLEND_ARRAYS
0.1
2.433E+05 5.108E+05 10 160 51080 WRITE_BIG
0.1
4.36E+05 4.672E+05 1 6
467161 MAKE_IMPLICIT_ADVECT
0.1
4.382E+05 4.382E+05 1 0
438197 MAKE_RADIAL_OPERATORS
0.0
395 1.353E+05 1 2
135333 WRITE_INPUT
0.0
7.035E+04 7.057E+04 1 8 70568 MPI_Init()
0.0
4.143E+04 4.143E+04 2 0 20714 MPI_Comm_split()
0.0
6163 6163 12 0 514 MPI_Reduce()
0.0
4756 4866 1 4 4866 MPI_Finalize()
0.0
1736 1736 1 0 1736 MAKE_PROFILES
0.0
113 113 4 0 28 MPI_Attr_put()
0.0
110 110 4 0 28 MPI_Keyval_free()
0.0
103 103 4 0 26 MPI_Attr_get()
0.0 72 72 4 0 18 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
70, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 71;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.076E+05
6.131E+08 1 2808
613102866 .TAU application
94.0
7.07E+06 5.766E+08 1000 2008
576591 DO_FULLADVANCE
92.5
2.232E+08 5.671E+08 1000 28000
567104 TIMESTEP_SSP_322
32.5
1.847E+08 1.992E+08 3000 6000 66401 GET_KINETIC_ADVANCE
16.9
1.034E+08 1.034E+08 24000 0 4308 MPI_Alltoall()
6.7
3.64E+07 4.13E+07 1001 2002 41263 GET_FIELD_EXPLICIT
4.0
2.478E+07 2.478E+07 10979 0 2257 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16173748 MAKE_LAMBDA_GRID
1.4
6.968E+06 8.289E+06 1 770
8289203 MAKE_MAXWELL_MATRIX
0.7
3.936E+06 4.35E+06 1 130
4350380 MAKE_POISSON_MATRIX
0.5
3.365E+06 3.365E+06 1122 0 2999 MPI_Bcast()
0.3
1.448E+05 2.029E+06 1000 1000 2029 GET_ERROR
0.2
7.696E+04 1.508E+06 1000 1000 1508 CATCH_HALT_SIGNAL
0.1
6.93E+05 6.93E+05 1 0
692980 MAKE_THETA_OPERATORS
0.1
1.564E+05 5.599E+05 1 257
559923 MAKE_BLEND_ARRAYS
0.1
2.318E+05 4.882E+05 10 160 48818 WRITE_BIG
0.1
4.405E+05 4.708E+05 1 6
470829 MAKE_IMPLICIT_ADVECT
0.1
4.35E+05 4.35E+05 1 0
434999 MAKE_RADIAL_OPERATORS
0.0
386 9.013E+04 1 2 90132 WRITE_INPUT
0.0
6.79E+04 6.81E+04 1 8 68104 MPI_Init()
0.0
3.044E+04 3.044E+04 2 0 15218 MPI_Comm_split()
0.0
5115 5229 1 4 5229 MPI_Finalize()
0.0
4345 4345 12 0 362 MPI_Reduce()
0.0
1706 1706 1 0 1706 MAKE_PROFILES
0.0
114 114 4
0 28 MPI_Keyval_free()
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
60 60 4 0
15 MPI_Comm_rank()
0.0
24 24 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
71, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 72;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.96E+05
6.122E+08 1 2808
612195503 .TAU application
94.1
7.071E+06 5.759E+08 1000 2008
575874 DO_FULLADVANCE
92.5
2.223E+08 5.664E+08 1000 28000
566385 TIMESTEP_SSP_322
32.6
1.844E+08 1.995E+08 3000 6000 66496 GET_KINETIC_ADVANCE
16.9
1.034E+08 1.034E+08 24000 0 4308 MPI_Alltoall()
6.7
3.633E+07 4.123E+07 1001 2002 41186 GET_FIELD_EXPLICIT
4.1
2.534E+07 2.534E+07 10979 0 2308 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16171374 MAKE_LAMBDA_GRID
1.4
6.963E+06 8.294E+06 1 770
8293935 MAKE_MAXWELL_MATRIX
0.7
3.938E+06 4.344E+06 1 130
4343777 MAKE_POISSON_MATRIX
0.5
3.303E+06 3.303E+06 1122 0 2944 MPI_Bcast()
0.3
1.437E+05 2.019E+06 1000 1000 2019 GET_ERROR
0.2
7.335E+04 1.504E+06 1000 1000 1504 CATCH_HALT_SIGNAL
0.1
6.931E+05 6.931E+05 1 0
693098 MAKE_THETA_OPERATORS
0.1
1.572E+05 5.721E+05 1 257
572112 MAKE_BLEND_ARRAYS
0.1
2.337E+05 4.997E+05 10 160 49973 WRITE_BIG
0.1
3.974E+05 3.974E+05 1 0
397426 MAKE_RADIAL_OPERATORS
0.1
3.379E+05 3.692E+05 1 6
369195 MAKE_IMPLICIT_ADVECT
0.0
387 8.814E+04 1 2 88145 WRITE_INPUT
0.0
7.062E+04 7.083E+04 1 8 70827 MPI_Init()
0.0
3.013E+04 3.013E+04 2 0 15063 MPI_Comm_split()
0.0
1.609E+04 1.609E+04 12 0 1341 MPI_Reduce()
0.0
4790 4894 1 4 4894 MPI_Finalize()
0.0
1653 1653 1 0 1653 MAKE_PROFILES
0.0
106 106 4 0 26 MPI_Attr_put()
0.0
104 104
4 0 26 MPI_Keyval_free()
0.0
103 103 4 0 26 MPI_Attr_get()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
72, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 73;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.144E+05
6.142E+08 1 2808
614243157 .TAU application
93.9
7.07E+06 5.769E+08 1000 2008
576915 DO_FULLADVANCE
92.4
2.197E+08 5.674E+08 1000 28000
567388 TIMESTEP_SSP_322
32.8
1.853E+08 2.015E+08 3000 6000 67166 GET_KINETIC_ADVANCE
17.0
1.044E+08 1.044E+08 24000 0 4349 MPI_Alltoall()
6.8
3.658E+07 4.185E+07 1001 2002 41807 GET_FIELD_EXPLICIT
4.4
2.688E+07 2.688E+07 10979 0 2449 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200454 MAKE_LAMBDA_GRID
1.3 6.97E+06
8.279E+06 1 770
8279023 MAKE_MAXWELL_MATRIX
0.7
3.956E+06 4.296E+06 1 130
4296265 MAKE_POISSON_MATRIX
0.7
4.241E+06 4.241E+06 1122 0 3780 MPI_Bcast()
0.3
1.428E+05 2.062E+06 1000 1000 2062 GET_ERROR
0.3
7.314E+04 1.538E+06 1000 1000 1538 CATCH_HALT_SIGNAL
0.1
7.007E+05 7.007E+05 1 0
700680 MAKE_THETA_OPERATORS
0.1
1.591E+05 5.726E+05 1 257
572585 MAKE_BLEND_ARRAYS
0.1
2.358E+05 5.008E+05 10 160 50082 WRITE_BIG
0.1
4.023E+05 4.023E+05 1 0
402269 MAKE_RADIAL_OPERATORS
0.1
3.459E+05 3.779E+05 1 6
377867 MAKE_IMPLICIT_ADVECT
0.0
391 1.355E+05 1 2
135498 WRITE_INPUT
0.0
6.823E+04 6.844E+04 1 8 68444 MPI_Init()
0.0
4.13E+04 4.13E+04 2 0 20652 MPI_Comm_split()
0.0
5517 5517 12 0 460 MPI_Reduce()
0.0
5028 5138 1 4 5138 MPI_Finalize()
0.0
1737 1737 1
0 1737 MAKE_PROFILES
0.0
110 110 4 0 28 MPI_Keyval_free()
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
101 101 4 0 25 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
73, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 74;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.967E+05
6.124E+08 1 2808
612411346 .TAU application
93.9
7.068E+06 5.75E+08 1000 2008
575031 DO_FULLADVANCE
92.3
2.2E+08 5.655E+08 1000 28000
565498 TIMESTEP_SSP_322
32.8
1.851E+08 2.009E+08 3000 6000 66951 GET_KINETIC_ADVANCE
16.8
1.028E+08 1.028E+08 24000 0 4282 MPI_Alltoall()
6.9
3.661E+07 4.196E+07 1001 2002 41917 GET_FIELD_EXPLICIT
4.3
2.657E+07 2.657E+07 10979 0 2420 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201172 MAKE_LAMBDA_GRID
1.4
6.97E+06 8.279E+06 1 770
8279284 MAKE_MAXWELL_MATRIX
0.7
3.954E+06 4.35E+06 1 130
4350205 MAKE_POISSON_MATRIX
0.7
4.281E+06 4.281E+06 1122 0 3816 MPI_Bcast()
0.3
1.421E+05 2.072E+06 1000 1000 2072 GET_ERROR
0.3
7.291E+04 1.586E+06 1000 1000 1586 CATCH_HALT_SIGNAL
0.1
7.005E+05 7.005E+05 1 0
700510 MAKE_THETA_OPERATORS
0.1 1.569E+05 5.645E+05 1 257
564549 MAKE_BLEND_ARRAYS
0.1
2.321E+05 4.982E+05 10 160 49816 WRITE_BIG
0.1
4.022E+05 4.022E+05 1 0
402231 MAKE_RADIAL_OPERATORS
0.1
3.411E+05 3.731E+05 1 6
373114 MAKE_IMPLICIT_ADVECT
0.0
402 1.353E+05 1 2
135268 WRITE_INPUT
0.0
7.044E+04 7.065E+04 1 8 70653 MPI_Init()
0.0
4.008E+04 4.008E+04 2 0 20042 MPI_Comm_split()
0.0
6647 6647 12 0 554 MPI_Reduce()
0.0
4816 4925 1 4 4925 MPI_Finalize()
0.0
1749 1749 1
0 1749 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Keyval_free()
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
102 102 4
0 26 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
33 33 2 0 16 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
74, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 75;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.945E+05
6.13E+08 1 2808
612999959 .TAU application
94.1
7.072E+06 5.767E+08 1000 2008
576671 DO_FULLADVANCE
92.5
2.23E+08 5.671E+08 1000 28000
567139 TIMESTEP_SSP_322
32.5
1.844E+08 1.995E+08 3000 6000 66502 GET_KINETIC_ADVANCE
16.8
1.03E+08 1.03E+08 24000 0 4290 MPI_Alltoall()
6.8
3.639E+07 4.172E+07 1001 2002 41683 GET_FIELD_EXPLICIT
4.2
2.586E+07 2.586E+07 10979 0 2355 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16174152 MAKE_LAMBDA_GRID
1.4
6.957E+06 8.324E+06 1 770
8324119 MAKE_MAXWELL_MATRIX
0.7
3.937E+06 4.31E+06 1 130
4309611 MAKE_POISSON_MATRIX
0.5
3.347E+06 3.347E+06 1122 0 2983 MPI_Bcast()
0.3
1.469E+05 2.077E+06 1000 1000 2077 GET_ERROR
0.2
7.222E+04 1.498E+06 1000 1000 1498 CATCH_HALT_SIGNAL
0.1
6.939E+05 6.939E+05 1 0
693905 MAKE_THETA_OPERATORS
0.1
1.575E+05 5.597E+05 1 257
559735 MAKE_BLEND_ARRAYS
0.1
2.336E+05 4.834E+05 10 160 48342 WRITE_BIG
0.1
3.989E+05 3.989E+05 1 0
398872 MAKE_RADIAL_OPERATORS
0.1
3.4E+05 3.711E+05 1 6
371122 MAKE_IMPLICIT_ADVECT
0.0
410 9.024E+04 1 2 90239 WRITE_INPUT
0.0
6.789E+04 6.809E+04 1 8 68094 MPI_Init()
0.0
2.989E+04 2.989E+04 2 0 14945 MPI_Comm_split()
0.0
5062 5168 1 4 5168 MPI_Finalize()
0.0
4081 4081 12 0 340 MPI_Reduce()
0.0
1715 1715 1 0 1715 MAKE_PROFILES
0.0
107 107 4 0 27 MPI_Attr_put()
0.0
106 106 4 0 26 MPI_Keyval_free()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
58 58 4
0 14 MPI_Comm_rank()
0.0
30 30 2 0 15 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
75, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 76;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.074E+05
6.155E+08 1
2808 615492591 .TAU
application
93.9
7.07E+06 5.78E+08 1000 2008
577992 DO_FULLADVANCE
92.3
2.21E+08 5.683E+08 1000 28000
568311 TIMESTEP_SSP_322
32.6
1.852E+08 2.005E+08 3000
6000 66817 GET_KINETIC_ADVANCE
17.0
1.049E+08 1.049E+08 24000 0 4371 MPI_Alltoall()
6.8
3.663E+07 4.195E+07 1001 2002 41913 GET_FIELD_EXPLICIT
4.2
2.609E+07 2.609E+07 10979
0 2376 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200192 MAKE_LAMBDA_GRID
1.3
6.965E+06 8.272E+06 1 770
8271505 MAKE_MAXWELL_MATRIX
0.7
3.945E+06 4.334E+06 1 130
4333694 MAKE_POISSON_MATRIX
0.7
4.266E+06 4.266E+06 1122 0 3802 MPI_Bcast()
0.4
1.42E+05 2.205E+06 1000 1000 2205 GET_ERROR
0.3
7.839E+04 1.572E+06 1000 1000 1572 CATCH_HALT_SIGNAL
0.1
7.005E+05 7.005E+05 1 0
700498 MAKE_THETA_OPERATORS
0.1
1.579E+05 5.673E+05 1 257
567284 MAKE_BLEND_ARRAYS
0.1
2.386E+05 5.127E+05 10 160 51268 WRITE_BIG
0.1
4.342E+05 4.652E+05 1 6
465205 MAKE_IMPLICIT_ADVECT
0.1
4.388E+05 4.388E+05 1 0
438775 MAKE_RADIAL_OPERATORS
0.0
410 1.354E+05 1 2
135410 WRITE_INPUT
0.0
6.825E+04 6.846E+04 1 8 68456 MPI_Init()
0.0
4.006E+04 4.006E+04 2 0 20032 MPI_Comm_split()
0.0
1.029E+04 1.029E+04 12 0 858 MPI_Reduce()
0.0
5118 5225 1 4 5225 MPI_Finalize()
0.0
1735 1735 1 0 1735 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0 107 107 4 0 27 MPI_Keyval_free()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
65 65 4 0 16 MPI_Comm_rank()
0.0
30 30 2 0 15 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
76, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096
0 Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 77;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.069E+05
6.098E+08 1 2808
609843557 .TAU application
94.0
7.07E+06 5.734E+08 1000 2008
573426 DO_FULLADVANCE
92.4
2.2E+08 5.638E+08 1000 28000
563760 TIMESTEP_SSP_322
32.7
1.846E+08 1.995E+08 3000 6000 66513 GET_KINETIC_ADVANCE
16.9
1.028E+08 1.028E+08 24000 0 4285 MPI_Alltoall()
6.8
3.641E+07 4.139E+07 1001 2002 41349 GET_FIELD_EXPLICIT
4.2
2.537E+07 2.537E+07 10979 0 2311 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16172591 MAKE_LAMBDA_GRID
1.3
6.955E+06 8.217E+06 1 770
8217292 MAKE_MAXWELL_MATRIX
0.7
3.933E+06 4.353E+06 1 130
4353151 MAKE_POISSON_MATRIX
0.5
3.349E+06 3.349E+06 1122 0 2985 MPI_Bcast()
0.4
1.428E+05 2.22E+06 1000 1000 2220 GET_ERROR
0.3
7.808E+04 1.541E+06 1000 1000 1541 CATCH_HALT_SIGNAL
0.1
6.939E+05 6.939E+05 1 0
693919 MAKE_THETA_OPERATORS
0.1
1.588E+05 5.643E+05 1 257
564269 MAKE_BLEND_ARRAYS
0.1
2.307E+05 4.754E+05 10 160 47542 WRITE_BIG
0.1
4.355E+05 4.658E+05 1 6
465832 MAKE_IMPLICIT_ADVECT
0.1
4.349E+05 4.349E+05 1 0
434853 MAKE_RADIAL_OPERATORS
0.0
405 9.017E+04 1 2 90174 WRITE_INPUT
0.0
7.052E+04 7.072E+04 1 8 70723 MPI_Init()
0.0
3.057E+04 3.057E+04 2 0 15286 MPI_Comm_split()
0.0
4802 4907 1 4 4907 MPI_Finalize()
0.0
4135 4135 12 0 345 MPI_Reduce()
0.0
1678 1678 1 0 1678 MAKE_PROFILES
0.0
105 105 4 0 26 MPI_Attr_put()
0.0
105 105 4
0 26 MPI_Keyval_free()
0.0
97 97 4 0 24 MPI_Attr_get()
0.0
59 59 4 0 15 MPI_Comm_rank()
0.0
29 29 2 0
14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
77, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 78;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.149E+05
6.112E+08 1 2808
611202066 .TAU application
94.1
7.07E+06 5.749E+08 1000 2008
574895 DO_FULLADVANCE
92.5
2.209E+08 5.654E+08 1000 28000
565433 TIMESTEP_SSP_322
32.7
1.846E+08 1.997E+08 3000 6000 66581 GET_KINETIC_ADVANCE
16.9
1.034E+08 1.034E+08 24000 0 4309 MPI_Alltoall()
6.8
3.643E+07 4.145E+07 1001 2002 41405 GET_FIELD_EXPLICIT
4.2
2.54E+07 2.54E+07 10979 0 2313 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16171166 MAKE_LAMBDA_GRID
1.4
6.965E+06 8.268E+06 1 770
8267559 MAKE_MAXWELL_MATRIX
0.7
3.937E+06 4.316E+06 1 130
4316318 MAKE_POISSON_MATRIX
0.5
3.21E+06 3.21E+06 1122 0 2861 MPI_Bcast()
0.3
1.452E+05 2.003E+06 1000 1000 2003 GET_ERROR
0.2
7.767E+04 1.494E+06 1000 1000 1494 CATCH_HALT_SIGNAL
0.1
6.938E+05 6.938E+05 1 0
693840 MAKE_THETA_OPERATORS
0.1
1.588E+05 5.655E+05 1 257
565472 MAKE_BLEND_ARRAYS
0.1
2.345E+05 4.886E+05 10 160 48863 WRITE_BIG
0.1
4.347E+05 4.652E+05 1 6
465169 MAKE_IMPLICIT_ADVECT
0.1
4.352E+05 4.352E+05 1 0
435222 MAKE_RADIAL_OPERATORS
0.0
398 8.681E+04 1 2 86809 WRITE_INPUT
0.0
6.802E+04 6.822E+04 1 8 68225 MPI_Init()
0.0
2.994E+04 2.994E+04 2 0 14972 MPI_Comm_split()
0.0
8869 8869 12 0 739 MPI_Reduce()
0.0
5116 5234 1 4 5234 MPI_Finalize()
0.0
1713 1713 1 0 1713 MAKE_PROFILES
0.0
118 118 4 0 30 MPI_Keyval_free()
0.0
110 110 4 0 28 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
68 68 4 0 17 MPI_Comm_rank()
0.0
29 29 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
78, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 79;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.054E+05
6.139E+08 1 2808
613911358 .TAU application
93.9
7.066E+06 5.764E+08 1000 2008
576442 DO_FULLADVANCE
92.3
2.207E+08 5.667E+08 1000 28000
566723 TIMESTEP_SSP_322
32.7 1.853E+08
2.007E+08 3000 6000 66889 GET_KINETIC_ADVANCE
16.9
1.037E+08 1.037E+08 24000 0 4322 MPI_Alltoall()
6.8
3.662E+07 4.166E+07 1001 2002 41620 GET_FIELD_EXPLICIT
4.2 2.604E+07
2.604E+07 10979 0 2371 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201225 MAKE_LAMBDA_GRID
1.3
6.975E+06 8.288E+06 1 770
8287764 MAKE_MAXWELL_MATRIX
0.7 3.96E+06 4.342E+06 1 130
4341695 MAKE_POISSON_MATRIX
0.7
4.222E+06 4.222E+06 1122 0 3763 MPI_Bcast()
0.4
1.426E+05 2.255E+06 1000 1000 2255 GET_ERROR
0.2
7.775E+04 1.53E+06 1000 1000 1530 CATCH_HALT_SIGNAL
0.1
7.019E+05 7.019E+05 1 0
701948 MAKE_THETA_OPERATORS
0.1
1.581E+05 5.598E+05 1 257
559820 MAKE_BLEND_ARRAYS
0.1
2.382E+05 5.012E+05 10 160 50117 WRITE_BIG
0.1
4.402E+05 4.715E+05 1 6
471488 MAKE_IMPLICIT_ADVECT
0.1
4.392E+05 4.392E+05 1 0
439215 MAKE_RADIAL_OPERATORS
0.0
392 1.351E+05 1 2
135136 WRITE_INPUT
0.0
7.049E+04 7.07E+04 1 8 70695 MPI_Init()
0.0
4.007E+04 4.007E+04 2 0 20035 MPI_Comm_split()
0.0
5270 5270 12
0 439 MPI_Reduce()
0.0
4771 4877 1 4 4877 MPI_Finalize()
0.0
1680 1680 1 0 1680 MAKE_PROFILES
0.0
108 108 4 0
27 MPI_Attr_put()
0.0
106 106 4 0 26 MPI_Keyval_free()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
79, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 80;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.969E+05
6.083E+08 1 2808
608346501 .TAU application
94.0
7.064E+06 5.719E+08 1000 2008
571938 DO_FULLADVANCE
92.4
2.203E+08 5.624E+08 1000 28000
562403 TIMESTEP_SSP_322
32.9
1.843E+08 2.001E+08 3000 6000 66713 GET_KINETIC_ADVANCE
16.5
1.001E+08 1.001E+08 24000 0 4171 MPI_Alltoall()
6.9
3.638E+07 4.187E+07 1001 2002 41831 GET_FIELD_EXPLICIT
4.4
2.683E+07 2.683E+07 10979 0 2443 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16172351 MAKE_LAMBDA_GRID
1.4
6.959E+06 8.318E+06 1 770
8318495 MAKE_MAXWELL_MATRIX
0.7
3.937E+06 4.342E+06 1 130
4341572 MAKE_POISSON_MATRIX
0.6
3.358E+06 3.358E+06 1122 0 2993 MPI_Bcast()
0.3
1.462E+05 2.086E+06 1000 1000 2086 GET_ERROR
0.2
6.659E+04 1.51E+06 1000 1000 1510 CATCH_HALT_SIGNAL
0.1
6.933E+05 6.933E+05 1 0
693313 MAKE_THETA_OPERATORS
0.1
1.577E+05 5.714E+05 1 257
571400 MAKE_BLEND_ARRAYS
0.1
2.303E+05 4.866E+05 10 160 48665 WRITE_BIG
0.1
3.976E+05 3.976E+05 1 0
397630 MAKE_RADIAL_OPERATORS
0.1
3.409E+05 3.722E+05 1 6
372206 MAKE_IMPLICIT_ADVECT
0.0
447 8.998E+04 1 2 89981 WRITE_INPUT
0.0
7.022E+04 7.042E+04 1 8 70418 MPI_Init()
0.0
3.07E+04 3.07E+04 2 0 15350 MPI_Comm_split()
0.0
6519 6519 12 0 543 MPI_Reduce()
0.0
4832 4950 1 4 4950 MPI_Finalize()
0.0
1725 1725 1 0 1725 MAKE_PROFILES
0.0
118 118 4 0 30 MPI_Keyval_free()
0.0
105 105 4 0 26 MPI_Attr_put()
0.0
96 96 4 0 24 MPI_Attr_get()
0.0
66 66 4 0
16 MPI_Comm_rank()
0.0
29 29 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
80, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 81;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.006E+05
6.14E+08 1 2808
613992807 .TAU application
93.9
7.066E+06 5.767E+08 1000 2008
576710 DO_FULLADVANCE
92.3
2.2E+08 5.669E+08 1000 28000
566891 TIMESTEP_SSP_322
32.7
1.85E+08 2.01E+08 3000 6000 66994 GET_KINETIC_ADVANCE
16.9
1.038E+08 1.038E+08 24000 0 4327 MPI_Alltoall()
6.9
3.662E+07 4.215E+07 1001 2002 42109 GET_FIELD_EXPLICIT
4.4
2.715E+07 2.715E+07 10979 0 2473 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201667 MAKE_LAMBDA_GRID
1.3
6.974E+06 8.244E+06 1 770
8244213 MAKE_MAXWELL_MATRIX
0.7
3.956E+06 4.323E+06 1 130
4322652 MAKE_POISSON_MATRIX
0.7 4.229E+06 4.229E+06 1122 0 3769 MPI_Bcast()
0.4
1.46E+05 2.356E+06 1000 1000 2356 GET_ERROR
0.2
7.212E+04 1.533E+06 1000 1000 1533 CATCH_HALT_SIGNAL
0.1
7.012E+05 7.012E+05 1 0
701171 MAKE_THETA_OPERATORS
0.1
1.581E+05 5.694E+05 1 257
569375 MAKE_BLEND_ARRAYS
0.1
2.313E+05 4.998E+05 10 160 49977 WRITE_BIG
0.1
4.016E+05 4.016E+05 1 0
401556 MAKE_RADIAL_OPERATORS
0.1
3.474E+05 3.793E+05 1 6
379325 MAKE_IMPLICIT_ADVECT
0.0
393 1.351E+05 1 2
135130 WRITE_INPUT
0.0
6.798E+04 6.818E+04 1 8 68184 MPI_Init()
0.0
4.072E+04 4.072E+04 2 0 20359 MPI_Comm_split()
0.0
5639 5639 12 0 470 MPI_Reduce()
0.0
5008 5114 1 4 5114 MPI_Finalize()
0.0
1653 1653 1 0 1653 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
106 106
4 0 26 MPI_Keyval_free()
0.0
99 99 4 0 25 MPI_Attr_get()
0.0
64 64 4 0 16 MPI_Comm_rank()
0.0
30 30 2 0 15 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
81, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 82;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.013E+05
6.137E+08 1 2808
613694645 .TAU application
93.9
7.067E+06 5.763E+08 1000 2008
576270 DO_FULLADVANCE
92.3
2.2E+08 5.666E+08 1000 28000
566621 TIMESTEP_SSP_322
32.8
1.851E+08 2.011E+08 3000 6000 67033 GET_KINETIC_ADVANCE
16.9
1.037E+08 1.037E+08 24000 0 4320 MPI_Alltoall()
6.8
3.66E+07 4.192E+07 1001 2002 41878 GET_FIELD_EXPLICIT
4.4
2.695E+07 2.695E+07 10979 0 2455 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201069 MAKE_LAMBDA_GRID
1.4 6.977E+06 8.349E+06 1 770
8348556 MAKE_MAXWELL_MATRIX
0.7
3.955E+06 4.355E+06 1 130
4354503 MAKE_POISSON_MATRIX
0.7
4.225E+06 4.225E+06 1122 0 3765 MPI_Bcast()
0.4
1.409E+05 2.183E+06 1000 1000 2183 GET_ERROR
0.2
7.209E+04 1.532E+06 1000 1000 1532 CATCH_HALT_SIGNAL
0.1
7.005E+05 7.005E+05 1 0
700501 MAKE_THETA_OPERATORS
0.1
1.583E+05 5.699E+05 1 257
569904 MAKE_BLEND_ARRAYS
0.1
2.322E+05 5.041E+05 10 160 50410 WRITE_BIG
0.1
4.022E+05 4.022E+05 1 0
402177 MAKE_RADIAL_OPERATORS
0.1
3.42E+05 3.735E+05 1 6
373496 MAKE_IMPLICIT_ADVECT
0.0
391 1.356E+05 1 2
135566 WRITE_INPUT
0.0
7.078E+04 7.101E+04 1 8 71008 MPI_Init()
0.0
4.082E+04 4.082E+04 2 0 20411 MPI_Comm_split()
0.0
6384 6384 12 0 532 MPI_Reduce()
0.0
4847 4956 1 4 4956 MPI_Finalize()
0.0
1777 1777 1
0 1777 MAKE_PROFILES
0.0
114 114 4 0 28 MPI_Attr_put()
0.0
110 110 4 0 28 MPI_Attr_get()
0.0
109 109 4 0 27 MPI_Keyval_free()
0.0
81 81 4 0 20 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
82, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345
Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 83;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.91E+05
6.106E+08 1 2808
610631193 .TAU application
94.1
7.072E+06 5.743E+08 1000 2008
574310 DO_FULLADVANCE
92.5
2.222E+08 5.646E+08 1000 28000
564621 TIMESTEP_SSP_322
32.6
1.845E+08 1.992E+08 3000 6000 66391 GET_KINETIC_ADVANCE
16.7
1.019E+08 1.019E+08 24000 0 4244 MPI_Alltoall()
6.8
3.638E+07 4.139E+07 1001 2002 41349 GET_FIELD_EXPLICIT
4.1
2.521E+07 2.521E+07 10979 0 2296 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16172149 MAKE_LAMBDA_GRID
1.4
6.959E+06 8.295E+06 1 770
8294855 MAKE_MAXWELL_MATRIX
0.7
3.938E+06 4.333E+06 1 130
4332647 MAKE_POISSON_MATRIX
0.6
3.363E+06 3.363E+06 1122 0 2998 MPI_Bcast()
0.4
1.45E+05 2.234E+06 1000 1000 2234 GET_ERROR
0.3
6.545E+04 1.545E+06 1000 1000 1545 CATCH_HALT_SIGNAL
0.1
6.944E+05 6.944E+05 1 0
694438 MAKE_THETA_OPERATORS
0.1 1.591E+05
5.652E+05 1 257
565205 MAKE_BLEND_ARRAYS
0.1
2.327E+05 4.818E+05 10 160 48184 WRITE_BIG
0.1
4.003E+05 4.003E+05 1 0
400346 MAKE_RADIAL_OPERATORS
0.1
3.363E+05 3.673E+05 1 6
367261 MAKE_IMPLICIT_ADVECT
0.0
390 9.047E+04 1 2 90468 WRITE_INPUT
0.0
6.794E+04 6.815E+04 1 8 68150 MPI_Init()
0.0
3.052E+04 3.052E+04 2 0 15259 MPI_Comm_split()
0.0
5119 5218 1 4 5218 MPI_Finalize()
0.0
4331 4331 12 0 361 MPI_Reduce()
0.0
1736 1736 1 0 1736 MAKE_PROFILES
0.0
108 108 4 0 27 MPI_Attr_put()
0.0
99 99 4 0 25 MPI_Keyval_free()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
59 59 4 0 15 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
83, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04 4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 84;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.1E+05
6.169E+08 1 2808
616897160 .TAU application
93.9
7.069E+06 5.795E+08 1000 2008
579458 DO_FULLADVANCE
92.4
2.239E+08 5.699E+08 1000 28000
569860 TIMESTEP_SSP_322
32.5 1.855E+08 2.004E+08 3000 6000 66803 GET_KINETIC_ADVANCE
16.8
1.039E+08 1.039E+08 24000 0 4331 MPI_Alltoall()
6.8
3.663E+07 4.17E+07 1001 2002 41660 GET_FIELD_EXPLICIT
4.1
2.542E+07 2.542E+07 10979 0 2315 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201585 MAKE_LAMBDA_GRID
1.3
6.973E+06 8.228E+06 1 770
8227605 MAKE_MAXWELL_MATRIX
0.7
3.943E+06 4.334E+06 1 130
4333668 MAKE_POISSON_MATRIX
0.7
4.248E+06 4.248E+06 1122 0 3786 MPI_Bcast()
0.3
1.391E+05 2.11E+06 1000 1000 2110 GET_ERROR
0.3
7.774E+04 1.557E+06 1000 1000 1557 CATCH_HALT_SIGNAL
0.1
7.008E+05 7.008E+05 1 0
700823 MAKE_THETA_OPERATORS
0.1
1.581E+05 5.662E+05 1 257
566175 MAKE_BLEND_ARRAYS
0.1
2.438E+05 5.263E+05 10 160 52633 WRITE_BIG
0.1
4.405E+05 4.715E+05 1 6
471542 MAKE_IMPLICIT_ADVECT
0.1
4.39E+05 4.39E+05 1 0
439003 MAKE_RADIAL_OPERATORS
0.0
392 1.363E+05 1 2
136349 WRITE_INPUT
0.0
6.808E+04 6.829E+04 1 8 68286 MPI_Init()
0.0
4.015E+04 4.015E+04 2 0 20074 MPI_Comm_split()
0.0
2.511E+04 2.511E+04 12 0 2093 MPI_Reduce()
0.0
5114 5217 1 4 5217 MPI_Finalize()
0.0
1671 1671 1 0 1671 MAKE_PROFILES
0.0
106 106 4 0 26 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Attr_get()
0.0
103 103 4 0 26 MPI_Keyval_free()
0.0
67 67 4 0 17 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
84, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 85;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.113E+05
6.122E+08 1 2808
612177975 .TAU application
94.1
7.069E+06 5.759E+08 1000 2008
575931 DO_FULLADVANCE
92.5
2.23E+08 5.664E+08 1000 28000
566373 TIMESTEP_SSP_322
32.6 1.846E+08
1.993E+08 3000 6000 66444 GET_KINETIC_ADVANCE
16.8
1.027E+08 1.027E+08 24000 0 4280 MPI_Alltoall()
6.8
3.64E+07 4.136E+07 1001 2002 41315 GET_FIELD_EXPLICIT
4.1 2.496E+07
2.496E+07 10979 0 2273 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16172664 MAKE_LAMBDA_GRID
1.3
6.954E+06 8.213E+06 1 770
8213490 MAKE_MAXWELL_MATRIX
0.7 3.933E+06 4.301E+06 1 130
4300532 MAKE_POISSON_MATRIX
0.5
3.228E+06 3.228E+06 1122 0 2877 MPI_Bcast()
0.3
1.42E+05 2.1E+06 1000 1000 2100 GET_ERROR
0.2
7.663E+04 1.508E+06 1000 1000 1508 CATCH_HALT_SIGNAL
0.1
6.936E+05 6.936E+05 1 0
693640 MAKE_THETA_OPERATORS
0.1
1.589E+05 5.662E+05 1 257
566206 MAKE_BLEND_ARRAYS
0.1
2.338E+05 4.893E+05 10 160 48929 WRITE_BIG
0.1
4.39E+05 4.696E+05 1 6
469582 MAKE_IMPLICIT_ADVECT
0.1
4.354E+05 4.354E+05 1 0
435446 MAKE_RADIAL_OPERATORS
0.0
384 8.702E+04 1 2 87016 WRITE_INPUT
0.0
7.044E+04 7.064E+04 1 8 70642 MPI_Init()
0.0
2.982E+04 2.982E+04 2 0 14910 MPI_Comm_split()
0.0
4757 4859 1
4 4859 MPI_Finalize()
0.0
4121 4121 12 0 343 MPI_Reduce()
0.0
1620 1620 1 0 1620 MAKE_PROFILES
0.0
105 105 4 0
26 MPI_Attr_put()
0.0
102 102 4 0 26 MPI_Keyval_free()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
59 59 4 0
15 MPI_Comm_rank()
0.0
24 24 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
85, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 86;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.131E+05
6.144E+08 1 2808
614354138 .TAU application
93.9
7.07E+06 5.768E+08 1000 2008
576831 DO_FULLADVANCE
92.3
2.209E+08 5.671E+08 1000 28000
567058 TIMESTEP_SSP_322
32.5
1.854E+08 1.998E+08 3000 6000 66585 GET_KINETIC_ADVANCE
17.1
1.049E+08 1.049E+08 24000 0 4370 MPI_Alltoall()
6.8
3.665E+07 4.157E+07 1001 2002 41527 GET_FIELD_EXPLICIT
4.1
2.493E+07 2.493E+07 10979 0 2271 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200091 MAKE_LAMBDA_GRID
1.3
6.98E+06 8.272E+06 1 770
8272016 MAKE_MAXWELL_MATRIX
0.7
3.954E+06 4.349E+06 1 130
4349362 MAKE_POISSON_MATRIX
0.7
4.271E+06 4.271E+06 1122 0 3807 MPI_Bcast()
0.4
1.447E+05 2.305E+06 1000 1000 2305 GET_ERROR
0.3
7.848E+04 1.571E+06 1000 1000 1571 CATCH_HALT_SIGNAL
0.1
7.018E+05 7.018E+05 1 0
701759 MAKE_THETA_OPERATORS
0.1
1.597E+05 5.675E+05 1 257
567495 MAKE_BLEND_ARRAYS
0.1
2.407E+05 5.041E+05 10 160 50412 WRITE_BIG
0.1
4.374E+05 4.687E+05 1 6
468708 MAKE_IMPLICIT_ADVECT
0.1
4.394E+05 4.394E+05 1 0
439439 MAKE_RADIAL_OPERATORS
0.0
390 1.353E+05 1 2
135276 WRITE_INPUT
0.0
6.797E+04 6.818E+04 1 8 68185 MPI_Init()
0.0
4.087E+04 4.087E+04 2 0 20434 MPI_Comm_split()
0.0
6410 6410 12 0 534 MPI_Reduce()
0.0
5096 5196 1
4 5196 MPI_Finalize()
0.0
1723 1723 1 0 1723 MAKE_PROFILES
0.0
112 112 4 0 28 MPI_Attr_put()
0.0
104 104 4
0 26 MPI_Attr_get()
0.0
100 100 4 0 25 MPI_Keyval_free()
0.0
63 63 4 0 16 MPI_Comm_rank()
0.0
28 28 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
86, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 87;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.055E+05
6.132E+08 1 2808
613181300 .TAU application
94.1
7.073E+06 5.768E+08 1000 2008
576765 DO_FULLADVANCE
92.1
2.203E+08 5.649E+08 1000 28000
564944 TIMESTEP_SSP_322
32.4
1.845E+08 1.988E+08 3000 6000 66277 GET_KINETIC_ADVANCE
17.1
1.046E+08 1.046E+08 24000 0 4358 MPI_Alltoall()
6.7
3.642E+07 4.128E+07 1001 2002 41239 GET_FIELD_EXPLICIT
4.4
2.684E+07 2.684E+07 10979 0 2444 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16171692 MAKE_LAMBDA_GRID
1.3 6.957E+06
8.24E+06 1 770
8239633 MAKE_MAXWELL_MATRIX
0.7
3.94E+06 4.387E+06 1 130
4386615 MAKE_POISSON_MATRIX
0.5
3.289E+06 3.289E+06 1122 0 2931 MPI_Bcast()
0.4 2.351E+05 2.673E+06 10 160
267267 WRITE_BIG
0.4
1.43E+05 2.174E+06 1000 1000 2174 GET_ERROR
0.2
7.732E+04 1.53E+06 1000 1000 1530 CATCH_HALT_SIGNAL
0.1
6.935E+05 6.935E+05 1 0
693541 MAKE_THETA_OPERATORS
0.1
1.582E+05 5.645E+05 1 257
564514 MAKE_BLEND_ARRAYS
0.1
4.331E+05 4.638E+05 1 6
463793 MAKE_IMPLICIT_ADVECT
0.1
4.359E+05 4.359E+05 1 0
435944 MAKE_RADIAL_OPERATORS
0.0
396 8.895E+04 1 2 88947 WRITE_INPUT
0.0
7.072E+04 7.094E+04 1 8 70935 MPI_Init()
0.0
2.976E+04 2.976E+04 2 0 14878 MPI_Comm_split()
0.0
4789 4893 1 4 4893 MPI_Finalize()
0.0
4442 4442 12 0 370 MPI_Reduce()
0.0
1742 1742 1 0 1742 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
104 104 4 0 26 MPI_Keyval_free()
0.0
102 102 4 0 26 MPI_Attr_get()
0.0
59 59 4 0 15 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
87, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev. Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 88;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.94E+05
6.144E+08 1 2808
614384970 .TAU application
93.9
7.07E+06 5.771E+08 1000 2008
577065 DO_FULLADVANCE
92.4
2.224E+08 5.676E+08 1000 28000
567620 TIMESTEP_SSP_322
32.6
1.85E+08 2.004E+08 3000 6000 66788 GET_KINETIC_ADVANCE
16.8
1.031E+08 1.031E+08 24000 0 4298 MPI_Alltoall()
6.8
3.663E+07 4.173E+07 1001 2002 41693 GET_FIELD_EXPLICIT
4.2
2.579E+07 2.579E+07 10979 0 2349 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201252 MAKE_LAMBDA_GRID
1.3
6.975E+06 8.282E+06 1 770
8281851 MAKE_MAXWELL_MATRIX
0.7
3.953E+06 4.348E+06 1 130
4347944 MAKE_POISSON_MATRIX
0.7
4.22E+06 4.22E+06 1122 0 3761 MPI_Bcast()
0.3
1.438E+05 1.973E+06 1000 1000 1973 GET_ERROR
0.2
7.245E+04 1.523E+06 1000 1000 1523 CATCH_HALT_SIGNAL
0.1
7.002E+05 7.002E+05 1 0
700206 MAKE_THETA_OPERATORS
0.1
1.568E+05 5.66E+05 1 257
565978 MAKE_BLEND_ARRAYS
0.1
2.372E+05 5.081E+05 10 160 50812 WRITE_BIG
0.1
4.016E+05 4.016E+05 1 0
401580 MAKE_RADIAL_OPERATORS
0.1
3.409E+05 3.725E+05 1 6
372481 MAKE_IMPLICIT_ADVECT
0.0
398 1.359E+05 1 2
135881 WRITE_INPUT
0.0
6.808E+04 6.828E+04 1 8 68280 MPI_Init()
0.0
4.04E+04 4.04E+04 2 0 20200 MPI_Comm_split()
0.0
7456 7456 12 0 621 MPI_Reduce()
0.0
5058 5166 1 4 5166 MPI_Finalize()
0.0
1718 1718 1 0 1718 MAKE_PROFILES
0.0
109 109 4
0 27 MPI_Attr_put()
0.0
108 108 4 0 27 MPI_Keyval_free()
0.0
95 95 4 0 24 MPI_Attr_get()
0.0
61 61 4 0
15 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
88, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 89;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.964E+05
6.101E+08 1 2808
610128679 .TAU application
94.1
7.069E+06 5.74E+08 1000 2008
574006 DO_FULLADVANCE
92.5
2.196E+08 5.646E+08 1000 28000
564615 TIMESTEP_SSP_322
32.7
1.843E+08 1.997E+08 3000 6000 66579 GET_KINETIC_ADVANCE
17.0
1.037E+08 1.037E+08 24000 0 4321 MPI_Alltoall()
6.8
3.639E+07 4.158E+07 1001 2002 41536 GET_FIELD_EXPLICIT
4.2
2.58E+07 2.58E+07 10979 0 2350 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16171932 MAKE_LAMBDA_GRID
1.3
6.953E+06 8.23E+06 1 770
8230001 MAKE_MAXWELL_MATRIX
0.7
3.938E+06 4.322E+06 1 130
4321809 MAKE_POISSON_MATRIX
0.5
3.216E+06 3.216E+06 1122 0 2866 MPI_Bcast()
0.3
1.416E+05 1.934E+06 1000 1000 1934 GET_ERROR
0.2
7.316E+04 1.521E+06 1000 1000 1521 CATCH_HALT_SIGNAL
0.1
6.931E+05 6.931E+05 1 0
693082 MAKE_THETA_OPERATORS
0.1
1.581E+05 5.621E+05 1 257
562147 MAKE_BLEND_ARRAYS
0.1
2.276E+05 4.887E+05 10 160 48865 WRITE_BIG
0.1 3.981E+05
3.981E+05 1 0
398060 MAKE_RADIAL_OPERATORS
0.1
3.429E+05 3.739E+05 1 6
373866 MAKE_IMPLICIT_ADVECT
0.0
408 8.449E+04 1 2 84486 WRITE_INPUT
0.0 6.998E+04
7.019E+04 1 8 70194 MPI_Init()
0.0
3.018E+04 3.018E+04 2 0 15088 MPI_Comm_split()
0.0
4809 4917 1 4 4917 MPI_Finalize()
0.0
4442 4442 12 0 370 MPI_Reduce()
0.0
1731 1731 1 0 1731 MAKE_PROFILES
0.0
110 110 4 0 28 MPI_Attr_put()
0.0
108 108 4 0 27 MPI_Keyval_free()
0.0
102 102 4 0 26 MPI_Attr_get()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
26 26 2
0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
89, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0
0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 90;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.98E+05
6.125E+08 1 2808
612476525 .TAU application
94.1
7.066E+06 5.762E+08 1000 2008
576168 DO_FULLADVANCE
92.0
2.196E+08 5.632E+08 1000 28000
563203 TIMESTEP_SSP_322
32.6
1.844E+08 1.995E+08 3000 6000 66514 GET_KINETIC_ADVANCE
16.7
1.022E+08 1.022E+08 24000 0 4260 MPI_Alltoall()
6.8
3.64E+07 4.188E+07 1001 2002 41840 GET_FIELD_EXPLICIT
4.8
2.945E+07 2.945E+07 10979 0 2682 MPI_Allreduce()
2.6
1.617E+07 1.617E+07 1 0
16174263 MAKE_LAMBDA_GRID
1.3
6.959E+06 8.265E+06 1 770
8264741 MAKE_MAXWELL_MATRIX
0.9
1.443E+05 5.519E+06 1000 1000 5519 GET_ERROR
0.7
3.937E+06 4.326E+06 1 130
4325547 MAKE_POISSON_MATRIX
0.5
3.349E+06 3.349E+06 1122 0 2985 MPI_Bcast()
0.2
7.342E+04 1.517E+06 1000 1000 1517 CATCH_HALT_SIGNAL
0.1
6.941E+05 6.941E+05 1 0
694114 MAKE_THETA_OPERATORS
0.1
1.589E+05 5.619E+05 1 257
561919 MAKE_BLEND_ARRAYS
0.1
2.26E+05 4.785E+05 10 160 47845 WRITE_BIG
0.1
3.994E+05 3.994E+05 1 0
399431 MAKE_RADIAL_OPERATORS
0.1
3.429E+05 3.737E+05 1 6
373657 MAKE_IMPLICIT_ADVECT
0.0
408 9.105E+04 1 2 91050 WRITE_INPUT
0.0
6.831E+04 6.851E+04 1 8 68514 MPI_Init()
0.0
2.973E+04 2.973E+04 2
0 14866 MPI_Comm_split()
0.0
5181 5287 1 4 5287 MPI_Finalize()
0.0
5141 5141 12 0 428 MPI_Reduce()
0.0
1689 1689 1 0 1689 MAKE_PROFILES
0.0
106 106 4 0 26 MPI_Keyval_free()
0.0
104 104 4 0 26 MPI_Attr_put()
0.0 102 102 4 0 26 MPI_Attr_get()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
25 25 2 0 12 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
90, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 91;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.969E+05
6.151E+08 1 2808
615075538 .TAU application
93.9
7.07E+06 5.777E+08 1000 2008
577741 DO_FULLADVANCE
92.3
2.222E+08 5.68E+08 1000 28000
567950 TIMESTEP_SSP_322
32.7
1.852E+08 2.009E+08 3000 6000 66955 GET_KINETIC_ADVANCE
16.8
1.033E+08 1.033E+08 24000 0 4304 MPI_Alltoall()
6.8
3.658E+07 4.161E+07 1001 2002 41565 GET_FIELD_EXPLICIT
4.3
2.635E+07 2.635E+07 10979 0 2400 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201491 MAKE_LAMBDA_GRID
1.3 6.974E+06
8.291E+06 1 770
8290678 MAKE_MAXWELL_MATRIX
0.7
3.959E+06 4.324E+06 1 130
4323512 MAKE_POISSON_MATRIX
0.7
4.247E+06 4.247E+06 1122 0 3785 MPI_Bcast()
0.4 1.436E+05 2.328E+06 1000 1000 2328 GET_ERROR
0.3
7.269E+04 1.556E+06 1000 1000 1556 CATCH_HALT_SIGNAL
0.1
7.01E+05 7.01E+05 1 0
701018 MAKE_THETA_OPERATORS
0.1
1.597E+05 5.685E+05 1 257
568513 MAKE_BLEND_ARRAYS
0.1
2.367E+05 4.974E+05 10 160 49740 WRITE_BIG
0.1
4.027E+05 4.027E+05 1 0
402672 MAKE_RADIAL_OPERATORS
0.1
3.406E+05 3.724E+05 1 6
372416 MAKE_IMPLICIT_ADVECT
0.0
389 1.36E+05 1 2
135984 WRITE_INPUT
0.0
7.045E+04 7.068E+04 1 8 70677 MPI_Init()
0.0
3.999E+04 3.999E+04 2 0 19996 MPI_Comm_split()
0.0
5431 5431 12 0 453 MPI_Reduce()
0.0
4757 4870 1 4 4870 MPI_Finalize()
0.0
1743 1743 1 0 1743 MAKE_PROFILES
0.0
124 124 4 0 31 MPI_Attr_put()
0.0
113 113 4 0 28 MPI_Keyval_free()
0.0
101 101 4 0
25 MPI_Attr_get()
0.0
65 65 4 0 16 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
91, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 92;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.026E+05
6.095E+08 1 2808
609503507 .TAU application
94.0
7.068E+06 5.732E+08 1000 2008
573180 DO_FULLADVANCE
92.5
2.231E+08 5.637E+08 1000 28000
563655 TIMESTEP_SSP_322
32.6
1.847E+08 1.988E+08 3000 6000 66271 GET_KINETIC_ADVANCE
16.5
1.004E+08 1.004E+08 24000 0 4182 MPI_Alltoall()
6.8
3.638E+07 4.139E+07 1001 2002 41347 GET_FIELD_EXPLICIT
4.0
2.445E+07 2.445E+07 10979 0 2227 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16171824 MAKE_LAMBDA_GRID
1.4
6.96E+06 8.245E+06 1 770
8245234 MAKE_MAXWELL_MATRIX
0.7
3.925E+06 4.313E+06 1 130
4312866 MAKE_POISSON_MATRIX
0.5
3.27E+06 3.27E+06 1122 0 2914 MPI_Bcast()
0.3
1.473E+05 2.047E+06 1000 1000 2047 GET_ERROR
0.2
7.665E+04 1.508E+06 1000 1000 1508 CATCH_HALT_SIGNAL
0.1
6.937E+05 6.937E+05 1 0
693684 MAKE_THETA_OPERATORS
0.1
1.574E+05 5.613E+05 1 257
561321 MAKE_BLEND_ARRAYS
0.1
2.386E+05 5.093E+05 10 160 50925 WRITE_BIG
0.1
4.394E+05 4.699E+05 1 6
469870 MAKE_IMPLICIT_ADVECT
0.1
4.356E+05 4.356E+05 1 0
435557 MAKE_RADIAL_OPERATORS
0.0
386 8.672E+04 1 2 86723 WRITE_INPUT
0.0
6.783E+04 6.803E+04 1 8 68028 MPI_Init()
0.0
3.004E+04 3.004E+04 2 0 15018 MPI_Comm_split()
0.0
2.976E+04 2.976E+04 12 0 2480 MPI_Reduce()
0.0
5063 5160 1 4 5160 MPI_Finalize()
0.0
1653 1653 1 0 1653 MAKE_PROFILES
0.0
104 104 4 0 26 MPI_Attr_put()
0.0
98 98 4 0 24 MPI_Attr_get()
0.0
97 97 4 0 24 MPI_Keyval_free()
0.0
69 69 4
0 17 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
92, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 93;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.102E+05
6.131E+08 1 2808
613058652 .TAU application
93.9 7.068E+06
5.756E+08 1000 2008
575649 DO_FULLADVANCE
92.4
2.207E+08 5.662E+08 1000 28000
566169 TIMESTEP_SSP_322
32.6
1.854E+08 2E+08 3000 6000 66669 GET_KINETIC_ADVANCE
16.9 1.038E+08 1.038E+08 24000 0 4326 MPI_Alltoall()
6.8
3.662E+07 4.166E+07 1001 2002 41613 GET_FIELD_EXPLICIT
4.1
2.49E+07 2.49E+07 10979 0 2268 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16201577 MAKE_LAMBDA_GRID
1.4
6.977E+06 8.277E+06 1 770
8277298 MAKE_MAXWELL_MATRIX
0.7
3.954E+06 4.321E+06 1 130
4320915 MAKE_POISSON_MATRIX
0.7
4.217E+06 4.217E+06 1122 0 3759 MPI_Bcast()
0.3
1.423E+05 2.014E+06 1000 1000 2014 GET_ERROR
0.2
7.905E+04 1.527E+06 1000 1000 1527 CATCH_HALT_SIGNAL
0.1
7.004E+05 7.004E+05 1 0
700376 MAKE_THETA_OPERATORS
0.1
1.593E+05 5.589E+05 1 257
558942 MAKE_BLEND_ARRAYS
0.1
2.411E+05 5.037E+05 10 160 50368 WRITE_BIG
0.1
4.411E+05 4.722E+05 1 6
472224 MAKE_IMPLICIT_ADVECT
0.1
4.394E+05 4.394E+05 1 0
439380 MAKE_RADIAL_OPERATORS
0.0
392 1.36E+05 1 2
135950 WRITE_INPUT
0.0
7.055E+04 7.076E+04 1 8 70759 MPI_Init()
0.0
4.007E+04 4.007E+04 2 0 20034 MPI_Comm_split()
0.0
5467 5467 12 0 456 MPI_Reduce()
0.0
4874 4979 1 4 4979 MPI_Finalize()
0.0
1771 1771 1 0 1771 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
105 105 4 0 26 MPI_Keyval_free()
0.0
103 103 4 0 26 MPI_Attr_get()
0.0
58 58 4 0 14 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
93, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 94;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.136E+05
6.15E+08 1 2808
615023040 .TAU application
93.9
7.07E+06 5.775E+08 1000 2008
577503 DO_FULLADVANCE
92.4
2.241E+08 5.68E+08 1000 28000
568015 TIMESTEP_SSP_322
32.7
1.854E+08 2.01E+08 3000 6000 66984 GET_KINETIC_ADVANCE
16.4
1.011E+08 1.011E+08 24000 0 4212 MPI_Alltoall()
6.8
3.664E+07 4.196E+07 1001 2002 41919 GET_FIELD_EXPLICIT
4.3
2.622E+07 2.622E+07 10979 0 2388 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16200848 MAKE_LAMBDA_GRID
1.4
6.986E+06 8.313E+06 1 770
8312952 MAKE_MAXWELL_MATRIX
0.7
3.955E+06 4.353E+06 1 130
4352903 MAKE_POISSON_MATRIX
0.7
4.215E+06 4.215E+06 1122 0 3756 MPI_Bcast()
0.3
1.403E+05 2.016E+06 1000 1000 2016 GET_ERROR
0.3
7.774E+04 1.556E+06 1000 1000 1556 CATCH_HALT_SIGNAL
0.1
7.001E+05 7.001E+05 1 0
700054 MAKE_THETA_OPERATORS
0.1
1.598E+05 5.685E+05 1 257
568467 MAKE_BLEND_ARRAYS
0.1
2.431E+05 5.098E+05 10 160 50975 WRITE_BIG
0.1
4.465E+05 4.774E+05 1 6
477418 MAKE_IMPLICIT_ADVECT
0.1
4.4E+05 4.4E+05 1 0
440003 MAKE_RADIAL_OPERATORS
0.0
385 1.337E+05 1 2
133731 WRITE_INPUT
0.0
7.067E+04 7.088E+04 1 8 70877 MPI_Init()
0.0
4.072E+04 4.072E+04 2 0 20362 MPI_Comm_split()
0.0
6435 6435 12 0 536 MPI_Reduce()
0.0
5122 5228 1
4 5228 MPI_Finalize()
0.0
1638 1638 1 0 1638 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
106 106 4 0
26 MPI_Keyval_free()
0.0
97 97 4 0 24 MPI_Attr_get()
0.0
64 64 4 0 16 MPI_Comm_rank()
0.0
27 27 2 0
14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
94, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122
8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0
0 0 Message size received from all nodes
0 0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 95;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 8.07E+05
6.085E+08 1 2808
608546112 .TAU application
94.0
7.072E+06 5.72E+08 1000 2008
572038 DO_FULLADVANCE
92.4
2.2E+08 5.624E+08 1000 28000
562391 TIMESTEP_SSP_322
32.8
1.846E+08 1.997E+08 3000 6000 66568 GET_KINETIC_ADVANCE
16.6
1.007E+08 1.007E+08 24000 0 4197 MPI_Alltoall()
6.9
3.644E+07 4.197E+07 1001
2002 41931
GET_FIELD_EXPLICIT
4.3
2.614E+07 2.614E+07 10979 0 2381 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16172015 MAKE_LAMBDA_GRID
1.4
6.962E+06 8.284E+06 1
770 8283992
MAKE_MAXWELL_MATRIX
0.7
3.94E+06 4.334E+06 1 130
4334235 MAKE_POISSON_MATRIX
0.6
3.384E+06 3.384E+06 1122 0 3016 MPI_Bcast()
0.4
1.447E+05 2.19E+06 1000 1000 2190 GET_ERROR
0.3
7.878E+04 1.607E+06 1000 1000 1607 CATCH_HALT_SIGNAL
0.1
6.927E+05 6.927E+05 1 0
692696 MAKE_THETA_OPERATORS
0.1
1.578E+05 5.659E+05 1 257
565911 MAKE_BLEND_ARRAYS
0.1
2.346E+05 4.833E+05 10 160 48326 WRITE_BIG
0.1
4.354E+05 4.658E+05 1 6
465789 MAKE_IMPLICIT_ADVECT
0.1 4.361E+05
4.361E+05 1 0
436084 MAKE_RADIAL_OPERATORS
0.0
396 8.844E+04 1 2 88436 WRITE_INPUT
0.0
6.767E+04 6.787E+04 1 8 67868 MPI_Init()
0.0
3.053E+04 3.053E+04 2 0 15264 MPI_Comm_split()
0.0
5042 5149 1 4 5149 MPI_Finalize()
0.0
4138 4138 12 0 345 MPI_Reduce()
0.0
1711 1711 1 0 1711 MAKE_PROFILES
0.0
107 107 4 0 27 MPI_Keyval_free()
0.0
105 105 4 0 26 MPI_Attr_put()
0.0
96 96 4 0 24 MPI_Attr_get()
0.0
60 60 4 0 15 MPI_Comm_rank()
0.0
26 26 2 0 13 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
95, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
0
0 0 0 0
Message size sent to all nodes
---------------------------------------------------------------------------------------
NODE 96;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.788E+05
6.094E+08 1 2808
609396315 .TAU application
94.0
7.185E+06 5.731E+08 1000 3008
573110 DO_FULLADVANCE
92.4
2.217E+08 5.633E+08 1000 28000
563253 TIMESTEP_SSP_322
32.6
1.843E+08 1.988E+08 3000 6000 66278 GET_KINETIC_ADVANCE
16.7
1.016E+08 1.016E+08 24000 0 4233 MPI_Alltoall()
6.8
3.637E+07 4.121E+07 1001
2002 41165 GET_FIELD_EXPLICIT
4.1
2.487E+07 2.487E+07 10979 0 2265 MPI_Allreduce()
2.7
1.617E+07 1.617E+07 1 0
16172058 MAKE_LAMBDA_GRID
1.4
6.955E+06 8.277E+06 1 770
8276604 MAKE_MAXWELL_MATRIX
0.7
3.939E+06 4.388E+06 1 130
4388213 MAKE_POISSON_MATRIX
0.5
3.255E+06 3.255E+06 1122 0 2901 MPI_Bcast()
0.3
1.479E+05 2.09E+06 1000 1000 2090 GET_ERROR
0.2
6.415E+04 1.453E+06 1000 1000 1453 CATCH_HALT_SIGNAL
0.1
6.935E+05 6.935E+05 1 0
693464 MAKE_THETA_OPERATORS
0.1
1.568E+05 5.713E+05 1 257
571335 MAKE_BLEND_ARRAYS
0.1
2.386E+05 5.011E+05 10 196 50109 WRITE_BIG
0.1
3.976E+05 3.976E+05 1 0
397553 MAKE_RADIAL_OPERATORS
0.1
3.399E+05 3.715E+05 1 6
371501 MAKE_IMPLICIT_ADVECT
0.0
1.89E+05 1.89E+05 1037 0 182 MPI_Send()
0.0
589 8.939E+04 1 3 89389 WRITE_INPUT
0.0
7.056E+04 7.076E+04 1 8 70764 MPI_Init()
0.0
3.062E+04 3.062E+04 2 0 15311 MPI_Comm_split()
0.0
1.019E+04 1.019E+04 12 0 849 MPI_Reduce()
0.0
5614 5723 1 4 5723 MPI_Finalize()
0.0
1737 1737 1 0 1737 MAKE_PROFILES
0.0
109 109 4 0 27 MPI_Attr_put()
0.0
109 109 4 0 27 MPI_Keyval_free()
0.0 98 98 4 0 24 MPI_Attr_get()
0.0
61 61 4 0 15 MPI_Comm_rank()
0.0
27 27 2 0 14 MPI_Comm_size()
---------------------------------------------------------------------------------------
USER EVENTS Profile :NODE
96, CONTEXT 0, THREAD 0
---------------------------------------------------------------------------------------
NumSamples MaxValue
MinValue MeanValue Std. Dev.
Event Name
---------------------------------------------------------------------------------------
2.4E+04
4096 4096 4096 0
Message size for all-to-all
1122 8 4
4.232 0.9345 Message size for broadcast
1.098E+04
4.915E+04 8 9349 5122
Message size for gather
12 16 8
14.67 2.981 Message size for reduce
0 0 0 0 0
Message size received from all nodes
1037
8192 8 111.5
761.7 Message size sent to all
nodes
---------------------------------------------------------------------------------------
NODE 97;CONTEXT 0;THREAD 0:
---------------------------------------------------------------------------------------
%Time Exclusive
Inclusive #Call #Subrs Count/Call Name
counts total counts
---------------------------------------------------------------------------------------
100.0 7.933E+05
6.2E+08 1 2808
620030383 .TAU application
94.0
7.068E+06 5.828E+08 1000 2008
582795 DO_FULLADVANCE
91.9
2.225E+08 5.698E+08 1000 28000
569787 TIMESTEP_SSP_322
32.3
1.853E+08 2.005E+08 3000 6000 66819 GET_KINETIC_ADVANCE
17.0
1.052E+08 1.052E+08 24000 0 4384 MPI_Alltoall()
6.7
3.662E+07 4.165E+07 1001 2002 41603 GET_FIELD_EXPLICIT
4.7
2.901E+07 2.901E+07 10979 0 2642 MPI_Allreduce()
2.6
1.62E+07 1.62E+07 1 0
16199850 MAKE_LAMBDA_GRID
1.3
6.979E+06 8.27E+06 1 770
8269903 MAKE_MAXWELL_MATRIX
0.9
1.432E+05 5.546E+06 1000 1000 5546 GET_ERROR
0.7
3.958E+06 4.288E+06 1 130
4287502 MAKE_POISSON_MATRIX
0.7 4.214E+06 4.214E+06 1122 0 3756 MPI_Bcast()
0.2
6.699E+04 1.547E+06 1000 1000 1547 CATCH_HALT_SIGNAL
0.1
7.015E+05 7.015E+05 1 0
701518 MAKE_THETA_OPERATORS
0.1
1.59E+05 5.748E+05 1 257
574753 MAKE_BLEND_ARRAYS
0.1
2.351E+05 4.994E+05 10 160 49944 WRITE_BIG
0.1
4.016E+05 4.016E+05 1 0
401621 MAKE_RADIAL_OPERATORS
0.1
3.468E+05 3.791E+05 1 6
379139 MAKE_IMPLICIT_ADVECT
0.0
402 1.337E+05 1 2
133727 WRITE_INPUT
0.0
6.798E+04 6.819E+04 1 8 68192 MPI_Init()
0.0
4.027E+04 4.027E+04 2 0 20133 MPI_Comm_split()
0.0
5137 5233 1 4 5233 MPI_Finalize()
0.0
2815 2815 12 0 235 MPI_Reduce()
0.0
1738 1738 1
0 1738 MAKE_PROFILES