Key:-c [--collop_time_per_rank]
Available options: -C [--communicators], -M [--comm_id]
This charts shows the time each rank spent in MPI collective operations. Extend the chart using available options:
$ aps-report ./aps_result_<postfix> -c
| Collective Operations Time and Communicators for all ranks |-------------------------------------------------------------- | Rank LifeTime(sec) CollOp Time(sec) CollOp Time(%) |--------------------------------------------------------- 0003 3.48 2.32 66.73 0002 3.48 2.21 63.38 0001 3.48 2.08 59.72 0000 4.53 2.52 55.67 |========================================================= | TOTAL 14.96 9.12 60.98 | AVG 3.74 2.28 60.98 |
This charts shows the time each rank spent in MPI collective operation in a specific communicator.
$ aps-report ./aps_result_<postfix> -c -M 1
| Collective Operations Time and Communicators for all ranks |-------------------------------------------------------------- | Rank LifeTime(sec) CollOp Time(sec) CollOp Time(%) |--------------------------------------------------------- 0003 3.48 2.32 66.73 0002 3.48 2.21 63.38 0001 3.48 2.08 59.72 0000 4.53 2.52 55.67 |========================================================= | TOTAL 14.96 9.12 60.98 | AVG 3.74 2.28 60.98
This chart shows the time spent by each MPI collective operation in all communicators.
$ aps-report ./aps_result_<postfix> -c -C
Collective Operations Time and Communicators for all ranks |-------------------------------------------------------------- |-------------------------------------------------------------------------------------------------------------------------- | Function Time(sec) Time(%) Volume(MB) Volume(%) Calls AvrMsgSize(B) |-------------------------------------------------------------------------------------------------------------------------- MPI_Allreduce 3.61 24.11 4157.97 30.00 198016 22018 MPI_Alltoall 2.99 20.00 4157.99 30.00 219396 19873 MPI_Gather 1.28 8.55 2772.00 20.00 220228 13198 MPI_Reduce 1.24 8.27 2771.97 20.00 195384 14876 MPI_Bcast 0.01 0.04 0.00 0.00 612 2 |========================================================================================================================== | TOTAL 9.12 60.98 13859.93 100.00 833636 17433 | |-------------------------------------------------------------------------------------------------------------------------- | Function Time(sec) Time(%) Volume(MB) Volume(%) Calls MsgSize(B) |-------------------------------------------------------------------------------------------------------------------------- MPI_Allreduce 3.61 24.11 4157.97 100.00 198016 ALL Comm.: Id Size 1 4 3.11 86.11 2771.98 66.67 132192 ALL 0 2 0.50 13.89 1385.99 33.33 65824 ALL |-------------------------------------------------------------------------------------------------------------------------- MPI_Alltoall 2.99 20.00 4157.99 100.00 219396 ALL Comm.: Id Size 1 4 2.63 87.74 2771.99 66.67 146264 ALL 0 2 0.37 12.26 1386.00 33.33 73132 ALL |-------------------------------------------------------------------------------------------------------------------------- MPI_Gather 1.28 8.55 2772.00 100.00 220228 ALL Comm.: Id Size 1 4 0.99 77.15 2079.00 75.00 147096 ALL 0 2 0.29 22.85 693.00 25.00 73132 ALL |-------------------------------------------------------------------------------------------------------------------------- MPI_Reduce 1.24 8.27 2771.97 100.00 195384 ALL Comm.: Id Size 1 4 0.96 77.61 2078.98 75.00 130256 ALL 0 2 0.28 22.39 692.99 25.00 65128 ALL |-------------------------------------------------------------------------------------------------------------------------- MPI_Bcast 0.01 0.04 0.00 100.00 612 ALL Comm.: Id Size 1 4 0.01 95.90 0.00 50.21 380 ALL 0 2 0.00 4.10 0.00 49.79 232 ALL |==========================================================================================================================
-c and -c -M Columns | |
---|---|
-c -C Columns | |
Rows | |
Rank | Rank number |
LifeTime(sec) | Total execution time of the rank (in seconds) |
CollOpTime(sec) | Time spent in MPI collective operations (in seconds) |
CollOpTime(%) | Percentage of the MPI collective operations time in the application lifetime |
Rank | Rank number |
LifeTime(sec) | Total execution time of the rank (in seconds) |
CollOpTime(sec) | Time spent in MPI collective operations (in seconds) |
CollOpTime(%) | Percentage of the MPI collective operations time in the application lifetime |
Function | Function name |
Time(sec) | Total time of the function execution in all ranks (in seconds) |
Time(%) | Percentage of the function execution time in the application lifetime |
Idle(sec) | The time when the collective operation was idle (in seconds) |
Idle(%) | Percentage of the time when the operation was idle |
Volume(MB) | Amount of data transferred by the function in all ranks (in megabytes) |
Volume(%) | Percentage of the data transferred by the function |
Calls | Number of calls of this function in all ranks |
AvrMsgSize(B) | Average size of messages |
TOTAL | Total values for all columns |
AVG | Average values for all columns |