aryansingh/gpt-2-inference-engine
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ECE 408 sem project

Browse Source
This commit is contained in:
Aryan Singh committed 2026-06-07 13:43:20 -05:00
1 parent 1d79ccd2e3
commit fa3efcaf07
139 files changed
+11634 -1
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

output/perf/ncu/verify_op_11.err
View File
Whitespace-only changes.
output/perf/ncu/verify_op_11.out
+173
View File
@@ -0,0 +1,173 @@
job 18002677 starting on gpub089.delta.ncsa.illinois.edu
==PROF== Connected to process 810795 (/u/asingh41/sp26_ece408_Temperature1.5/build/test_gpt2_op_11)
==PROF== Profiling "encoder_forward_kernel" - 0: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 1: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 2: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 3: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 4: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 5: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 6: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 7: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 8: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 9: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 10: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 11: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 12: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 13: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 14: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 15: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 16: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 17: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 18: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 19: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 20: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 21: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 22: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 23: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 24: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 25: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 26: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 27: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 28: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 29: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 30: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 31: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 32: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 33: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 34: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 35: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 36: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 37: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 38: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 39: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 40: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 41: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 42: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 43: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 44: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 45: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 46: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 47: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 48: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 49: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 50: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 51: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 52: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 53: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 54: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 55: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 56: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 57: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 58: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 59: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 60: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 61: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 62: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 63: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 64: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 65: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 66: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 67: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 68: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 69: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 70: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 71: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 72: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 73: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 74: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 75: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 76: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 77: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 78: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 79: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 80: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 81: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 82: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 83: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 84: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 85: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 86: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 87: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 88: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 89: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 90: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 91: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 92: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 93: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 94: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 95: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 96: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 97: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 98: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 99: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 100: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 101: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 102: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 103: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 104: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 105: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 106: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 107: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 108: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 109: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 110: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 111: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 112: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 113: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 114: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 115: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 116: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 117: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 118: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 119: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 120: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 121: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 122: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 123: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 124: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 125: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 126: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 127: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 128: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 129: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 130: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 131: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 132: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 133: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 134: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 135: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 136: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 137: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 138: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 139: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 140: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 141: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 142: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 143: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 144: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 145: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 146: 0%
==WARNING== Launching the workload is taking more time than expected. If this continues to hang, terminate the profile and re-try by profiling the range of all related launches using '--replay-mode range'. See https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html#replay for more details.
....50%....100% - 40 passes
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000069, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
==PROF== Disconnected from process 810795
==PROF== Report: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/analysis_op_11.ncu-rep
output/perf/ncu/verify_op_12.err
View File
Whitespace-only changes.
output/perf/ncu/verify_op_12.out
+197
View File
@@ -0,0 +1,197 @@
job 18002228 starting on gpub022.delta.ncsa.illinois.edu
==PROF== Connected to process 355678 (/u/asingh41/sp26_ece408_Temperature1.5/build/test_gpt2_op_12)
==PROF== Profiling "encoder_forward_kernel" - 0: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 1: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 2: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 3: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 4: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 5: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 6: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 7: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 8: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 9: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 10: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 11: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 12: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 13: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 14: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 15: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 16: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 17: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 18: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 19: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 20: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 21: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 22: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 23: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 24: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 25: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 26: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 27: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 28: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 29: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 30: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 31: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 32: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 33: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 34: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 35: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 36: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 37: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 38: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 39: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 40: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 41: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 42: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 43: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 44: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 45: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 46: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 47: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 48: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 49: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 50: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 51: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 52: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 53: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 54: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 55: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 56: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 57: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 58: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 59: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 60: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 61: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 62: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 63: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 64: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 65: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 66: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 67: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 68: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 69: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 70: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 71: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 72: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 73: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 74: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 75: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 76: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 77: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 78: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 79: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 80: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 81: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 82: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 83: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 84: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 85: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 86: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 87: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 88: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 89: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 90: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 91: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 92: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 93: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 94: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 95: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 96: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 97: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 98: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 99: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 100: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 101: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 102: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 103: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 104: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 105: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 106: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 107: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 108: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 109: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 110: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 111: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 112: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 113: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 114: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 115: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 116: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 117: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 118: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 119: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 120: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 121: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 122: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 123: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 124: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 125: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 126: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 127: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 128: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 129: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 130: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 131: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 132: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 133: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 134: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 135: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 136: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 137: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 138: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 139: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 140: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 141: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 142: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 143: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 144: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 145: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 146: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 147: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 148: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 149: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 150: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 151: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 152: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 153: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 154: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 155: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 156: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 157: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 158: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 159: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 160: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 161: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 162: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 163: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 164: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 165: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 166: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 167: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 168: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 169: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_kernel" - 170: 0%
==WARNING== Launching the workload is taking more time than expected. If this continues to hang, terminate the profile and re-try by profiling the range of all related launches using '--replay-mode range'. See https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html#replay for more details.
....50%....100% - 40 passes
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431793
-39.836346, -39.836502
-43.065910, -43.066078
-42.828045, -42.828197
-43.529541, -43.529701
-44.318398, -44.318558
-41.227425, -41.227592
-41.270760, -41.270931
-42.541393, -42.541584
-42.394997, -42.395176
---------------- Logits OK! (max err 0.000050, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
==PROF== Disconnected from process 355678
==PROF== Report: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/analysis_op_12.ncu-rep
output/perf/ncu/verify_op_13.err
View File
Whitespace-only changes.
output/perf/ncu/verify_op_13.out
+197
View File
@@ -0,0 +1,197 @@
job 18003871 starting on gpub081.delta.ncsa.illinois.edu
==PROF== Connected to process 459778 (/u/asingh41/sp26_ece408_Temperature1.5/build/test_gpt2_op_13)
==PROF== Profiling "encoder_forward_kernel" - 0: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 1: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 2: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 3: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 4: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 5: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 6: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 7: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 8: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 9: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 10: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 11: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 12: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 13: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 14: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 15: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 16: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 17: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 18: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 19: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 20: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 21: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 22: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 23: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 24: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 25: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 26: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 27: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 28: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 29: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 30: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 31: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 32: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 33: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 34: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 35: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 36: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 37: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 38: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 39: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 40: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 41: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 42: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 43: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 44: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 45: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 46: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 47: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 48: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 49: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 50: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 51: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 52: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 53: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 54: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 55: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 56: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 57: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 58: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 59: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 60: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 61: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 62: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 63: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 64: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 65: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 66: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 67: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 68: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 69: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 70: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 71: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 72: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 73: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 74: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 75: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 76: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 77: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 78: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 79: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 80: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 81: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 82: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 83: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 84: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 85: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 86: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 87: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 88: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 89: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 90: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 91: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 92: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 93: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 94: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 95: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 96: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 97: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 98: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 99: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 100: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 101: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 102: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 103: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 104: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 105: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 106: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 107: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 108: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 109: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 110: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 111: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 112: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 113: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 114: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 115: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 116: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 117: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 118: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 119: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 120: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 121: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 122: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 123: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 124: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 125: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 126: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 127: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 128: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 129: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 130: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 131: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 132: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 133: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 134: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 135: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 136: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 137: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 138: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 139: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 140: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 141: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 142: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 143: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 144: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 145: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 146: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 147: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 148: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 149: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 150: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 151: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 152: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 153: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 154: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 155: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 156: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 157: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 158: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 159: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 160: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 161: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 162: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 163: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 164: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 165: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 166: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 167: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 168: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 169: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 170: 0%
==WARNING== Launching the workload is taking more time than expected. If this continues to hang, terminate the profile and re-try by profiling the range of all related launches using '--replay-mode range'. See https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html#replay for more details.
....50%....100% - 40 passes
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000043, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
==PROF== Disconnected from process 459778
==PROF== Report: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/analysis_op_13.ncu-rep
output/perf/ncu/verify_op_16.err
View File
Whitespace-only changes.
output/perf/ncu/verify_op_16.out
+231
View File
@@ -0,0 +1,231 @@
job 18004587 starting on gpub005.delta.ncsa.illinois.edu
==PROF== Connected to process 408869 (/u/asingh41/sp26_ece408_Temperature1.5/build/test_gpt2_op_16)
==PROF== Profiling "encoder_forward_kernel" - 0: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 1: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 2: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 3: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 4: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 5: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 6: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 7: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 8: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 9: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 10: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 11: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 12: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 13: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 14: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 15: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 16: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 17: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 18: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 19: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 20: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 21: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 22: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 23: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 24: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 25: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 26: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 27: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 28: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 29: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 30: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 31: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 32: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 33: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 34: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 35: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 36: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 37: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 38: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 39: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 40: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 41: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 42: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 43: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 44: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 45: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 46: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 47: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 48: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 49: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 50: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 51: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 52: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 53: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 54: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 55: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 56: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 57: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 58: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 59: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 60: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 61: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 62: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 63: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 64: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 65: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 66: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 67: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 68: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 69: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 70: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 71: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 72: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 73: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 74: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 75: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 76: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 77: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 78: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 79: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 80: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 81: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 82: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 83: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 84: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 85: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 86: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 87: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 88: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 89: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 90: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 91: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 92: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 93: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 94: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 95: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 96: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 97: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 98: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 99: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 100: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 101: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 102: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 103: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 104: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 105: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 106: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 107: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 108: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 109: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 110: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 111: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 112: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 113: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 114: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 115: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 116: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 117: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 118: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 119: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 120: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 121: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 122: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 123: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 124: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 125: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 126: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 127: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 128: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 129: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 130: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 131: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 132: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 133: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 134: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 135: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 136: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 137: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 138: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 139: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 140: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 141: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 142: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 143: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 144: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 145: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 146: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 147: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 148: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 149: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 150: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 151: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 152: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 153: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 154: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 155: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 156: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 157: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 158: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 159: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 160: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 161: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 162: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 163: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 164: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 165: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 166: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 167: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 168: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 169: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 170: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 171: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 172: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 173: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 174: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 175: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 176: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 177: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 178: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 179: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 180: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 181: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 182: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 183: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 184: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 185: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 186: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 187: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 188: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 189: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 190: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 191: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 192: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 193: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 194: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 195: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x32_sliced1x4_tn" - 196: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 197: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 198: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 199: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 200: 0%....50%....100% - 40 passes
==PROF== Profiling "Kernel" - 201: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_64x64_tn" - 202: 0%....50%....100% - 40 passes
==PROF== Profiling "bias_add_filler_kernel" - 203: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 204: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 205: 0%....50%....100% - 40 passes
==PROF== Profiling "ampere_sgemm_128x64_tn" - 206: 0%....50%....100% - 40 passes
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.315117
-39.836346, -39.732384
-43.065910, -42.960098
-42.828045, -42.723515
-43.529541, -43.418613
-44.318398, -44.210033
-41.227425, -41.119377
-41.270760, -41.164490
-42.541393, -42.428894
-42.394997, -42.288929
---------------- Logits OK! (max err 0.082334, RMSE 0.002728) ----------------
_________________________ All tests passed! _________________________
==PROF== Disconnected from process 408869
==PROF== Report: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/analysis_op_16.ncu-rep
output/perf/ncu/verify_op_8.err
View File
Whitespace-only changes.
output/perf/ncu/verify_op_8.out
+197
View File
@@ -0,0 +1,197 @@
job 17968511 starting on gpub005.delta.ncsa.illinois.edu
==PROF== Connected to process 3198281 (/u/asingh41/sp26_ece408_Temperature1.5/build/test_gpt2_op_8)
==PROF== Profiling "encoder_forward_kernel" - 0: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 1: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 2: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 3: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 4: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 5: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 6: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 7: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 8: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 9: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 10: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 11: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 12: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 13: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 14: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 15: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 16: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 17: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 18: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 19: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 20: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 21: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 22: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 23: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 24: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 25: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 26: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 27: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 28: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 29: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 30: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 31: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 32: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 33: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 34: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 35: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 36: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 37: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 38: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 39: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 40: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 41: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 42: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 43: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 44: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 45: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 46: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 47: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 48: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 49: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 50: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 51: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 52: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 53: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 54: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 55: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 56: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 57: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 58: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 59: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 60: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 61: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 62: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 63: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 64: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 65: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 66: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 67: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 68: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 69: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 70: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 71: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 72: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 73: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 74: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 75: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 76: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 77: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 78: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 79: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 80: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 81: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 82: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 83: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 84: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 85: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 86: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 87: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 88: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 89: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 90: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 91: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 92: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 93: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 94: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 95: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 96: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 97: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 98: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 99: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 100: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 101: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 102: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 103: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 104: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 105: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 106: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 107: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 108: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 109: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 110: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 111: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 112: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 113: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 114: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 115: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 116: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 117: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 118: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 119: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 120: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 121: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 122: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 123: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 124: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 125: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 126: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 127: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 128: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 129: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 130: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 131: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 132: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 133: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 134: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 135: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 136: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 137: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 138: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 139: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 140: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 141: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 142: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 143: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 144: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 145: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 146: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 147: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 148: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 149: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 150: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 151: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 152: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 153: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 154: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 155: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 156: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 157: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 158: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 159: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 160: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 161: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 162: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 163: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 164: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 165: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 166: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 167: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 168: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 169: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 170: 0%
==WARNING== Launching the workload is taking more time than expected. If this continues to hang, terminate the profile and re-try by profiling the range of all related launches using '--replay-mode range'. See https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html#replay for more details.
....50%....100% - 40 passes
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000043, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
==PROF== Disconnected from process 3198281
==PROF== Report: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/analysis_op_8.ncu-rep
output/perf/ncu/verify_op_9.err
View File
Whitespace-only changes.
output/perf/ncu/verify_op_9.out
+197
View File
@@ -0,0 +1,197 @@
job 17968384 starting on gpub015.delta.ncsa.illinois.edu
==PROF== Connected to process 1935467 (/u/asingh41/sp26_ece408_Temperature1.5/build/test_gpt2_op_9)
==PROF== Profiling "encoder_forward_kernel" - 0: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 1: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 2: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 3: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 4: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 5: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 6: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 7: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 8: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 9: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 10: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 11: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 12: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 13: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 14: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 15: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 16: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 17: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 18: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 19: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 20: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 21: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 22: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 23: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 24: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 25: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 26: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 27: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 28: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 29: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 30: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 31: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 32: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 33: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 34: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 35: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 36: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 37: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 38: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 39: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 40: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 41: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 42: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 43: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 44: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 45: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 46: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 47: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 48: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 49: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 50: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 51: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 52: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 53: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 54: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 55: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 56: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 57: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 58: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 59: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 60: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 61: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 62: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 63: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 64: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 65: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 66: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 67: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 68: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 69: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 70: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 71: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 72: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 73: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 74: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 75: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 76: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 77: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 78: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 79: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 80: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 81: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 82: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 83: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 84: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 85: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 86: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 87: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 88: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 89: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 90: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 91: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 92: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 93: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 94: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 95: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 96: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 97: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 98: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 99: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 100: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 101: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 102: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 103: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 104: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 105: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 106: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 107: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 108: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 109: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 110: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 111: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 112: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 113: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 114: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 115: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 116: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 117: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 118: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 119: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 120: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 121: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 122: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 123: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 124: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 125: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 126: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 127: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 128: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 129: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 130: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 131: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 132: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 133: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 134: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 135: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 136: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 137: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 138: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 139: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 140: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 141: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 142: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 143: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 144: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 145: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 146: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 147: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 148: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 149: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 150: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 151: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 152: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 153: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 154: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 155: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 156: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 157: 0%....50%....100% - 40 passes
==PROF== Profiling "preattention_scores_kernel" - 158: 0%....50%....100% - 40 passes
==PROF== Profiling "softmax_forward_kernel" - 159: 0%....50%....100% - 40 passes
==PROF== Profiling "attention_v_kernel" - 160: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 161: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 162: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 163: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 164: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 165: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 166: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 167: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 168: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 169: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 170: 0%
==WARNING== Launching the workload is taking more time than expected. If this continues to hang, terminate the profile and re-try by profiling the range of all related launches using '--replay-mode range'. See https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html#replay for more details.
....50%....100% - 40 passes
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000043, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
==PROF== Disconnected from process 1935467
==PROF== Report: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/analysis_op_9.ncu-rep
output/perf/ncu/verify_req_4.err
View File
Whitespace-only changes.
output/perf/ncu/verify_req_4.out
+173
View File
@@ -0,0 +1,173 @@
job 17971939 starting on gpub022.delta.ncsa.illinois.edu
==PROF== Connected to process 3614659 (/u/asingh41/sp26_ece408_Temperature1.5/build/test_gpt2_req_4)
==PROF== Profiling "encoder_forward_kernel" - 0: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 1: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 2: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 3: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 4: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 5: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 6: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 7: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 8: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 9: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 10: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 11: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 12: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 13: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 14: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 15: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 16: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 17: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 18: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 19: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 20: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 21: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 22: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 23: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 24: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 25: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 26: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 27: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 28: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 29: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 30: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 31: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 32: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 33: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 34: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 35: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 36: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 37: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 38: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 39: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 40: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 41: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 42: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 43: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 44: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 45: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 46: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 47: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 48: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 49: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 50: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 51: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 52: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 53: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 54: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 55: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 56: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 57: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 58: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 59: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 60: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 61: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 62: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 63: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 64: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 65: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 66: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 67: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 68: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 69: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 70: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 71: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 72: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 73: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 74: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 75: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 76: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 77: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 78: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 79: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 80: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 81: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 82: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 83: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 84: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 85: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 86: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 87: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 88: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 89: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 90: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 91: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 92: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 93: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 94: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 95: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 96: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 97: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 98: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 99: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 100: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 101: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 102: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 103: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 104: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 105: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 106: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 107: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 108: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 109: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 110: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 111: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 112: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 113: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 114: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 115: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 116: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 117: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 118: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 119: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 120: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 121: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 122: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 123: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 124: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 125: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 126: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 127: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 128: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 129: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 130: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 131: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 132: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 133: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 134: 0%....50%....100% - 40 passes
==PROF== Profiling "permute_kernel" - 135: 0%....50%....100% - 40 passes
==PROF== Profiling "flash_attention_forward_kernel" - 136: 0%....50%....100% - 40 passes
==PROF== Profiling "unpermute_kernel" - 137: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 138: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 139: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 140: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 141: 0%....50%....100% - 40 passes
==PROF== Profiling "gelu_forward_kernel" - 142: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 143: 0%....50%....100% - 40 passes
==PROF== Profiling "residual_forward_kernel" - 144: 0%....50%....100% - 40 passes
==PROF== Profiling "layernorm_forward_kernel" - 145: 0%....50%....100% - 40 passes
==PROF== Profiling "matmul_forward_kernel" - 146: 0%
==WARNING== Launching the workload is taking more time than expected. If this continues to hang, terminate the profile and re-try by profiling the range of all related launches using '--replay-mode range'. See https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html#replay for more details.
....50%....100% - 40 passes
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000069, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
==PROF== Disconnected from process 3614659
==PROF== Report: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/analysis_req_4.ncu-rep
output/perf/nsys/verify_op_11.err
+9
View File
@@ -0,0 +1,9 @@
WARNING: Device-side CUDA Event completion trace is currently enabled.
This may increase runtime overhead and the likelihood of false
dependencies across CUDA Streams. If you wish to avoid this, please
disable the feature with --cuda-event-trace=false.
Failed to create '/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_11.nsys-rep': File exists.
Use `--force-overwrite true` to overwrite existing files.
Failed to create '/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_11.sqlite': File exists.
Use `--force-overwrite true` to overwrite existing files.
SKIPPED: /tmp/nsys-report-472f.sqlite does not contain NV Tools Extension (NVTX) data.
output/perf/nsys/verify_op_11.out
+107
View File
@@ -0,0 +1,107 @@
job 18002676 starting on gpub022.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000069, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
Collecting data...
Generating '/tmp/nsys-report-4b2b.qdstrm'
␍[1/8] [0% ] nsys-report-d663.nsys-rep␍[1/8] [0% ] nsys-report-d663.nsys-rep␍[1/8] [6% ] nsys-report-d663.nsys-rep␍[1/8] [=====29% ] nsys-report-d663.nsys-rep␍[1/8] [===========52% ] nsys-report-d663.nsys-rep␍[1/8] [===========53% ] nsys-report-d663.nsys-rep␍[1/8] [=====32% ] nsys-report-d663.nsys-rep␍[1/8] [===========52% ] nsys-report-d663.nsys-rep␍[1/8] [===========53% ] nsys-report-d663.nsys-rep␍[1/8] [============54% ] nsys-report-d663.nsys-rep␍[1/8] [============57% ] nsys-report-d663.nsys-rep␍[1/8] [=============58% ] nsys-report-d663.nsys-rep␍[1/8] [=============59% ] nsys-report-d663.nsys-rep␍[1/8] [=================74% ] nsys-report-d663.nsys-rep␍[1/8] [======================92% ] nsys-report-d663.nsys-rep␍[1/8] [========================100%] nsys-report-d663.nsys-rep␍[1/8] [========================100%] nsys-report-d663.nsys-rep
[2/8] [0% ] nsys-report-472f.sqlite [2/8] [1% ] nsys-report-472f.sqlite [2/8] [2% ] nsys-report-472f.sqlite [2/8] [3% ] nsys-report-472f.sqlite [2/8] [4% ] nsys-report-472f.sqlite [2/8] [5% ] nsys-report-472f.sqlite [2/8] [6% ] nsys-report-472f.sqlite [2/8] [7% ] nsys-report-472f.sqlite [2/8] [8% ] nsys-report-472f.sqlite [2/8] [9% ] nsys-report-472f.sqlite [2/8] [10% ] nsys-report-472f.sqlite [2/8] [11% ] nsys-report-472f.sqlite [2/8] [12% ] nsys-report-472f.sqlite [2/8] [13% ] nsys-report-472f.sqlite [2/8] [14% ] nsys-report-472f.sqlite [2/8] [=15% ] nsys-report-472f.sqlite [2/8] [=16% ] nsys-report-472f.sqlite [2/8] [=17% ] nsys-report-472f.sqlite [2/8] [==18% ] nsys-report-472f.sqlite [2/8] [==19% ] nsys-report-472f.sqlite [2/8] [==20% ] nsys-report-472f.sqlite [2/8] [==21% ] nsys-report-472f.sqlite [2/8] [===22% ] nsys-report-472f.sqlite [2/8] [===23% ] nsys-report-472f.sqlite [2/8] [===24% ] nsys-report-472f.sqlite [2/8] [====25% ] nsys-report-472f.sqlite [2/8] [====26% ] nsys-report-472f.sqlite [2/8] [====27% ] nsys-report-472f.sqlite [2/8] [====28% ] nsys-report-472f.sqlite [2/8] [=====29% ] nsys-report-472f.sqlite [2/8] [=====30% ] nsys-report-472f.sqlite [2/8] [=====31% ] nsys-report-472f.sqlite [2/8] [=====32% ] nsys-report-472f.sqlite [2/8] [======33% ] nsys-report-472f.sqlite [2/8] [======34% ] nsys-report-472f.sqlite [2/8] [======35% ] nsys-report-472f.sqlite [2/8] [=======36% ] nsys-report-472f.sqlite [2/8] [=======37% ] nsys-report-472f.sqlite [2/8] [=======38% ] nsys-report-472f.sqlite [2/8] [=======39% ] nsys-report-472f.sqlite [2/8] [========40% ] nsys-report-472f.sqlite [2/8] [========41% ] nsys-report-472f.sqlite [2/8] [========42% ] nsys-report-472f.sqlite [2/8] [=========43% ] nsys-report-472f.sqlite [2/8] [=========44% ] nsys-report-472f.sqlite [2/8] [=========45% ] nsys-report-472f.sqlite [2/8] [=========46% ] nsys-report-472f.sqlite [2/8] [==========47% ] nsys-report-472f.sqlite [2/8] [==========48% ] nsys-report-472f.sqlite [2/8] [==========49% ] nsys-report-472f.sqlite [2/8] [===========50% ] nsys-report-472f.sqlite [2/8] [===========51% ] nsys-report-472f.sqlite [2/8] [===========52% ] nsys-report-472f.sqlite [2/8] [===========53% ] nsys-report-472f.sqlite [2/8] [============54% ] nsys-report-472f.sqlite [2/8] [============55% ] nsys-report-472f.sqlite [2/8] [============56% ] nsys-report-472f.sqlite [2/8] [============57% ] nsys-report-472f.sqlite [2/8] [=============58% ] nsys-report-472f.sqlite [2/8] [=============59% ] nsys-report-472f.sqlite [2/8] [=============60% ] nsys-report-472f.sqlite [2/8] [==============61% ] nsys-report-472f.sqlite [2/8] [==============62% ] nsys-report-472f.sqlite [2/8] [==============63% ] nsys-report-472f.sqlite [2/8] [==============64% ] nsys-report-472f.sqlite [2/8] [===============65% ] nsys-report-472f.sqlite [2/8] [===============66% ] nsys-report-472f.sqlite [2/8] [===============67% ] nsys-report-472f.sqlite [2/8] [================68% ] nsys-report-472f.sqlite [2/8] [================69% ] nsys-report-472f.sqlite [2/8] [================70% ] nsys-report-472f.sqlite [2/8] [================71% ] nsys-report-472f.sqlite [2/8] [=================72% ] nsys-report-472f.sqlite [2/8] [=================73% ] nsys-report-472f.sqlite [2/8] [=================74% ] nsys-report-472f.sqlite [2/8] [==================75% ] nsys-report-472f.sqlite [2/8] [==================76% ] nsys-report-472f.sqlite [2/8] [==================77% ] nsys-report-472f.sqlite [2/8] [==================78% ] nsys-report-472f.sqlite [2/8] [===================79% ] nsys-report-472f.sqlite [2/8] [===================80% ] nsys-report-472f.sqlite [2/8] [===================81% ] nsys-report-472f.sqlLine truncated
[3/8] Executing 'nvtx_sum' stats report
[4/8] Executing 'osrt_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ----------------------
65.5 1046221746 20 52311087.3 36977154.5 1343 144034477 47744624.9 poll
25.3 403984081 582 694130.7 495337.5 1012 19219062 1320637.1 ioctl
7.8 124190546 6 20698424.3 67121.0 7685 115338096 46492858.4 fread
0.5 7610123 1 7610123.0 7610123.0 7610123 7610123 0.0 pthread_cond_wait
0.4 7042952 1 7042952.0 7042952.0 7042952 7042952 0.0 pthread_cond_broadcast
0.1 2350546 25 94021.8 9598.0 8175 1635631 323841.6 mmap64
0.1 1607267 38 42296.5 6812.5 3627 684381 126942.5 fopen
0.1 872311 11 79301.0 27341.0 1052 372016 110728.5 write
0.0 662102 32 20690.7 2364.0 1362 311784 71411.8 fclose
0.0 599354 17 35256.1 4849.0 2225 375633 91066.5 mmap
0.0 363971 2 181985.5 181985.5 164879 199092 24192.2 pthread_create
0.0 355093 55 6456.2 6452.0 2405 12613 2601.2 open64
0.0 207257 1 207257.0 207257.0 207257 207257 0.0 fflush
0.0 77074 1 77074.0 77074.0 77074 77074 0.0 fgets
0.0 39155 8 4894.4 5225.0 2806 6762 1570.3 munmap
0.0 36088 6 6014.7 5986.5 3056 8997 2029.8 open
0.0 21803 3 7267.7 8657.0 3928 9218 2905.8 pipe2
0.0 18696 2 9348.0 9348.0 5150 13546 5936.9 socket
0.0 17072 8 2134.0 1773.5 1132 3908 1047.0 read
0.0 10660 1 10660.0 10660.0 10660 10660 0.0 connect
0.0 6462 1 6462.0 6462.0 6462 6462 0.0 fopen64
0.0 3516 2 1758.0 1758.0 1412 2104 489.3 fwrite
0.0 3275 3 1091.7 1082.0 1071 1122 26.8 fcntl
0.0 1634 1 1634.0 1634.0 1634 1634 0.0 bind
[5/8] Executing 'cuda_api_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ---------------------------------
85.9 292488145 3 97496048.3 36522068.0 19767 255946310 138430374.4 cudaMemcpy
5.8 19849256 147 135029.0 2775.0 2665 18870790 1556382.3 cudaLaunchKernel
2.9 9971082 1 9971082.0 9971082.0 9971082 9971082 0.0 cudaGetDeviceProperties_v2_v12000
2.6 8726842 6 1454473.7 1517760.0 3446 2841288 1477680.0 cudaMalloc
1.6 5319195 12 443266.3 5159.5 171 1876362 633355.6 cudaFree
0.7 2439536 1 2439536.0 2439536.0 2439536 2439536 0.0 cudaFreeHost
0.4 1329358 1 1329358.0 1329358.0 1329358 1329358 0.0 cudaMallocHost
0.1 173678 838 207.3 190.0 80 1042 105.9 cuGetProcAddress_v2
0.0 35757 1 35757.0 35757.0 35757 35757 0.0 cudaMemset
0.0 19408 18 1078.2 276.0 250 7765 1931.9 cudaEventCreateWithFlags
0.0 12392 4 3098.0 2845.0 1002 5700 2009.4 cudaDeviceSynchronize
0.0 7253 18 402.9 285.0 200 1854 379.7 cudaEventDestroy
0.0 5671 4 1417.8 1428.0 1263 1552 141.7 cuInit
0.0 1483 3 494.3 151.0 110 1222 630.5 cuModuleGetLoadingMode
0.0 1262 2 631.0 631.0 461 801 240.4 cudaGetDriverEntryPoint_v11030
[6/8] Executing 'cuda_gpu_kern_sum' stats report
Time (%) Total Time (ns) Instances Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- --------- --------- -------- -------- ----------- ----------------------------------------------------------------------------------------------------
97.6 243397681 49 4967299.6 4539935.0 1227263 73746344 10132529.6 matmul_forward_kernel(float *, const float *, const float *, const float *, int, int, int, int)
1.4 3576415 25 143056.6 142880.0 141472 146976 1318.1 layernorm_forward_kernel(float *, float *, float *, const float *, const float *, const float *, in…
0.8 1879295 12 156607.9 156656.0 155584 157120 383.9 void flash_attention_forward_kernel<(int)64, (int)64, (int)64>(const float *, const float *, const …
0.1 228672 12 19056.0 19040.0 18880 19264 90.0 gelu_forward_kernel(float *, const float *, int)
0.1 146208 24 6092.0 6080.0 5728 6560 355.1 residual_forward_kernel(float *, const float *, const float *, int)
0.0 80768 1 80768.0 80768.0 80768 80768 0.0 encoder_forward_kernel(float *, const int *, const float *, const float *, int, int, int)
0.0 78560 12 6546.7 6480.0 6400 6912 158.2 permute_kernel(float *, float *, float *, const float *, int, int, int, int)
0.0 37952 12 3162.7 3152.0 3040 3296 108.2 unpermute_kernel(const float *, float *, int, int, int, int)
[7/8] Executing 'cuda_gpu_mem_time_sum' stats report
Time (%) Total Time (ns) Count Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Operation
-------- --------------- ----- ---------- ---------- -------- -------- ----------- ----------------------------
82.5 36437716 2 18218858.0 18218858.0 832 36436884 25764179.4 [CUDA memcpy Host-to-Device]
16.5 7304990 1 7304990.0 7304990.0 7304990 7304990 0.0 [CUDA memcpy Device-to-Host]
0.9 406816 1 406816.0 406816.0 406816 406816 0.0 [CUDA memset]
[8/8] Executing 'cuda_gpu_mem_size_sum' stats report
Total (MB) Count Avg (MB) Med (MB) Min (MB) Max (MB) StdDev (MB) Operation
---------- ----- -------- -------- -------- -------- ----------- ----------------------------
497.905 2 248.952 248.952 0.001 497.904 352.070 [CUDA memcpy Host-to-Device]
232.392 1 232.392 232.392 232.392 232.392 0.000 [CUDA memset]
51.511 1 51.511 51.511 51.511 51.511 0.000 [CUDA memcpy Device-to-Host]
Generated:
/tmp/nsys-report-d663.nsys-rep
/tmp/nsys-report-472f.sqlite
output/perf/nsys/verify_op_12.err
+9
View File
@@ -0,0 +1,9 @@
WARNING: Device-side CUDA Event completion trace is currently enabled.
This may increase runtime overhead and the likelihood of false
dependencies across CUDA Streams. If you wish to avoid this, please
disable the feature with --cuda-event-trace=false.
Failed to create '/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_12.nsys-rep': File exists.
Use `--force-overwrite true` to overwrite existing files.
Failed to create '/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_12.sqlite': File exists.
Use `--force-overwrite true` to overwrite existing files.
SKIPPED: /tmp/nsys-report-9b06.sqlite does not contain NV Tools Extension (NVTX) data.
output/perf/nsys/verify_op_12.out
+109
View File
@@ -0,0 +1,109 @@
job 18002391 starting on gpub073.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431793
-39.836346, -39.836502
-43.065910, -43.066078
-42.828045, -42.828197
-43.529541, -43.529701
-44.318398, -44.318558
-41.227425, -41.227592
-41.270760, -41.270931
-42.541393, -42.541584
-42.394997, -42.395176
---------------- Logits OK! (max err 0.000050, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
Collecting data...
Generating '/tmp/nsys-report-c6cc.qdstrm'
␍[1/8] [0% ] nsys-report-d644.nsys-rep␍[1/8] [0% ] nsys-report-d644.nsys-rep␍[1/8] [====26% ] nsys-report-d644.nsys-rep␍[1/8] [==========49% ] nsys-report-d644.nsys-rep␍[1/8] [===========53% ] nsys-report-d644.nsys-rep␍[1/8] [=======36% ] nsys-report-d644.nsys-rep␍[1/8] [===========52% ] nsys-report-d644.nsys-rep␍[1/8] [============54% ] nsys-report-d644.nsys-rep␍[1/8] [============55% ] nsys-report-d644.nsys-rep␍[1/8] [============56% ] nsys-report-d644.nsys-rep␍[1/8] [==============62% ] nsys-report-d644.nsys-rep␍[1/8] [==================78% ] nsys-report-d644.nsys-rep␍[1/8] [======================92% ] nsys-report-d644.nsys-rep␍[1/8] [========================100%] nsys-report-d644.nsys-rep␍[1/8] [========================100%] nsys-report-d644.nsys-rep
[2/8] [0% ] nsys-report-9b06.sqlite [2/8] [1% ] nsys-report-9b06.sqlite [2/8] [2% ] nsys-report-9b06.sqlite [2/8] [3% ] nsys-report-9b06.sqlite [2/8] [4% ] nsys-report-9b06.sqlite [2/8] [5% ] nsys-report-9b06.sqlite [2/8] [6% ] nsys-report-9b06.sqlite [2/8] [7% ] nsys-report-9b06.sqlite [2/8] [8% ] nsys-report-9b06.sqlite [2/8] [9% ] nsys-report-9b06.sqlite [2/8] [10% ] nsys-report-9b06.sqlite [2/8] [11% ] nsys-report-9b06.sqlite [2/8] [12% ] nsys-report-9b06.sqlite [2/8] [13% ] nsys-report-9b06.sqlite [2/8] [14% ] nsys-report-9b06.sqlite [2/8] [=15% ] nsys-report-9b06.sqlite [2/8] [=16% ] nsys-report-9b06.sqlite [2/8] [=17% ] nsys-report-9b06.sqlite [2/8] [==18% ] nsys-report-9b06.sqlite [2/8] [==19% ] nsys-report-9b06.sqlite [2/8] [==20% ] nsys-report-9b06.sqlite [2/8] [==21% ] nsys-report-9b06.sqlite [2/8] [===22% ] nsys-report-9b06.sqlite [2/8] [===23% ] nsys-report-9b06.sqlite [2/8] [===24% ] nsys-report-9b06.sqlite [2/8] [====25% ] nsys-report-9b06.sqlite [2/8] [====26% ] nsys-report-9b06.sqlite [2/8] [====27% ] nsys-report-9b06.sqlite [2/8] [====28% ] nsys-report-9b06.sqlite [2/8] [=====29% ] nsys-report-9b06.sqlite [2/8] [=====30% ] nsys-report-9b06.sqlite [2/8] [=====31% ] nsys-report-9b06.sqlite [2/8] [=====32% ] nsys-report-9b06.sqlite [2/8] [======33% ] nsys-report-9b06.sqlite [2/8] [======34% ] nsys-report-9b06.sqlite [2/8] [======35% ] nsys-report-9b06.sqlite [2/8] [=======36% ] nsys-report-9b06.sqlite [2/8] [=======37% ] nsys-report-9b06.sqlite [2/8] [=======38% ] nsys-report-9b06.sqlite [2/8] [=======39% ] nsys-report-9b06.sqlite [2/8] [========40% ] nsys-report-9b06.sqlite [2/8] [========41% ] nsys-report-9b06.sqlite [2/8] [========42% ] nsys-report-9b06.sqlite [2/8] [=========43% ] nsys-report-9b06.sqlite [2/8] [=========44% ] nsys-report-9b06.sqlite [2/8] [=========45% ] nsys-report-9b06.sqlite [2/8] [=========46% ] nsys-report-9b06.sqlite [2/8] [==========47% ] nsys-report-9b06.sqlite [2/8] [==========48% ] nsys-report-9b06.sqlite [2/8] [==========49% ] nsys-report-9b06.sqlite [2/8] [===========50% ] nsys-report-9b06.sqlite [2/8] [===========51% ] nsys-report-9b06.sqlite [2/8] [===========52% ] nsys-report-9b06.sqlite [2/8] [===========53% ] nsys-report-9b06.sqlite [2/8] [============54% ] nsys-report-9b06.sqlite [2/8] [============55% ] nsys-report-9b06.sqlite [2/8] [============56% ] nsys-report-9b06.sqlite [2/8] [============57% ] nsys-report-9b06.sqlite [2/8] [=============58% ] nsys-report-9b06.sqlite [2/8] [=============59% ] nsys-report-9b06.sqlite [2/8] [=============60% ] nsys-report-9b06.sqlite [2/8] [==============61% ] nsys-report-9b06.sqlite [2/8] [==============62% ] nsys-report-9b06.sqlite [2/8] [==============63% ] nsys-report-9b06.sqlite [2/8] [==============64% ] nsys-report-9b06.sqlite [2/8] [===============65% ] nsys-report-9b06.sqlite [2/8] [===============66% ] nsys-report-9b06.sqlite [2/8] [===============67% ] nsys-report-9b06.sqlite [2/8] [================68% ] nsys-report-9b06.sqlite [2/8] [================69% ] nsys-report-9b06.sqlite [2/8] [================70% ] nsys-report-9b06.sqlite [2/8] [================71% ] nsys-report-9b06.sqlite [2/8] [=================72% ] nsys-report-9b06.sqlite [2/8] [=================73% ] nsys-report-9b06.sqlite [2/8] [=================74% ] nsys-report-9b06.sqlite [2/8] [==================75% ] nsys-report-9b06.sqlite [2/8] [==================76% ] nsys-report-9b06.sqlite [2/8] [==================77% ] nsys-report-9b06.sqlite [2/8] [==================78% ] nsys-report-9b06.sqlite [2/8] [===================79% ] nsys-report-9b06.sqlite [2/8] [===================80% ] nsys-report-9b06.sqlite [2/8] [===================81% ] nsys-report-9b06.sqlLine truncated
[3/8] Executing 'nvtx_sum' stats report
[4/8] Executing 'osrt_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ----------------------
61.1 737068265 17 43356956.8 31803265.0 2204 136058155 46033325.4 poll
29.3 352852419 581 607319.1 280456.0 1082 20080933 1246338.6 ioctl
7.2 86629708 6 14438284.7 75346.0 5560 78483379 31535650.3 fread
1.1 13589855 1 13589855.0 13589855.0 13589855 13589855 0.0 pthread_cond_wait
0.6 7448723 1 7448723.0 7448723.0 7448723 7448723 0.0 pthread_cond_broadcast
0.2 2347161 25 93886.4 9638.0 7675 1629425 322632.7 mmap64
0.2 2110733 32 65960.4 2189.0 1352 1703774 304577.4 fclose
0.1 1693364 38 44562.2 6256.0 3587 809497 144162.0 fopen
0.1 746287 11 67844.3 27722.0 1132 318406 94484.9 write
0.0 594968 17 34998.1 4739.0 2575 402164 96189.6 mmap
0.0 361558 2 180779.0 180779.0 164709 196849 22726.4 pthread_create
0.0 353489 55 6427.1 6332.0 2344 13415 2698.2 open64
0.0 221045 1 221045.0 221045.0 221045 221045 0.0 fflush
0.0 76323 1 76323.0 76323.0 76323 76323 0.0 fgets
0.0 37250 8 4656.3 4112.5 2875 7805 1734.3 munmap
0.0 36069 6 6011.5 5996.5 3086 9027 2010.2 open
0.0 20939 3 6979.7 8526.0 3727 8686 2818.0 pipe2
0.0 18773 9 2085.9 2083.0 1082 3727 979.8 read
0.0 17263 2 8631.5 8631.5 6523 10740 2981.9 socket
0.0 9477 1 9477.0 9477.0 9477 9477 0.0 connect
0.0 6272 1 6272.0 6272.0 6272 6272 0.0 fopen64
0.0 4249 2 2124.5 2124.5 1413 2836 1006.2 fwrite
0.0 2134 1 2134.0 2134.0 2134 2134 0.0 bind
0.0 2054 2 1027.0 1027.0 1012 1042 21.2 fcntl
[5/8] Executing 'cuda_api_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- -------- ----------- ---------------------------------
76.2 121358249 3 40452749.7 58133092.0 22181 63202976 35105542.2 cudaMemcpy
9.7 15479869 6 2579978.2 1263944.5 4198 8351576 3362372.0 cudaMalloc
6.3 10045470 1 10045470.0 10045470.0 10045470 10045470 0.0 cudaGetDeviceProperties_v2_v12000
3.2 5100454 12 425037.8 6347.5 180 1804402 614184.2 cudaFree
1.9 2948206 1 2948206.0 2948206.0 2948206 2948206 0.0 cudaMallocHost
1.6 2474178 1 2474178.0 2474178.0 2474178 2474178 0.0 cudaFreeHost
1.1 1675156 171 9796.2 2805.0 2675 1050069 80118.8 cudaLaunchKernel
0.1 154831 838 184.8 150.0 80 762 92.5 cuGetProcAddress_v2
0.0 39014 1 39014.0 39014.0 39014 39014 0.0 cudaMemset
0.0 17903 18 994.6 296.0 250 6583 1655.0 cudaEventCreateWithFlags
0.0 12222 4 3055.5 2619.5 982 6001 2113.3 cudaDeviceSynchronize
0.0 6803 18 377.9 251.0 210 2014 418.1 cudaEventDestroy
0.0 5841 4 1460.3 1362.5 1182 1934 328.0 cuInit
0.0 1352 3 450.7 120.0 90 1142 598.9 cuModuleGetLoadingMode
0.0 852 2 426.0 426.0 301 551 176.8 cudaGetDriverEntryPoint_v11030
[6/8] Executing 'cuda_gpu_kern_sum' stats report
Time (%) Total Time (ns) Instances Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- -------- -------- -------- -------- ----------- ----------------------------------------------------------------------------------------------------
88.5 45226826 49 922996.4 851682.0 243969 12757309 1750034.1 matmul_kernel(const float *, float *, const float *, const float *, int, int, int, int)
7.0 3578022 25 143120.9 143072.0 141120 146912 1197.9 layernorm_forward_kernel(float *, float *, float *, const float *, const float *, const float *, in…
2.3 1153027 12 96085.6 96112.5 95904 96192 94.9 preattention_scores_kernel(const float *, const float *, int, int, int, int, float *)
0.7 346208 12 28850.7 28688.0 28352 29760 446.4 attention_v_kernel(float *, const float *, const float *, int, int, int, int)
0.4 228480 12 19040.0 19040.0 18880 19232 79.6 gelu_forward_kernel(float *, const float *, int)
0.4 210304 12 17525.3 17520.0 17344 17792 116.8 softmax_forward_kernel(float *, float, const float *, int, int)
0.3 144992 24 6041.3 6048.0 5664 6464 367.5 residual_forward_kernel(float *, const float *, const float *, int)
0.2 84320 12 7026.7 7024.0 6880 7168 92.0 permute_kernel(float *, float *, float *, const float *, int, int, int, int)
0.2 80545 1 80545.0 80545.0 80545 80545 0.0 encoder_forward_kernel(float *, const int *, const float *, const float *, int, int, int)
0.1 70400 12 5866.7 5872.0 5792 5888 28.4 unpermute_kernel(const float *, float *, int, int, int, int)
[7/8] Executing 'cuda_gpu_mem_time_sum' stats report
Time (%) Total Time (ns) Count Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Operation
-------- --------------- ----- ---------- ---------- -------- -------- ----------- ----------------------------
88.9 63114351 2 31557175.5 31557175.5 832 63113519 44627409.0 [CUDA memcpy Host-to-Device]
10.5 7436977 1 7436977.0 7436977.0 7436977 7436977 0.0 [CUDA memcpy Device-to-Host]
0.6 407969 1 407969.0 407969.0 407969 407969 0.0 [CUDA memset]
[8/8] Executing 'cuda_gpu_mem_size_sum' stats report
Total (MB) Count Avg (MB) Med (MB) Min (MB) Max (MB) StdDev (MB) Operation
---------- ----- -------- -------- -------- -------- ----------- ----------------------------
497.905 2 248.952 248.952 0.001 497.904 352.070 [CUDA memcpy Host-to-Device]
232.392 1 232.392 232.392 232.392 232.392 0.000 [CUDA memset]
51.511 1 51.511 51.511 51.511 51.511 0.000 [CUDA memcpy Device-to-Host]
Generated:
/tmp/nsys-report-d644.nsys-rep
/tmp/nsys-report-9b06.sqlite
output/perf/nsys/verify_op_13.err
+5
View File
@@ -0,0 +1,5 @@
WARNING: Device-side CUDA Event completion trace is currently enabled.
This may increase runtime overhead and the likelihood of false
dependencies across CUDA Streams. If you wish to avoid this, please
disable the feature with --cuda-event-trace=false.
SKIPPED: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_13.sqlite does not contain NV Tools Extension (NVTX) data.
output/perf/nsys/verify_op_13.out
+108
View File
@@ -0,0 +1,108 @@
job 18003870 starting on gpub055.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000043, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
Collecting data...
Generating '/tmp/nsys-report-357a.qdstrm'
␍[1/8] [0% ] report_op_13.nsys-rep␍[1/8] [0% ] report_op_13.nsys-rep␍[1/8] [====26% ] report_op_13.nsys-rep␍[1/8] [==========49% ] report_op_13.nsys-rep␍[1/8] [===========50% ] report_op_13.nsys-rep␍[1/8] [===========51% ] report_op_13.nsys-rep␍[1/8] [=======38% ] report_op_13.nsys-rep␍[1/8] [===========51% ] report_op_13.nsys-rep␍[1/8] [===========52% ] report_op_13.nsys-rep␍[1/8] [===========53% ] report_op_13.nsys-rep␍[1/8] [==============64% ] report_op_13.nsys-rep␍[1/8] [======================90% ] report_op_13.nsys-rep␍[1/8] [========================100%] report_op_13.nsys-rep␍[1/8] [========================100%] report_op_13.nsys-rep
[2/8] [0% ] report_op_13.sqlite [2/8] [1% ] report_op_13.sqlite [2/8] [2% ] report_op_13.sqlite [2/8] [3% ] report_op_13.sqlite [2/8] [4% ] report_op_13.sqlite [2/8] [5% ] report_op_13.sqlite [2/8] [6% ] report_op_13.sqlite [2/8] [7% ] report_op_13.sqlite [2/8] [8% ] report_op_13.sqlite [2/8] [9% ] report_op_13.sqlite [2/8] [10% ] report_op_13.sqlite [2/8] [11% ] report_op_13.sqlite [2/8] [12% ] report_op_13.sqlite [2/8] [13% ] report_op_13.sqlite [2/8] [14% ] report_op_13.sqlite [2/8] [=15% ] report_op_13.sqlite [2/8] [=16% ] report_op_13.sqlite [2/8] [=17% ] report_op_13.sqlite [2/8] [==18% ] report_op_13.sqlite [2/8] [==19% ] report_op_13.sqlite [2/8] [==20% ] report_op_13.sqlite [2/8] [==21% ] report_op_13.sqlite [2/8] [===22% ] report_op_13.sqlite [2/8] [===23% ] report_op_13.sqlite [2/8] [===24% ] report_op_13.sqlite [2/8] [====25% ] report_op_13.sqlite [2/8] [====26% ] report_op_13.sqlite [2/8] [====27% ] report_op_13.sqlite [2/8] [====28% ] report_op_13.sqlite [2/8] [=====29% ] report_op_13.sqlite [2/8] [=====30% ] report_op_13.sqlite [2/8] [=====31% ] report_op_13.sqlite [2/8] [=====32% ] report_op_13.sqlite [2/8] [======33% ] report_op_13.sqlite [2/8] [======34% ] report_op_13.sqlite [2/8] [======35% ] report_op_13.sqlite [2/8] [=======36% ] report_op_13.sqlite [2/8] [=======37% ] report_op_13.sqlite [2/8] [=======38% ] report_op_13.sqlite [2/8] [=======39% ] report_op_13.sqlite [2/8] [========40% ] report_op_13.sqlite [2/8] [========41% ] report_op_13.sqlite [2/8] [========42% ] report_op_13.sqlite [2/8] [=========43% ] report_op_13.sqlite [2/8] [=========44% ] report_op_13.sqlite [2/8] [=========45% ] report_op_13.sqlite [2/8] [=========46% ] report_op_13.sqlite [2/8] [==========47% ] report_op_13.sqlite [2/8] [==========48% ] report_op_13.sqlite [2/8] [==========49% ] report_op_13.sqlite [2/8] [===========50% ] report_op_13.sqlite [2/8] [===========51% ] report_op_13.sqlite [2/8] [===========52% ] report_op_13.sqlite [2/8] [===========53% ] report_op_13.sqlite [2/8] [============54% ] report_op_13.sqlite [2/8] [============55% ] report_op_13.sqlite [2/8] [============56% ] report_op_13.sqlite [2/8] [============57% ] report_op_13.sqlite [2/8] [=============58% ] report_op_13.sqlite [2/8] [=============59% ] report_op_13.sqlite [2/8] [=============60% ] report_op_13.sqlite [2/8] [==============61% ] report_op_13.sqlite [2/8] [==============62% ] report_op_13.sqlite [2/8] [==============63% ] report_op_13.sqlite [2/8] [==============64% ] report_op_13.sqlite [2/8] [===============65% ] report_op_13.sqlite [2/8] [===============66% ] report_op_13.sqlite [2/8] [===============67% ] report_op_13.sqlite [2/8] [================68% ] report_op_13.sqlite [2/8] [================69% ] report_op_13.sqlite [2/8] [================70% ] report_op_13.sqlite [2/8] [================71% ] report_op_13.sqlite [2/8] [=================72% ] report_op_13.sqlite [2/8] [=================73% ] report_op_13.sqlite [2/8] [=================74% ] report_op_13.sqlite [2/8] [==================75% ] report_op_13.sqlite [2/8] [==================76% ] report_op_13.sqlite [2/8] [==================77% ] report_op_13.sqlite [2/8] [==================78% ] report_op_13.sqlite [2/8] [===================79% ] report_op_13.sqlite [2/8] [===================80% ] report_op_13.sqlite [2/8] [===================81% ] report_op_13.sqlite [2/8] [===================82% ] report_op_13.sqlite [2/8] [====================83% ] report_op_13.sqlite [2/8] [====================84% ] report_op_13.sqlite [2/8] [====================85% ] report_op_13.sqlite [2/8] [=====================86% ] report_op_13.sqlite [2/8] [=====================87% ] reLine truncated
[3/8] Executing 'nvtx_sum' stats report
[4/8] Executing 'osrt_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ----------------------
62.3 1192177975 22 54189908.0 60718799.0 1293 100135381 46848153.7 poll
27.8 532636664 6 88772777.3 616568.5 7834 482699408 193951324.3 fread
7.6 145642532 586 248536.7 21555.0 1032 28349154 1447180.4 ioctl
1.1 20477585 1 20477585.0 20477585.0 20477585 20477585 0.0 pthread_cond_wait
0.5 9619823 38 253153.2 6146.0 3567 8444540 1368023.7 fopen
0.4 7231291 1 7231291.0 7231291.0 7231291 7231291 0.0 pthread_cond_broadcast
0.1 2321011 25 92840.4 9037.0 7353 1601907 317206.9 mmap64
0.1 1381765 10 138176.5 29961.0 22552 501578 193339.9 write
0.0 665856 32 20808.0 2144.0 1353 313456 72473.1 fclose
0.0 574715 17 33806.8 4579.0 1844 383638 91786.3 mmap
0.0 347480 2 173740.0 173740.0 156984 190496 23696.6 pthread_create
0.0 313434 55 5698.8 5871.0 2334 9788 2009.1 open64
0.0 206186 1 206186.0 206186.0 206186 206186 0.0 fflush
0.0 77074 1 77074.0 77074.0 77074 77074 0.0 fgets
0.0 48280 8 6035.0 4824.0 2665 12333 3569.5 munmap
0.0 35765 6 5960.8 6016.0 3055 8936 1976.5 open
0.0 23155 3 7718.3 9318.0 3457 10380 3728.4 pipe2
0.0 17893 9 1988.1 1463.0 1012 3617 1022.1 read
0.0 15579 2 7789.5 7789.5 5791 9788 2826.3 socket
0.0 9318 1 9318.0 9318.0 9318 9318 0.0 connect
0.0 6342 1 6342.0 6342.0 6342 6342 0.0 fopen64
0.0 3918 2 1959.0 1959.0 1453 2465 715.6 fwrite
0.0 1623 1 1623.0 1623.0 1623 1623 0.0 bind
[5/8] Executing 'cuda_api_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ---------------------------------
86.7 292338879 3 97446293.0 36179585.0 18745 256140549 138617465.5 cudaMemcpy
10.4 34929986 12 2910832.2 8676.5 230 29248496 8331465.4 cudaFree
1.1 3690547 171 21582.1 2875.0 2735 3041560 232340.0 cudaLaunchKernel
0.7 2477256 1 2477256.0 2477256.0 2477256 2477256 0.0 cudaFreeHost
0.4 1441077 1 1441077.0 1441077.0 1441077 1441077 0.0 cudaGetDeviceProperties_v2_v12000
0.4 1293189 1 1293189.0 1293189.0 1293189 1293189 0.0 cudaMallocHost
0.2 708847 6 118141.2 115361.0 3406 282809 107158.9 cudaMalloc
0.1 176229 838 210.3 180.0 89 5641 213.4 cuGetProcAddress_v2
0.0 34836 1 34836.0 34836.0 34836 34836 0.0 cudaMemset
0.0 16491 4 4122.8 3136.5 1021 9197 3611.3 cudaDeviceSynchronize
0.0 11905 18 661.4 280.5 250 3005 821.3 cudaEventCreateWithFlags
0.0 7888 18 438.2 270.5 210 2675 575.7 cudaEventDestroy
0.0 5102 4 1275.5 1278.0 1233 1313 33.0 cuInit
0.0 1753 3 584.3 461.0 110 1182 546.5 cuModuleGetLoadingMode
0.0 993 2 496.5 496.5 331 662 234.1 cudaGetDriverEntryPoint_v11030
[6/8] Executing 'cuda_gpu_kern_sum' stats report
Time (%) Total Time (ns) Instances Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- --------- --------- -------- -------- ----------- ----------------------------------------------------------------------------------------------------
97.6 243490921 49 4969202.5 4539029.0 1227366 73795414 10139382.4 matmul_forward_kernel(float *, const float *, const float *, const float *, int, int, int, int)
1.5 3693489 25 147739.6 148096.0 144832 149601 1394.9 layernorm_forward_kernel(float *, float *, float *, const float *, const float *, const float *, in…
0.5 1145094 12 95424.5 95232.5 95073 96065 370.8 preattention_scores_kernel(const float *, const float *, int, int, int, int, float *)
0.1 334432 12 27869.3 27968.0 26880 28704 511.5 attention_v_kernel(float *, const float *, const float *, int, int, int, int)
0.1 227360 12 18946.7 18976.0 18720 19008 74.1 gelu_forward_kernel(float *, const float *, int)
0.1 207585 12 17298.8 17344.0 17120 17440 111.3 softmax_forward_kernel(float *, float, const float *, int, int)
0.1 145664 24 6069.3 6128.0 5664 6464 351.1 residual_forward_kernel(float *, const float *, const float *, int)
0.0 96320 12 8026.7 8016.0 7840 8224 115.6 permute_kernel(float *, float *, float *, const float *, int, int, int, int)
0.0 80128 1 80128.0 80128.0 80128 80128 0.0 encoder_forward_kernel(float *, const int *, const float *, const float *, int, int, int)
0.0 70530 12 5877.5 5888.0 5856 5920 20.8 unpermute_kernel(const float *, float *, int, int, int, int)
[7/8] Executing 'cuda_gpu_mem_time_sum' stats report
Time (%) Total Time (ns) Count Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Operation
-------- --------------- ----- ---------- ---------- -------- -------- ----------- ----------------------------
82.8 36065319 2 18032659.5 18032659.5 832 36064487 25500855.0 [CUDA memcpy Host-to-Device]
16.3 7099265 1 7099265.0 7099265.0 7099265 7099265 0.0 [CUDA memcpy Device-to-Host]
0.9 407490 1 407490.0 407490.0 407490 407490 0.0 [CUDA memset]
[8/8] Executing 'cuda_gpu_mem_size_sum' stats report
Total (MB) Count Avg (MB) Med (MB) Min (MB) Max (MB) StdDev (MB) Operation
---------- ----- -------- -------- -------- -------- ----------- ----------------------------
497.905 2 248.952 248.952 0.001 497.904 352.070 [CUDA memcpy Host-to-Device]
232.393 1 232.393 232.393 232.393 232.393 0.000 [CUDA memset]
51.511 1 51.511 51.511 51.511 51.511 0.000 [CUDA memcpy Device-to-Host]
Generated:
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_13.nsys-rep
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_13.sqlite
output/perf/nsys/verify_op_16.err
+5
View File
@@ -0,0 +1,5 @@
WARNING: Device-side CUDA Event completion trace is currently enabled.
This may increase runtime overhead and the likelihood of false
dependencies across CUDA Streams. If you wish to avoid this, please
disable the feature with --cuda-event-trace=false.
SKIPPED: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_16.sqlite does not contain NV Tools Extension (NVTX) data.
output/perf/nsys/verify_op_16.out
+116
View File
@@ -0,0 +1,116 @@
job 18004586 starting on gpub041.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.315117
-39.836346, -39.732384
-43.065910, -42.960098
-42.828045, -42.723515
-43.529541, -43.418613
-44.318398, -44.210033
-41.227425, -41.119377
-41.270760, -41.164490
-42.541393, -42.428894
-42.394997, -42.288929
---------------- Logits OK! (max err 0.082334, RMSE 0.002728) ----------------
_________________________ All tests passed! _________________________
Collecting data...
Generating '/tmp/nsys-report-cacf.qdstrm'
␍[1/8] [0% ] report_op_16.nsys-rep␍[1/8] [0% ] report_op_16.nsys-rep␍[1/8] [====27% ] report_op_16.nsys-rep␍[1/8] [====28% ] report_op_16.nsys-rep␍[1/8] [=====29% ] report_op_16.nsys-rep␍[1/8] [=====30% ] report_op_16.nsys-rep␍[1/8] [=======38% ] report_op_16.nsys-rep␍[1/8] [=========46% ] report_op_16.nsys-rep␍[1/8] [==========47% ] report_op_16.nsys-rep␍[1/8] [==========48% ] report_op_16.nsys-rep␍[1/8] [==========49% ] report_op_16.nsys-rep␍[1/8] [============55% ] report_op_16.nsys-rep␍[1/8] [============56% ] report_op_16.nsys-rep␍[1/8] [============57% ] report_op_16.nsys-rep␍[1/8] [=============58% ] report_op_16.nsys-rep␍[1/8] [==================77% ] report_op_16.nsys-rep␍[1/8] [=======================94% ] report_op_16.nsys-rep␍[1/8] [========================100%] report_op_16.nsys-rep␍[1/8] [========================100%] report_op_16.nsys-rep
[2/8] [0% ] report_op_16.sqlite [2/8] [1% ] report_op_16.sqlite [2/8] [2% ] report_op_16.sqlite [2/8] [3% ] report_op_16.sqlite [2/8] [4% ] report_op_16.sqlite [2/8] [5% ] report_op_16.sqlite [2/8] [6% ] report_op_16.sqlite [2/8] [7% ] report_op_16.sqlite [2/8] [8% ] report_op_16.sqlite [2/8] [9% ] report_op_16.sqlite [2/8] [10% ] report_op_16.sqlite [2/8] [11% ] report_op_16.sqlite [2/8] [12% ] report_op_16.sqlite [2/8] [13% ] report_op_16.sqlite [2/8] [14% ] report_op_16.sqlite [2/8] [=15% ] report_op_16.sqlite [2/8] [=16% ] report_op_16.sqlite [2/8] [=17% ] report_op_16.sqlite [2/8] [==18% ] report_op_16.sqlite [2/8] [==19% ] report_op_16.sqlite [2/8] [==20% ] report_op_16.sqlite [2/8] [==21% ] report_op_16.sqlite [2/8] [===22% ] report_op_16.sqlite [2/8] [===23% ] report_op_16.sqlite [2/8] [===24% ] report_op_16.sqlite [2/8] [====25% ] report_op_16.sqlite [2/8] [====26% ] report_op_16.sqlite [2/8] [====27% ] report_op_16.sqlite [2/8] [====28% ] report_op_16.sqlite [2/8] [=====29% ] report_op_16.sqlite [2/8] [=====30% ] report_op_16.sqlite [2/8] [=====31% ] report_op_16.sqlite [2/8] [=====32% ] report_op_16.sqlite [2/8] [======33% ] report_op_16.sqlite [2/8] [======34% ] report_op_16.sqlite [2/8] [======35% ] report_op_16.sqlite [2/8] [=======36% ] report_op_16.sqlite [2/8] [=======37% ] report_op_16.sqlite [2/8] [=======38% ] report_op_16.sqlite [2/8] [=======39% ] report_op_16.sqlite [2/8] [========40% ] report_op_16.sqlite [2/8] [========41% ] report_op_16.sqlite [2/8] [========42% ] report_op_16.sqlite [2/8] [=========43% ] report_op_16.sqlite [2/8] [=========44% ] report_op_16.sqlite [2/8] [=========45% ] report_op_16.sqlite [2/8] [=========46% ] report_op_16.sqlite [2/8] [==========47% ] report_op_16.sqlite [2/8] [==========48% ] report_op_16.sqlite [2/8] [==========49% ] report_op_16.sqlite [2/8] [===========50% ] report_op_16.sqlite [2/8] [===========51% ] report_op_16.sqlite [2/8] [===========52% ] report_op_16.sqlite [2/8] [===========53% ] report_op_16.sqlite [2/8] [============54% ] report_op_16.sqlite [2/8] [============55% ] report_op_16.sqlite [2/8] [============56% ] report_op_16.sqlite [2/8] [============57% ] report_op_16.sqlite [2/8] [=============58% ] report_op_16.sqlite [2/8] [=============59% ] report_op_16.sqlite [2/8] [=============60% ] report_op_16.sqlite [2/8] [==============61% ] report_op_16.sqlite [2/8] [==============62% ] report_op_16.sqlite [2/8] [==============63% ] report_op_16.sqlite [2/8] [==============64% ] report_op_16.sqlite [2/8] [===============65% ] report_op_16.sqlite [2/8] [===============66% ] report_op_16.sqlite [2/8] [===============67% ] report_op_16.sqlite [2/8] [================68% ] report_op_16.sqlite [2/8] [================69% ] report_op_16.sqlite [2/8] [================70% ] report_op_16.sqlite [2/8] [================71% ] report_op_16.sqlite [2/8] [=================72% ] report_op_16.sqlite [2/8] [=================73% ] report_op_16.sqlite [2/8] [=================74% ] report_op_16.sqlite [2/8] [==================75% ] report_op_16.sqlite [2/8] [==================76% ] report_op_16.sqlite [2/8] [==================77% ] report_op_16.sqlite [2/8] [==================78% ] report_op_16.sqlite [2/8] [===================79% ] report_op_16.sqlite [2/8] [===================80% ] report_op_16.sqlite [2/8] [===================81% ] report_op_16.sqlite [2/8] [===================82% ] report_op_16.sqlite [2/8] [====================83% ] report_op_16.sqlite [2/8] [====================84% ] report_op_16.sqlite [2/8] [====================85% ] report_op_16.sqlite [2/8] [=====================86% ] report_op_16.sqlite [2/8] [=====================87% ] reLine truncated
[3/8] Executing 'nvtx_sum' stats report
[4/8] Executing 'osrt_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ----------- ---------- -------- --------- ----------- --------------
58.3 1107473464 21 52736831.6 32614355.0 1322 102246365 47278143.8 poll
32.4 615776283 6 102629380.5 9779477.0 6893 526459513 209303008.5 fread
8.7 164992616 592 278703.7 23008.0 1041 22196543 1294110.2 ioctl
0.2 4255919 38 111997.9 6477.0 3637 3134408 512054.3 fopen
0.1 2359811 25 94392.4 8766.0 7474 1634199 323598.5 mmap64
0.1 999815 32 31244.2 2119.0 1333 525704 114089.3 fclose
0.0 836867 11 76078.8 29566.0 1603 467756 132907.7 write
0.0 568205 17 33423.8 8155.0 1943 376916 90182.7 mmap
0.0 345457 2 172728.5 172728.5 163176 182281 13509.3 pthread_create
0.0 328243 55 5968.1 5862.0 2344 13786 2471.7 open64
0.0 223538 1 223538.0 223538.0 223538 223538 0.0 fflush
0.0 80441 1 80441.0 80441.0 80441 80441 0.0 fgets
0.0 46428 9 5158.7 4038.0 2575 9789 2506.9 munmap
0.0 37720 6 6286.7 6542.5 3276 8926 1948.3 open
0.0 27181 3 9060.3 8235.0 6613 12333 2948.0 pipe2
0.0 19594 11 1781.3 1282.0 1002 3506 893.6 read
0.0 17683 2 8841.5 8841.5 5851 11832 4229.2 socket
0.0 10981 1 10981.0 10981.0 10981 10981 0.0 connect
0.0 6862 1 6862.0 6862.0 6862 6862 0.0 fopen64
0.0 4558 2 2279.0 2279.0 2074 2484 289.9 fwrite
0.0 2174 1 2174.0 2174.0 2174 2174 0.0 bind
0.0 1062 1 1062.0 1062.0 1062 1062 0.0 listen
[5/8] Executing 'cuda_api_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- -------- ----------- ---------------------------------
54.2 51711904 3 17237301.3 15706111.0 19487 35986306 18032232.9 cudaMemcpy
22.1 21075926 12 1756327.2 15158.5 190 16493445 4691526.2 cudaFree
10.2 9764437 4 2441109.3 2908985.5 877652 3068814 1045305.5 cuLibraryLoadData
3.8 3644594 207 17606.7 4519.0 2725 2467218 171275.3 cudaLaunchKernel
2.5 2409812 1 2409812.0 2409812.0 2409812 2409812 0.0 cudaFreeHost
1.7 1608458 18 89358.8 305.5 260 1597961 376498.2 cudaEventCreateWithFlags
1.4 1351740 1 1351740.0 1351740.0 1351740 1351740 0.0 cudaMallocHost
1.4 1315042 1 1315042.0 1315042.0 1315042 1315042 0.0 cudaGetDeviceProperties_v2_v12000
1.1 1038424 6 173070.7 135794.0 4840 445143 174111.4 cudaMalloc
1.0 977961 4 244490.3 219145.5 1072 538598 283164.8 cuLibraryGetKernel
0.2 229567 838 273.9 180.0 90 48441 1670.5 cuGetProcAddress_v2
0.1 137239 36 3812.2 3146.0 2094 14667 2165.3 cudaMemsetAsync
0.1 96319 36 2675.5 2049.0 1833 7945 1134.7 cudaEventRecord
0.0 44693 1 44693.0 44693.0 44693 44693 0.0 cudaMemset
0.0 38138 180 211.9 130.0 70 12083 894.1 cuStreamGetCaptureInfo_v2
0.0 21040 4 5260.0 3141.0 1503 13255 5424.7 cudaDeviceSynchronize
0.0 9043 18 502.4 275.5 200 3386 741.5 cudaEventDestroy
0.0 5723 4 1430.8 1443.5 1163 1673 221.4 cuInit
0.0 3677 1 3677.0 3677.0 3677 3677 0.0 cudaEventQuery
0.0 1714 3 571.3 140.0 111 1463 772.3 cuModuleGetLoadingMode
0.0 941 2 470.5 470.5 300 641 241.1 cudaGetDriverEntryPoint_v11030
[6/8] Executing 'cuda_gpu_kern_sum' stats report
Time (%) Total Time (ns) Instances Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- -------- -------- -------- -------- ----------- ----------------------------------------------------------------------------------------------------
35.8 3686110 25 147444.4 147616.0 145600 149216 1035.4 layernorm_forward_kernel(float *, float *, float *, const float *, const float *, const float *, in…
16.2 1663328 13 127948.3 59392.0 58912 950752 247221.8 ampere_sgemm_128x64_tn
14.9 1536224 12 128018.7 128112.0 125600 129472 1083.3 ampere_sgemm_64x64_tn
11.1 1146944 12 95578.7 95616.0 94720 95904 297.0 preattention_scores_kernel(const float *, const float *, int, int, int, int, float *)
6.5 670624 12 55885.3 55840.0 55136 56928 530.5 void cutlass::Kernel<cutlass::gemm::kernel::Gemm<cutlass::gemm::threadblock::MmaPipelined<cutlass::…
3.9 398208 12 33184.0 33184.0 32928 33344 111.7 ampere_sgemm_64x32_sliced1x4_tn
3.3 335424 12 27952.0 27888.0 27296 28672 440.4 attention_v_kernel(float *, const float *, const float *, int, int, int, int)
2.7 279648 48 5826.0 4320.0 2848 12448 3668.6 bias_add_filler_kernel(float *, const float *, int, int)
2.0 205856 12 17154.7 17136.0 17024 17312 83.5 softmax_forward_kernel(float *, float, const float *, int, int)
1.4 141824 24 5909.3 5936.0 5600 6240 246.7 residual_forward_kernel(float *, const float *, const float *, int)
0.8 80928 1 80928.0 80928.0 80928 80928 0.0 encoder_forward_kernel(float *, const int *, const float *, const float *, int, int, int)
0.7 76896 12 6408.0 6384.0 6304 6592 84.3 permute_kernel(float *, float *, float *, const float *, int, int, int, int)
0.7 70688 12 5890.7 5888.0 5824 5952 37.3 unpermute_kernel(const float *, float *, int, int, int, int)
[7/8] Executing 'cuda_gpu_mem_time_sum' stats report
Time (%) Total Time (ns) Count Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Operation
-------- --------------- ----- ---------- ---------- -------- -------- ----------- ----------------------------
82.7 35891318 2 17945659.0 17945659.0 864 35890454 25377772.5 [CUDA memcpy Host-to-Device]
16.3 7061278 1 7061278.0 7061278.0 7061278 7061278 0.0 [CUDA memcpy Device-to-Host]
1.0 454592 37 12286.3 1184.0 480 408320 66917.2 [CUDA memset]
[8/8] Executing 'cuda_gpu_mem_size_sum' stats report
Total (MB) Count Avg (MB) Med (MB) Min (MB) Max (MB) StdDev (MB) Operation
---------- ----- -------- -------- -------- -------- ----------- ----------------------------
497.905 2 248.952 248.952 0.001 497.904 352.070 [CUDA memcpy Host-to-Device]
232.402 37 6.281 0.000 0.000 232.392 38.205 [CUDA memset]
51.511 1 51.511 51.511 51.511 51.511 0.000 [CUDA memcpy Device-to-Host]
Generated:
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_16.nsys-rep
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_16.sqlite
output/perf/nsys/verify_op_8.err
+5
View File
@@ -0,0 +1,5 @@
WARNING: Device-side CUDA Event completion trace is currently enabled.
This may increase runtime overhead and the likelihood of false
dependencies across CUDA Streams. If you wish to avoid this, please
disable the feature with --cuda-event-trace=false.
SKIPPED: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_8.sqlite does not contain NV Tools Extension (NVTX) data.
output/perf/nsys/verify_op_8.out
+113
View File
@@ -0,0 +1,113 @@
job 17968510 starting on gpub019.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000043, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
Collecting data...
Generating '/tmp/nsys-report-8179.qdstrm'
␍[1/8] [0% ] report_op_8.nsys-rep␍[1/8] [0% ] report_op_8.nsys-rep␍[1/8] [=16% ] report_op_8.nsys-rep␍[1/8] [=17% ] report_op_8.nsys-rep␍[1/8] [==19% ] report_op_8.nsys-rep␍[1/8] [=======36% ] report_op_8.nsys-rep␍[1/8] [===========53% ] report_op_8.nsys-rep␍[1/8] [============54% ] report_op_8.nsys-rep␍[1/8] [============55% ] report_op_8.nsys-rep␍[1/8] [============56% ] report_op_8.nsys-rep␍[1/8] [============57% ] report_op_8.nsys-rep␍[1/8] [==============62% ] report_op_8.nsys-rep␍[1/8] [==================78% ] report_op_8.nsys-rep␍[1/8] [======================92% ] report_op_8.nsys-rep␍[1/8] [========================100%] report_op_8.nsys-rep␍[1/8] [========================100%] report_op_8.nsys-rep
[2/8] [0% ] report_op_8.sqlite [2/8] [1% ] report_op_8.sqlite [2/8] [2% ] report_op_8.sqlite [2/8] [3% ] report_op_8.sqlite [2/8] [4% ] report_op_8.sqlite [2/8] [5% ] report_op_8.sqlite [2/8] [6% ] report_op_8.sqlite [2/8] [7% ] report_op_8.sqlite [2/8] [8% ] report_op_8.sqlite [2/8] [9% ] report_op_8.sqlite [2/8] [10% ] report_op_8.sqlite [2/8] [11% ] report_op_8.sqlite [2/8] [12% ] report_op_8.sqlite [2/8] [13% ] report_op_8.sqlite [2/8] [14% ] report_op_8.sqlite [2/8] [=15% ] report_op_8.sqlite [2/8] [=16% ] report_op_8.sqlite [2/8] [=17% ] report_op_8.sqlite [2/8] [==18% ] report_op_8.sqlite [2/8] [==19% ] report_op_8.sqlite [2/8] [==20% ] report_op_8.sqlite [2/8] [==21% ] report_op_8.sqlite [2/8] [===22% ] report_op_8.sqlite [2/8] [===23% ] report_op_8.sqlite [2/8] [===24% ] report_op_8.sqlite [2/8] [====25% ] report_op_8.sqlite [2/8] [====26% ] report_op_8.sqlite [2/8] [====27% ] report_op_8.sqlite [2/8] [====28% ] report_op_8.sqlite [2/8] [=====29% ] report_op_8.sqlite [2/8] [=====30% ] report_op_8.sqlite [2/8] [=====31% ] report_op_8.sqlite [2/8] [=====32% ] report_op_8.sqlite [2/8] [======33% ] report_op_8.sqlite [2/8] [======34% ] report_op_8.sqlite [2/8] [======35% ] report_op_8.sqlite [2/8] [=======36% ] report_op_8.sqlite [2/8] [=======37% ] report_op_8.sqlite [2/8] [=======38% ] report_op_8.sqlite [2/8] [=======39% ] report_op_8.sqlite [2/8] [========40% ] report_op_8.sqlite [2/8] [========41% ] report_op_8.sqlite [2/8] [========42% ] report_op_8.sqlite [2/8] [=========43% ] report_op_8.sqlite [2/8] [=========44% ] report_op_8.sqlite [2/8] [=========45% ] report_op_8.sqlite [2/8] [=========46% ] report_op_8.sqlite [2/8] [==========47% ] report_op_8.sqlite [2/8] [==========48% ] report_op_8.sqlite [2/8] [==========49% ] report_op_8.sqlite [2/8] [===========50% ] report_op_8.sqlite [2/8] [===========51% ] report_op_8.sqlite [2/8] [===========52% ] report_op_8.sqlite [2/8] [===========53% ] report_op_8.sqlite [2/8] [============54% ] report_op_8.sqlite [2/8] [============55% ] report_op_8.sqlite [2/8] [============56% ] report_op_8.sqlite [2/8] [============57% ] report_op_8.sqlite [2/8] [=============58% ] report_op_8.sqlite [2/8] [=============59% ] report_op_8.sqlite [2/8] [=============60% ] report_op_8.sqlite [2/8] [==============61% ] report_op_8.sqlite [2/8] [==============62% ] report_op_8.sqlite [2/8] [==============63% ] report_op_8.sqlite [2/8] [==============64% ] report_op_8.sqlite [2/8] [===============65% ] report_op_8.sqlite [2/8] [===============66% ] report_op_8.sqlite [2/8] [===============67% ] report_op_8.sqlite [2/8] [================68% ] report_op_8.sqlite [2/8] [================69% ] report_op_8.sqlite [2/8] [================70% ] report_op_8.sqlite [2/8] [================71% ] report_op_8.sqlite [2/8] [=================72% ] report_op_8.sqlite [2/8] [=================73% ] report_op_8.sqlite [2/8] [=================74% ] report_op_8.sqlite [2/8] [==================75% ] report_op_8.sqlite [2/8] [==================76% ] report_op_8.sqlite [2/8] [==================77% ] report_op_8.sqlite [2/8] [==================78% ] report_op_8.sqlite [2/8] [===================79% ] report_op_8.sqlite [2/8] [===================80% ] report_op_8.sqlite [2/8] [===================81% ] report_op_8.sqlite [2/8] [===================82% ] report_op_8.sqlite [2/8] [====================83% ] report_op_8.sqlite [2/8] [====================84% ] report_op_8.sqlite [2/8] [====================85% ] report_op_8.sqlite [2/8] [=====================86% ] report_op_8.sqlite [2/8] [=====================87% ] report_op_8.sqlite [2/8] [=====================88% ] report_op_8.sqlite [2/8] [=======Line truncated
[3/8] Executing 'nvtx_sum' stats report
[4/8] Executing 'osrt_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ----------------------
67.3 1022283375 20 51114168.8 27054794.0 1583 120736053 48517322.9 poll
20.5 310542000 6 51757000.0 64761.5 7243 218143736 89469059.2 fread
11.5 175188820 581 301529.8 23153.0 1042 36499184 1835892.4 ioctl
0.2 2594881 11 235898.3 31449.0 1071 1069584 376213.4 write
0.2 2357747 25 94309.9 9197.0 7644 1637859 324412.9 mmap64
0.1 1729901 38 45523.7 6337.0 3606 635861 134756.5 fopen
0.0 749602 32 23425.1 2199.5 1352 361046 82772.6 fclose
0.0 589738 17 34690.5 7565.0 1883 392946 93966.9 mmap
0.0 578494 1 578494.0 578494.0 578494 578494 0.0 pthread_cond_wait
0.0 374161 2 187080.5 187080.5 171201 202960 22457.0 pthread_create
0.0 331501 55 6027.3 5911.0 2294 15840 2589.0 open64
0.0 207579 1 207579.0 207579.0 207579 207579 0.0 fflush
0.0 124864 1 124864.0 124864.0 124864 124864 0.0 pthread_cond_broadcast
0.0 76323 1 76323.0 76323.0 76323 76323 0.0 fgets
0.0 44675 10 4467.5 3977.0 2856 7935 1692.1 munmap
0.0 39913 6 6652.2 6271.5 3116 9868 2660.1 open
0.0 25270 9 2807.8 2014.0 1022 7725 2162.0 read
0.0 21710 3 7236.7 8766.0 3636 9308 3130.0 pipe2
0.0 18114 2 9057.0 9057.0 5981 12133 4350.1 socket
0.0 12684 1 12684.0 12684.0 12684 12684 0.0 connect
0.0 6432 1 6432.0 6432.0 6432 6432 0.0 fopen64
0.0 3236 2 1618.0 1618.0 1453 1783 233.3 fwrite
0.0 3096 1 3096.0 3096.0 3096 3096 0.0 bind
0.0 1273 1 1273.0 1273.0 1273 1273 0.0 listen
0.0 1032 1 1032.0 1032.0 1032 1032 0.0 fcntl
[5/8] Executing 'cuda_api_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ---------------------------------
85.6 293521673 3 97840557.7 36810988.0 22002 256688683 138790859.1 cudaMemcpy
10.7 36666998 1 36666998.0 36666998.0 36666998 36666998 0.0 cudaMallocHost
1.9 6357156 12 529763.0 5770.5 191 2259966 809011.1 cudaFree
0.7 2461734 1 2461734.0 2461734.0 2461734 2461734 0.0 cudaFreeHost
0.5 1640935 1 1640935.0 1640935.0 1640935 1640935 0.0 cudaGetDeviceProperties_v2_v12000
0.3 1027820 171 6010.6 2836.0 2705 408266 31163.4 cudaLaunchKernel
0.2 808456 6 134742.7 121007.0 5671 361818 133007.2 cudaMalloc
0.1 236221 50 4724.4 4078.0 3597 14758 1955.6 cudaMemcpyToSymbolAsync
0.1 183980 838 219.5 180.0 80 5500 215.8 cuGetProcAddress_v2
0.0 35717 1 35717.0 35717.0 35717 35717 0.0 cudaMemset
0.0 22080 18 1226.7 265.0 250 11311 2818.0 cudaEventCreateWithFlags
0.0 13526 4 3381.5 2960.5 1123 6482 2246.3 cudaDeviceSynchronize
0.0 6731 18 373.9 240.0 210 1874 397.0 cudaEventDestroy
0.0 5430 4 1357.5 1362.5 1192 1513 131.5 cuInit
0.0 1493 3 497.7 120.0 120 1253 654.1 cuModuleGetLoadingMode
0.0 1251 2 625.5 625.5 280 971 488.6 cudaGetDriverEntryPoint_v11030
[6/8] Executing 'cuda_gpu_kern_sum' stats report
Time (%) Total Time (ns) Instances Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- --------- --------- -------- -------- ----------- -----------------------------------------------------------------------------------------------
97.8 243408998 49 4967530.6 4539333.0 1227137 73740560 10131683.4 matmul_forward_kernel(float *, const float *, const float *, const float *, int, int, int, int)
1.3 3217858 25 128714.3 128672.0 123872 131969 2081.9 layernorm_forward_kernel(float *, float *, float *, const float *, int, int, int)
0.5 1152128 12 96010.7 96016.0 95840 96192 115.2 preattention_scores_kernel(const float *, const float *, int, int, int, int, float *)
0.1 346496 12 28874.7 28800.0 28160 29824 545.5 attention_v_kernel(float *, const float *, const float *, int, int, int, int)
0.1 226400 12 18866.7 18880.0 18656 19008 83.5 gelu_forward_kernel(float *, const float *, int)
0.1 209731 12 17477.6 17488.5 17280 17696 134.1 softmax_forward_kernel(float *, float, const float *, int, int)
0.1 145122 24 6046.8 6064.0 5632 6560 361.4 residual_forward_kernel(float *, const float *, const float *, int)
0.0 96960 12 8080.0 8016.0 7936 8288 135.4 permute_kernel(float *, float *, float *, const float *, int, int, int, int)
0.0 79648 1 79648.0 79648.0 79648 79648 0.0 encoder_forward_kernel(float *, const int *, const float *, const float *, int, int, int)
0.0 70560 12 5880.0 5888.0 5856 5888 14.5 unpermute_kernel(const float *, float *, int, int, int, int)
[7/8] Executing 'cuda_gpu_mem_time_sum' stats report
Time (%) Total Time (ns) Count Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Operation
-------- --------------- ----- ---------- ---------- -------- -------- ----------- ------------------------------
80.7 36707943 2 18353971.5 18353971.5 864 36707079 25955213.5 [CUDA memcpy Host-to-Device]
18.2 8280169 1 8280169.0 8280169.0 8280169 8280169 0.0 [CUDA memcpy Device-to-Host]
0.9 407233 1 407233.0 407233.0 407233 407233 0.0 [CUDA memset]
0.2 76416 50 1528.3 1472.0 1376 1952 143.8 [CUDA memcpy Device-to-Device]
[8/8] Executing 'cuda_gpu_mem_size_sum' stats report
Total (MB) Count Avg (MB) Med (MB) Min (MB) Max (MB) StdDev (MB) Operation
---------- ----- -------- -------- -------- -------- ----------- ------------------------------
497.905 2 248.952 248.952 0.001 497.904 352.070 [CUDA memcpy Host-to-Device]
232.392 1 232.392 232.392 232.392 232.392 0.000 [CUDA memset]
51.511 1 51.511 51.511 51.511 51.511 0.000 [CUDA memcpy Device-to-Host]
0.154 50 0.003 0.003 0.003 0.003 0.000 [CUDA memcpy Device-to-Device]
Generated:
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_8.nsys-rep
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_8.sqlite
output/perf/nsys/verify_op_9.err
+5
View File
@@ -0,0 +1,5 @@
WARNING: Device-side CUDA Event completion trace is currently enabled.
This may increase runtime overhead and the likelihood of false
dependencies across CUDA Streams. If you wish to avoid this, please
disable the feature with --cuda-event-trace=false.
SKIPPED: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_9.sqlite does not contain NV Tools Extension (NVTX) data.
output/perf/nsys/verify_op_9.out
+109
View File
@@ -0,0 +1,109 @@
job 17968383 starting on gpub013.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000043, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
Collecting data...
Generating '/tmp/nsys-report-ec00.qdstrm'
␍[1/8] [0% ] report_op_9.nsys-rep␍[1/8] [0% ] report_op_9.nsys-rep␍[1/8] [5% ] report_op_9.nsys-rep␍[1/8] [====28% ] report_op_9.nsys-rep␍[1/8] [===========51% ] report_op_9.nsys-rep␍[1/8] [======34% ] report_op_9.nsys-rep␍[1/8] [===========51% ] report_op_9.nsys-rep␍[1/8] [===========52% ] report_op_9.nsys-rep␍[1/8] [===========53% ] report_op_9.nsys-rep␍[1/8] [============54% ] report_op_9.nsys-rep␍[1/8] [============55% ] report_op_9.nsys-rep␍[1/8] [============56% ] report_op_9.nsys-rep␍[1/8] [============57% ] report_op_9.nsys-rep␍[1/8] [=============58% ] report_op_9.nsys-rep␍[1/8] [=============59% ] report_op_9.nsys-rep␍[1/8] [==================77% ] report_op_9.nsys-rep␍[1/8] [======================91% ] report_op_9.nsys-rep␍[1/8] [========================100%] report_op_9.nsys-rep␍[1/8] [========================100%] report_op_9.nsys-rep
[2/8] [0% ] report_op_9.sqlite [2/8] [1% ] report_op_9.sqlite [2/8] [2% ] report_op_9.sqlite [2/8] [3% ] report_op_9.sqlite [2/8] [4% ] report_op_9.sqlite [2/8] [5% ] report_op_9.sqlite [2/8] [6% ] report_op_9.sqlite [2/8] [7% ] report_op_9.sqlite [2/8] [8% ] report_op_9.sqlite [2/8] [9% ] report_op_9.sqlite [2/8] [10% ] report_op_9.sqlite [2/8] [11% ] report_op_9.sqlite [2/8] [12% ] report_op_9.sqlite [2/8] [13% ] report_op_9.sqlite [2/8] [14% ] report_op_9.sqlite [2/8] [=15% ] report_op_9.sqlite [2/8] [=16% ] report_op_9.sqlite [2/8] [=17% ] report_op_9.sqlite [2/8] [==18% ] report_op_9.sqlite [2/8] [==19% ] report_op_9.sqlite [2/8] [==20% ] report_op_9.sqlite [2/8] [==21% ] report_op_9.sqlite [2/8] [===22% ] report_op_9.sqlite [2/8] [===23% ] report_op_9.sqlite [2/8] [===24% ] report_op_9.sqlite [2/8] [====25% ] report_op_9.sqlite [2/8] [====26% ] report_op_9.sqlite [2/8] [====27% ] report_op_9.sqlite [2/8] [====28% ] report_op_9.sqlite [2/8] [=====29% ] report_op_9.sqlite [2/8] [=====30% ] report_op_9.sqlite [2/8] [=====31% ] report_op_9.sqlite [2/8] [=====32% ] report_op_9.sqlite [2/8] [======33% ] report_op_9.sqlite [2/8] [======34% ] report_op_9.sqlite [2/8] [======35% ] report_op_9.sqlite [2/8] [=======36% ] report_op_9.sqlite [2/8] [=======37% ] report_op_9.sqlite [2/8] [=======38% ] report_op_9.sqlite [2/8] [=======39% ] report_op_9.sqlite [2/8] [========40% ] report_op_9.sqlite [2/8] [========41% ] report_op_9.sqlite [2/8] [========42% ] report_op_9.sqlite [2/8] [=========43% ] report_op_9.sqlite [2/8] [=========44% ] report_op_9.sqlite [2/8] [=========45% ] report_op_9.sqlite [2/8] [=========46% ] report_op_9.sqlite [2/8] [==========47% ] report_op_9.sqlite [2/8] [==========48% ] report_op_9.sqlite [2/8] [==========49% ] report_op_9.sqlite [2/8] [===========50% ] report_op_9.sqlite [2/8] [===========51% ] report_op_9.sqlite [2/8] [===========52% ] report_op_9.sqlite [2/8] [===========53% ] report_op_9.sqlite [2/8] [============54% ] report_op_9.sqlite [2/8] [============55% ] report_op_9.sqlite [2/8] [============56% ] report_op_9.sqlite [2/8] [============57% ] report_op_9.sqlite [2/8] [=============58% ] report_op_9.sqlite [2/8] [=============59% ] report_op_9.sqlite [2/8] [=============60% ] report_op_9.sqlite [2/8] [==============61% ] report_op_9.sqlite [2/8] [==============62% ] report_op_9.sqlite [2/8] [==============63% ] report_op_9.sqlite [2/8] [==============64% ] report_op_9.sqlite [2/8] [===============65% ] report_op_9.sqlite [2/8] [===============66% ] report_op_9.sqlite [2/8] [===============67% ] report_op_9.sqlite [2/8] [================68% ] report_op_9.sqlite [2/8] [================69% ] report_op_9.sqlite [2/8] [================70% ] report_op_9.sqlite [2/8] [================71% ] report_op_9.sqlite [2/8] [=================72% ] report_op_9.sqlite [2/8] [=================73% ] report_op_9.sqlite [2/8] [=================74% ] report_op_9.sqlite [2/8] [==================75% ] report_op_9.sqlite [2/8] [==================76% ] report_op_9.sqlite [2/8] [==================77% ] report_op_9.sqlite [2/8] [==================78% ] report_op_9.sqlite [2/8] [===================79% ] report_op_9.sqlite [2/8] [===================80% ] report_op_9.sqlite [2/8] [===================81% ] report_op_9.sqlite [2/8] [===================82% ] report_op_9.sqlite [2/8] [====================83% ] report_op_9.sqlite [2/8] [====================84% ] report_op_9.sqlite [2/8] [====================85% ] report_op_9.sqlite [2/8] [=====================86% ] report_op_9.sqlite [2/8] [=====================87% ] report_op_9.sqlite [2/8] [=====================88% ] report_op_9.sqlite [2/8] [=======Line truncated
[3/8] Executing 'nvtx_sum' stats report
[4/8] Executing 'osrt_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ----------------------
72.6 792125713 18 44006984.1 22081891.5 1563 100126637 45893643.6 poll
12.9 141279538 6 23546589.7 15352175.0 7734 76623723 30441839.7 fread
12.4 135292558 580 233263.0 22111.5 1042 20047399 980798.2 ioctl
0.8 8667478 38 228091.5 6211.5 3536 7839121 1269544.8 fopen
0.6 7020827 1 7020827.0 7020827.0 7020827 7020827 0.0 pthread_cond_broadcast
0.2 2351517 25 94060.7 8677.0 7273 1636246 324069.4 mmap64
0.2 1904648 10 190464.8 28103.0 21490 803937 283644.9 write
0.1 655628 32 20488.4 2274.0 1373 286927 69823.0 fclose
0.1 586901 17 34523.6 4980.0 2384 383689 91890.5 mmap
0.0 336080 2 168040.0 168040.0 147897 188183 28486.5 pthread_create
0.0 322394 55 5861.7 5971.0 2254 13936 2322.7 open64
0.0 222768 1 222768.0 222768.0 222768 222768 0.0 fflush
0.0 80561 1 80561.0 80561.0 80561 80561 0.0 pthread_cond_wait
0.0 77105 1 77105.0 77105.0 77105 77105 0.0 fgets
0.0 38674 6 6445.7 6247.0 3537 9528 2196.9 open
0.0 36928 8 4616.0 4478.0 2985 6963 1491.7 munmap
0.0 25617 2 12808.5 12808.5 11451 14166 1919.8 socket
0.0 20939 3 6979.7 8236.0 4388 8315 2244.8 pipe2
0.0 14717 7 2102.4 1914.0 1032 3597 1008.0 read
0.0 10630 1 10630.0 10630.0 10630 10630 0.0 connect
0.0 6502 1 6502.0 6502.0 6502 6502 0.0 fopen64
0.0 3517 2 1758.5 1758.5 1343 2174 587.6 fwrite
0.0 1983 1 1983.0 1983.0 1983 1983 0.0 bind
0.0 1202 1 1202.0 1202.0 1202 1202 0.0 fcntl
[5/8] Executing 'cuda_api_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ---------------------------------
95.2 292665484 3 97555161.3 36209486.0 21961 256434037 138777725.5 cudaMemcpy
1.7 5116658 12 426388.2 6582.0 170 1779706 610228.6 cudaFree
1.0 3123114 171 18263.8 2885.0 2755 2481560 189524.2 cudaLaunchKernel
0.8 2449450 1 2449450.0 2449450.0 2449450 2449450 0.0 cudaFreeHost
0.5 1532732 1 1532732.0 1532732.0 1532732 1532732 0.0 cudaGetDeviceProperties_v2_v12000
0.5 1478801 1 1478801.0 1478801.0 1478801 1478801 0.0 cudaMallocHost
0.3 816982 6 136163.7 117796.0 3527 382948 140753.7 cudaMalloc
0.1 187447 838 223.7 151.0 70 32301 1113.0 cuGetProcAddress_v2
0.0 34615 1 34615.0 34615.0 34615 34615 0.0 cudaMemset
0.0 16342 18 907.9 305.0 250 6673 1587.6 cudaEventCreateWithFlags
0.0 13825 4 3456.3 3000.0 1032 6793 2455.7 cudaDeviceSynchronize
0.0 7285 18 404.7 260.5 200 2224 469.9 cudaEventDestroy
0.0 5220 4 1305.0 1287.5 1152 1493 140.7 cuInit
0.0 1262 3 420.7 140.0 100 1022 521.2 cuModuleGetLoadingMode
0.0 831 2 415.5 415.5 250 581 234.1 cudaGetDriverEntryPoint_v11030
[6/8] Executing 'cuda_gpu_kern_sum' stats report
Time (%) Total Time (ns) Instances Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- --------- --------- -------- -------- ----------- ----------------------------------------------------------------------------------------------------
97.6 243432571 49 4968011.7 4540305.0 1227228 73745324 10132336.9 matmul_forward_kernel(float *, const float *, const float *, const float *, int, int, int, int)
1.4 3577301 25 143092.0 143039.0 140960 146752 1309.3 layernorm_forward_kernel(float *, float *, float *, const float *, const float *, const float *, in…
0.5 1152060 12 96005.0 95984.0 95872 96128 81.5 preattention_scores_kernel(const float *, const float *, int, int, int, int, float *)
0.1 347807 12 28983.9 28816.0 28575 29984 463.4 attention_v_kernel(float *, const float *, const float *, int, int, int, int)
0.1 227615 12 18967.9 18960.0 18752 19072 88.6 gelu_forward_kernel(float *, const float *, int)
0.1 209791 12 17482.6 17488.0 17280 17888 184.8 softmax_forward_kernel(float *, float, const float *, int, int)
0.1 146176 24 6090.7 6144.0 5696 6560 350.5 residual_forward_kernel(float *, const float *, const float *, int)
0.0 97408 12 8117.3 8112.0 7872 8320 116.0 permute_kernel(float *, float *, float *, const float *, int, int, int, int)
0.0 79679 1 79679.0 79679.0 79679 79679 0.0 encoder_forward_kernel(float *, const int *, const float *, const float *, int, int, int)
0.0 70528 12 5877.3 5888.0 5856 5888 15.8 unpermute_kernel(const float *, float *, int, int, int, int)
[7/8] Executing 'cuda_gpu_mem_time_sum' stats report
Time (%) Total Time (ns) Count Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Operation
-------- --------------- ----- ---------- ---------- -------- -------- ----------- ----------------------------
82.0 36088520 2 18044260.0 18044260.0 864 36087656 25517215.3 [CUDA memcpy Host-to-Device]
17.0 7499463 1 7499463.0 7499463.0 7499463 7499463 0.0 [CUDA memcpy Device-to-Host]
0.9 407838 1 407838.0 407838.0 407838 407838 0.0 [CUDA memset]
[8/8] Executing 'cuda_gpu_mem_size_sum' stats report
Total (MB) Count Avg (MB) Med (MB) Min (MB) Max (MB) StdDev (MB) Operation
---------- ----- -------- -------- -------- -------- ----------- ----------------------------
497.905 2 248.952 248.952 0.001 497.904 352.070 [CUDA memcpy Host-to-Device]
232.392 1 232.392 232.392 232.392 232.392 0.000 [CUDA memset]
51.511 1 51.511 51.511 51.511 51.511 0.000 [CUDA memcpy Device-to-Host]
Generated:
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_9.nsys-rep
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_op_9.sqlite
output/perf/nsys/verify_req_4.err
+5
View File
@@ -0,0 +1,5 @@
WARNING: Device-side CUDA Event completion trace is currently enabled.
This may increase runtime overhead and the likelihood of false
dependencies across CUDA Streams. If you wish to avoid this, please
disable the feature with --cuda-event-trace=false.
SKIPPED: /u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_req_4.sqlite does not contain NV Tools Extension (NVTX) data.
output/perf/nsys/verify_req_4.out
+107
View File
@@ -0,0 +1,107 @@
job 17971938 starting on gpub022.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000069, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
Collecting data...
Generating '/tmp/nsys-report-9a09.qdstrm'
␍[1/8] [0% ] report_req_4.nsys-rep␍[1/8] [0% ] report_req_4.nsys-rep␍[1/8] [5% ] report_req_4.nsys-rep␍[1/8] [====28% ] report_req_4.nsys-rep␍[1/8] [===========50% ] report_req_4.nsys-rep␍[1/8] [===========51% ] report_req_4.nsys-rep␍[1/8] [======34% ] report_req_4.nsys-rep␍[1/8] [===========51% ] report_req_4.nsys-rep␍[1/8] [===========52% ] report_req_4.nsys-rep␍[1/8] [===========53% ] report_req_4.nsys-rep␍[1/8] [=============59% ] report_req_4.nsys-rep␍[1/8] [==================77% ] report_req_4.nsys-rep␍[1/8] [======================91% ] report_req_4.nsys-rep␍[1/8] [========================100%] report_req_4.nsys-rep␍[1/8] [========================100%] report_req_4.nsys-rep
[2/8] [0% ] report_req_4.sqlite [2/8] [1% ] report_req_4.sqlite [2/8] [2% ] report_req_4.sqlite [2/8] [3% ] report_req_4.sqlite [2/8] [4% ] report_req_4.sqlite [2/8] [5% ] report_req_4.sqlite [2/8] [6% ] report_req_4.sqlite [2/8] [7% ] report_req_4.sqlite [2/8] [8% ] report_req_4.sqlite [2/8] [9% ] report_req_4.sqlite [2/8] [10% ] report_req_4.sqlite [2/8] [11% ] report_req_4.sqlite [2/8] [12% ] report_req_4.sqlite [2/8] [13% ] report_req_4.sqlite [2/8] [14% ] report_req_4.sqlite [2/8] [=15% ] report_req_4.sqlite [2/8] [=16% ] report_req_4.sqlite [2/8] [=17% ] report_req_4.sqlite [2/8] [==18% ] report_req_4.sqlite [2/8] [==19% ] report_req_4.sqlite [2/8] [==20% ] report_req_4.sqlite [2/8] [==21% ] report_req_4.sqlite [2/8] [===22% ] report_req_4.sqlite [2/8] [===23% ] report_req_4.sqlite [2/8] [===24% ] report_req_4.sqlite [2/8] [====25% ] report_req_4.sqlite [2/8] [====26% ] report_req_4.sqlite [2/8] [====27% ] report_req_4.sqlite [2/8] [====28% ] report_req_4.sqlite [2/8] [=====29% ] report_req_4.sqlite [2/8] [=====30% ] report_req_4.sqlite [2/8] [=====31% ] report_req_4.sqlite [2/8] [=====32% ] report_req_4.sqlite [2/8] [======33% ] report_req_4.sqlite [2/8] [======34% ] report_req_4.sqlite [2/8] [======35% ] report_req_4.sqlite [2/8] [=======36% ] report_req_4.sqlite [2/8] [=======37% ] report_req_4.sqlite [2/8] [=======38% ] report_req_4.sqlite [2/8] [=======39% ] report_req_4.sqlite [2/8] [========40% ] report_req_4.sqlite [2/8] [========41% ] report_req_4.sqlite [2/8] [========42% ] report_req_4.sqlite [2/8] [=========43% ] report_req_4.sqlite [2/8] [=========44% ] report_req_4.sqlite [2/8] [=========45% ] report_req_4.sqlite [2/8] [=========46% ] report_req_4.sqlite [2/8] [==========47% ] report_req_4.sqlite [2/8] [==========48% ] report_req_4.sqlite [2/8] [==========49% ] report_req_4.sqlite [2/8] [===========50% ] report_req_4.sqlite [2/8] [===========51% ] report_req_4.sqlite [2/8] [===========52% ] report_req_4.sqlite [2/8] [===========53% ] report_req_4.sqlite [2/8] [============54% ] report_req_4.sqlite [2/8] [============55% ] report_req_4.sqlite [2/8] [============56% ] report_req_4.sqlite [2/8] [============57% ] report_req_4.sqlite [2/8] [=============58% ] report_req_4.sqlite [2/8] [=============59% ] report_req_4.sqlite [2/8] [=============60% ] report_req_4.sqlite [2/8] [==============61% ] report_req_4.sqlite [2/8] [==============62% ] report_req_4.sqlite [2/8] [==============63% ] report_req_4.sqlite [2/8] [==============64% ] report_req_4.sqlite [2/8] [===============65% ] report_req_4.sqlite [2/8] [===============66% ] report_req_4.sqlite [2/8] [===============67% ] report_req_4.sqlite [2/8] [================68% ] report_req_4.sqlite [2/8] [================69% ] report_req_4.sqlite [2/8] [================70% ] report_req_4.sqlite [2/8] [================71% ] report_req_4.sqlite [2/8] [=================72% ] report_req_4.sqlite [2/8] [=================73% ] report_req_4.sqlite [2/8] [=================74% ] report_req_4.sqlite [2/8] [==================75% ] report_req_4.sqlite [2/8] [==================76% ] report_req_4.sqlite [2/8] [==================77% ] report_req_4.sqlite [2/8] [==================78% ] report_req_4.sqlite [2/8] [===================79% ] report_req_4.sqlite [2/8] [===================80% ] report_req_4.sqlite [2/8] [===================81% ] report_req_4.sqlite [2/8] [===================82% ] report_req_4.sqlite [2/8] [====================83% ] report_req_4.sqlite [2/8] [====================84% ] report_req_4.sqlite [2/8] [====================85% ] report_req_4.sqlite [2/8] [=====================86% ] report_req_4.sqlite [2/8] [=====================87% ] reLine truncated
[3/8] Executing 'nvtx_sum' stats report
[4/8] Executing 'osrt_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ----------------------
72.0 706271722 17 41545395.4 21764331.0 2455 105577183 46178357.9 poll
16.0 157480946 581 271051.5 23474.0 1022 21596356 1265979.2 ioctl
9.7 95519940 6 15919990.0 69450.0 7424 87023730 34991208.6 fread
0.7 7116288 1 7116288.0 7116288.0 7116288 7116288 0.0 pthread_cond_broadcast
0.7 6683027 1 6683027.0 6683027.0 6683027 6683027 0.0 pthread_cond_wait
0.2 2395073 25 95802.9 8977.0 7164 1672329 331103.1 mmap64
0.2 2118124 11 192556.7 27401.0 1062 855742 300169.0 write
0.1 1406722 38 37019.0 6271.5 3787 518551 104526.5 fopen
0.1 660508 32 20640.9 2259.0 1383 314419 71634.1 fclose
0.1 601947 17 35408.6 4919.0 2525 384280 91907.4 mmap
0.0 360014 2 180007.0 180007.0 167553 192461 17612.6 pthread_create
0.0 336551 55 6119.1 6021.0 2394 16411 2749.7 open64
0.0 212838 1 212838.0 212838.0 212838 212838 0.0 fflush
0.0 78526 1 78526.0 78526.0 78526 78526 0.0 fgets
0.0 41037 8 5129.6 4934.5 3036 7744 1719.8 munmap
0.0 38151 6 6358.5 6477.0 3276 9107 2070.2 open
0.0 22041 3 7347.0 9007.0 3927 9107 2962.2 pipe2
0.0 21344 12 1778.7 1213.0 1022 3978 999.8 read
0.0 21000 1 21000.0 21000.0 21000 21000 0.0 connect
0.0 20027 2 10013.5 10013.5 8365 11662 2331.3 socket
0.0 6553 1 6553.0 6553.0 6553 6553 0.0 fopen64
0.0 3175 2 1587.5 1587.5 1523 1652 91.2 fwrite
0.0 2415 1 2415.0 2415.0 2415 2415 0.0 bind
0.0 2125 2 1062.5 1062.5 1032 1093 43.1 fcntl
[5/8] Executing 'cuda_api_sum' stats report
Time (%) Total Time (ns) Num Calls Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- ---------- ---------- -------- --------- ----------- ---------------------------------
83.6 293033557 3 97677852.3 36393158.0 19847 256620552 138844664.2 cudaMemcpy
8.4 29603500 147 201384.4 2815.0 2685 28629047 2361040.2 cudaLaunchKernel
4.5 15916639 6 2652773.2 122850.5 3066 15473999 6281511.1 cudaMalloc
1.5 5351312 12 445942.7 8245.5 201 1942786 645982.5 cudaFree
0.7 2445367 1 2445367.0 2445367.0 2445367 2445367 0.0 cudaFreeHost
0.6 1973533 1 1973533.0 1973533.0 1973533 1973533 0.0 cudaGetDeviceProperties_v2_v12000
0.5 1862896 1 1862896.0 1862896.0 1862896 1862896 0.0 cudaMallocHost
0.1 192866 838 230.2 191.0 90 7394 351.2 cuGetProcAddress_v2
0.0 37921 1 37921.0 37921.0 37921 37921 0.0 cudaMemset
0.0 12805 4 3201.3 2419.5 1223 6743 2437.3 cudaDeviceSynchronize
0.0 11563 18 642.4 300.5 270 2083 678.8 cudaEventCreateWithFlags
0.0 7813 18 434.1 315.0 230 2214 459.8 cudaEventDestroy
0.0 5469 4 1367.3 1367.0 1232 1503 116.6 cuInit
0.0 1773 3 591.0 130.0 110 1533 815.9 cuModuleGetLoadingMode
0.0 942 2 471.0 471.0 260 682 298.4 cudaGetDriverEntryPoint_v11030
[6/8] Executing 'cuda_gpu_kern_sum' stats report
Time (%) Total Time (ns) Instances Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Name
-------- --------------- --------- --------- --------- -------- -------- ----------- ----------------------------------------------------------------------------------------------------
97.3 243465041 49 4968674.3 4541726.0 1227616 73781940 10137437.2 matmul_forward_kernel(float *, const float *, const float *, const float *, int, int, int, int)
1.4 3574684 25 142987.4 142944.0 141408 147104 1126.4 layernorm_forward_kernel(float *, float *, float *, const float *, const float *, const float *, in…
1.0 2583647 12 215303.9 215328.0 214879 215392 136.1 void flash_attention_forward_kernel<(int)64, (int)64, (int)64>(const float *, const float *, const …
0.1 227840 12 18986.7 19008.0 18688 19136 115.2 gelu_forward_kernel(float *, const float *, int)
0.1 146144 24 6089.3 6128.0 5696 6464 346.6 residual_forward_kernel(float *, const float *, const float *, int)
0.0 79776 12 6648.0 6640.0 6464 6816 119.1 permute_kernel(float *, float *, float *, const float *, int, int, int, int)
0.0 79296 1 79296.0 79296.0 79296 79296 0.0 encoder_forward_kernel(float *, const int *, const float *, const float *, int, int, int)
0.0 36640 12 3053.3 3040.0 3040 3104 21.4 unpermute_kernel(const float *, float *, int, int, int, int)
[7/8] Executing 'cuda_gpu_mem_time_sum' stats report
Time (%) Total Time (ns) Count Avg (ns) Med (ns) Min (ns) Max (ns) StdDev (ns) Operation
-------- --------------- ----- ---------- ---------- -------- -------- ----------- ----------------------------
82.7 36306187 2 18153093.5 18153093.5 832 36305355 25671174.4 [CUDA memcpy Host-to-Device]
16.4 7213212 1 7213212.0 7213212.0 7213212 7213212 0.0 [CUDA memcpy Device-to-Host]
0.9 407200 1 407200.0 407200.0 407200 407200 0.0 [CUDA memset]
[8/8] Executing 'cuda_gpu_mem_size_sum' stats report
Total (MB) Count Avg (MB) Med (MB) Min (MB) Max (MB) StdDev (MB) Operation
---------- ----- -------- -------- -------- -------- ----------- ----------------------------
497.905 2 248.952 248.952 0.001 497.904 352.070 [CUDA memcpy Host-to-Device]
232.392 1 232.392 232.392 232.392 232.392 0.000 [CUDA memset]
51.511 1 51.511 51.511 51.511 51.511 0.000 [CUDA memcpy Device-to-Host]
Generated:
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_req_4.nsys-rep
/u/asingh41/sp26_ece408_Temperature1.5/output/perf/reports/report_req_4.sqlite
output/simple/verify_op_11.err
View File
Whitespace-only changes.
output/simple/verify_op_11.out
+21
View File
@@ -0,0 +1,21 @@
job 18002664 is starting on gpub022.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000069, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
output/simple/verify_op_12.err
View File
Whitespace-only changes.
output/simple/verify_op_12.out
+21
View File
@@ -0,0 +1,21 @@
job 18002591 is starting on gpub089.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431793
-39.836346, -39.836502
-43.065910, -43.066078
-42.828045, -42.828197
-43.529541, -43.529701
-44.318398, -44.318558
-41.227425, -41.227592
-41.270760, -41.270931
-42.541393, -42.541584
-42.394997, -42.395176
---------------- Logits OK! (max err 0.000050, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
output/simple/verify_op_13.err
View File
Whitespace-only changes.
output/simple/verify_op_13.out
+21
View File
@@ -0,0 +1,21 @@
job 18003856 is starting on gpub022.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000043, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
output/simple/verify_op_16.err
View File
Whitespace-only changes.
output/simple/verify_op_16.out
+21
View File
@@ -0,0 +1,21 @@
job 18004581 is starting on gpub035.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.315117
-39.836346, -39.732384
-43.065910, -42.960098
-42.828045, -42.723515
-43.529541, -43.418613
-44.318398, -44.210033
-41.227425, -41.119377
-41.270760, -41.164490
-42.541393, -42.428894
-42.394997, -42.288929
---------------- Logits OK! (max err 0.082334, RMSE 0.002728) ----------------
_________________________ All tests passed! _________________________
output/simple/verify_op_8.err
View File
Whitespace-only changes.
output/simple/verify_op_8.out
+21
View File
@@ -0,0 +1,21 @@
job 17968509 is starting on gpub005.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000043, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
output/simple/verify_op_9.err
View File
Whitespace-only changes.
output/simple/verify_op_9.out
+21
View File
@@ -0,0 +1,21 @@
job 17968382 is starting on gpub003.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000043, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________
output/simple/verify_req_4.err
View File
Whitespace-only changes.
output/simple/verify_req_4.out
+21
View File
@@ -0,0 +1,21 @@
job 17971913 is starting on gpub022.delta.ncsa.illinois.edu
test-gpt2-kernels - TF32 Mode is turned OFF.
test-gpt2 - [State]
Batch Size: 4
Sequence Length: 64
GPT2 - Allocated 221 MiB for activations
test-gpt2 - Printing 10 Expected vs Actual Logits:
-43.431618, -43.431740
-39.836346, -39.836452
-43.065910, -43.066021
-42.828045, -42.828144
-43.529541, -43.529644
-44.318398, -44.318497
-41.227425, -41.227528
-41.270760, -41.270863
-42.541393, -42.541523
-42.394997, -42.395119
---------------- Logits OK! (max err 0.000069, RMSE 0.000002) ----------------
_________________________ All tests passed! _________________________