[WIP] DecoderMaskedMultiHeadAttention CPU kernel. #22292
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| unit_cost.compute_cycles = static_cast<double>((SafeInt<ptrdiff_t>(2) * head_size - 1) * total_sequence_length); | ||
| unit_cost.bytes_loaded = static_cast<double>(SafeInt<ptrdiff_t>(2) * head_size * total_sequence_length * sizeof(T)); | ||
| unit_cost.bytes_stored = static_cast<double>(head_size * total_sequence_length * sizeof(T)); |
Check warning
Code scanning / PREfast
Arithmetic overflow: Using operator '' on a 4 byte value and then casting the result to a 8 byte value. Cast the value to the wider type before calling operator '' to avoid overflow (io.2). Warning
|
|
||
| unit_cost.compute_cycles = static_cast<double>((SafeInt<ptrdiff_t>(2) * head_size - 1) * total_sequence_length); | ||
| unit_cost.bytes_loaded = static_cast<double>(SafeInt<ptrdiff_t>(2) * head_size * total_sequence_length * sizeof(T)); | ||
| unit_cost.bytes_stored = static_cast<double>(head_size * total_sequence_length * sizeof(T)); |
Check warning
Code scanning / PREfast
Arithmetic overflow: Using operator '' on a 4 byte value and then casting the result to a 8 byte value. Cast the value to the wider type before calling operator '' to avoid overflow (io.2). Warning
| unit_cost.bytes_stored = static_cast<double>(head_size * total_sequence_length * sizeof(T)); | ||
|
|
||
| if (attn_bias_data != nullptr) { | ||
| unit_cost.bytes_loaded += probs_matrix_bytes * 2; |
Check warning
Code scanning / PREfast
Arithmetic overflow: Using operator '' on a 4 byte value and then casting the result to a 8 byte value. Cast the value to the wider type before calling operator '' to avoid overflow (io.2). Warning
|
|
||
| math::Scale<T, CPUMathUtil>(v_head_size, | ||
| static_cast<float>(*attn_probs_ptr), | ||
| past_value_vec + j * v_head_size, |
Check warning
Code scanning / PREfast
Arithmetic overflow: Using operator '' on a 4 byte value and then casting the result to a 8 byte value. Cast the value to the wider type before calling operator '' to avoid overflow (io.2). Warning
Description
DecoderMaskedMultiHeadAttention CPU kernel.