sleepy/llama.cpp
1
0
Fork 0
Code Issues 16 Pull Requests Actions 224 Packages Projects Releases Wiki Activity

vulkan: preprocess mul_mat_id experts and discard workgroups more quickly (#18352)

Browse Source 
Run a preprocess to count how many times each expert is used, and use this to
quickly discard workgroups that aren't needed.
This commit is contained in:
Jeff Bolz
2025-12-26 16:12:58 -06:00
committed by GitHub
parent 7ac8902133
commit c9ced4910b
7 changed files with 294 additions and 184 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
+2
View File
@@ -945,6 +945,8 @@ void process_shaders() {
string_to_spv("count_equal_i32", "count_equal.comp", merge_maps(base_dict, {{"A_TYPE", "int"}, {"B_TYPE", "int"}, {"D_TYPE", "int"}}));
string_to_spv("cumsum_f32", "cumsum.comp", merge_maps(base_dict, {{"A_TYPE", "float"}, {"D_TYPE", "float"}}));
string_to_spv("count_experts", "count_experts.comp", merge_maps(base_dict, {{"A_TYPE", "uint"}, {"D_TYPE", "uint"}}));
for (std::string dim_str : {"", "_3d"}) {
for (bool bda : {false, true}) {
std::string bda_str = bda ? "_bda" : "";