| Meno: | Jana Viktória |
|---|---|
| Priezvisko: | Kováčiková |
| Názov: | Iterative blocked pruning of neural networks |
| Vedúci: | Mgr. Vladimír Boľa, PhD. |
| Rok: | 2025 |
| Kµúčové slová: | block pruning, iterative pruning, neural networks, structured sparsity, Wide Residual Network |
| Abstrakt: | In modern neural networks, a large portion of computational effort is consumed by matrix multiplications involving weight matrices and activations. While unstructured pruning techniques can remove a high percentage of individual weights with minimal loss in accuracy, they suffer from high indexing overhead and limited compatibility with existing hardware due to irregular sparsity patterns. In contrast, structured pruning - particularly block pruning - has emerged as a promising alternative that aligns better with hardware acceleration, though it often incurs greater accuracy degradation. This thesis investigates iterative block pruning strategies, particularly whether an iterative approach can improve the accuracy of block-sparse models, thereby narrowing the trade-off between performance and computational efficiency. Using a Wide Residual Network and the CIFAR-100 dataset, we evaluate global block pruning under various configurations: one-shot, iterative, and gradual pruning schedules, as well as a method called AC/DC, combined with different block importance metrics. Our results show that iterative and gradual block pruning significantly outperform one-shot approaches at high sparsity levels such as 90%. Overall, our findings demonstrate that block pruning, when combined with carefully designed iterative strategies, can preserve high model accuracy while inducing sparsity patterns more amenable to hardware acceleration - thereby bridging the gap between theoretical efficiency and practical usability. |
Súbory diplomovej práce:
| kovacikova_diploma_thesis.pdf |
Súbory prezentácie na obhajobe: