| Meno: | Júlia |
|---|---|
| Priezvisko: | Lichmanová |
| Názov: | Improving Structured Pruning of Deep Neural Networks |
| Vedúci: | Mgr. Vladimír Boľa, PhD. |
| Rok: | 2025 |
| Kµúčové slová: | deep neural networks, structured pruning, Monarch decomposition, low-rank approximations, spectral clustering, ILP, permutation algorithms, model compression |
| Abstrakt: | To make large neural networks more efficient, this thesis focuses on structured compression using Monarch decomposition, a method for approximating weight matrices. To further improve Monarch efficiency, we propose to permute rows and columns of the weight matrices. We introduce two permutation algorithms, one based on Integer Linear Programming (ILP) and another using Spectral KNN, both designed to improve the accuracy of the decomposition. Our results show that Spectral KNN permutation algorithm offers a great trade-off between speed and accuracy, while running in a fraction of the time against ILP permutation algorithm. We also show that incorporating input activation statistics into both methods improves reconstruction quality and overall model accuracy. All code is available at https://github.com/Lima239/master-thesis-pruning-dnn |
Súbory diplomovej práce:
| master-thesis-pruning-dnn.zip |
| master_thesis_lichmanova.pdf |
Súbory prezentácie na obhajobe: