| Abstrakt: | Deep neural networks are currently one of the most powerful machine learning methods. Unlike shallow neural networks, their depth allows them to model the input data at various levels of abstraction. They find application in diverse fields, such as self-driving cars, medical diagnostics, and machine translation. However, such networks can be very large and training them can take several weeks, depending on the hardware.
In our thesis, we explored various novel ways of improving learning in deep neural networks, without increasing the computational cost of doing so. We propose two novel types of layers, called shifting and scaling layers, which are essentially already present in most neural network architectures, but not at the right places. We hypothesised that putting them in a different position would enable the network to learn faster, and we evaluated this in a series of experiments. The results of our experiments suggest that our shifting and scaling layers innovations can indeed improve the pace of learning,
compared to networks that do not utilize them.
|
|---|