Bayesian Weight Decay on Bounded Approximation for Deep Convolutional Neural Networks

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 16
  • Download : 0
This paper determines the weight decay parameter value of a deep convolutional neural network (CNN) that yields a good generalization. To obtain such a CNN in practice, numerical trials with different weight decay values are needed. However, the larger the CNN architecture is, the higher is the computational cost of the trials. To address this problem, this paper formulates an analytical solution for the decay parameter through a proposed objective function in conjunction with Bayesian probability distributions. For computational efficiency, a novel method to approximate this solution is suggested. This method uses a small amount of information in the Hessian matrix. Theoretically, the approximate solution is guaranteed by a provable bound and is obtained by a proposed algorithm, where its time complexity is linear in terms of both the depth and width of the CNN. The hound provides a consistent result for the proposed learning scheme. By reducing the computational cast of determining the decay value, the approximation allows for the fast investigation of a deep CNN (DCNN) which yields a small generalization error. Experimental results show that our assumption verified with different DCNNs is suitable for real-world image data sets. In addition, the proposed method significantly reduces the time cost of learning with setting the weight decay parameter while achieving good classification performances.
Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Issue Date
2019-09
Language
English
Article Type
Article
Citation

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, v.30, no.9, pp.2866 - 2875

ISSN
2162-237X
DOI
10.1109/TNNLS.2018.2886995
URI
http://hdl.handle.net/10203/267482
Appears in Collection
CS-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0