How to "prune" a one hidden layer NN such that the off diagonal elements of the weights, that is, "net.IW{1}" and "net.LW{2}", are zeros?
조회 수: 2 (최근 30일)
이전 댓글 표시
How can I create a single hidden layered NN such that the off main diagonal elements of the weights, that is, "net.IW{1}" and "net.LW{2}", are zeros?
Please see my partial script below with a question there. Can someone help me with this toy example?
%% Create a dataset with a network
[x,t] = crab_dataset; %load the dataset
x = x(1:2,:); % Input length
net = patternnet(2); %create the network
net = configure(net,x,t);
view(net);
w1 = net.IW{1}; %the input-to-hidden layer weights (but w1
w2 = net.LW{2}; %the hidden-to-output layer weights
b1 = net.b{1}; %the input-to-hidden layer bias
b2 = net.b{2}; %the hidden-to-output layer bias
%% QUESTION (or INTENTION):
% How to setup the net such that the off diagonal elements of w1 or net.IW{1} and w2 or net.LW{2} are ZEROs after training the network?
%% then, I would like to train the network with my defined network!
% [net,tr] = train(net,x,t); %train the network
댓글 수: 0
답변 (2개)
Srivardhan Gadila
2020년 5월 29일
As per my knowledge w.r.t shallow nerual networks you cannot freeze non-diagonal weights & make the diagonal wieghts only to update since the property net.layerWeights{i,j}.learn is defined for the entire connections between layers i and j.
I would suggest you to use Deep Nerual networks instead of shallow nerual networks & define custom Deep Learning layer to achieve your functionality. Refer to Define Custom Deep Learning Layers & Define Custom Deep Learning Layer with Learnable Parameters
Other suggestion w.r.t shallow nerual networks approach: (may or may not be useful)
Set the net.trainParam.epochs to 1 & place the [net,tr] = train(net,x,t); in a for loop iterated over total number of epochs, then after each epoch set the non-diagonal weights to zero.
참고 항목
카테고리
Help Center 및 File Exchange에서 Image Data Workflows에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!