RBM - linear to binary layer
조회 수: 5 (최근 30일)
이전 댓글 표시
I am working in Matlab and developing a project which is focused on deep learning. Since I have to apply this technique to images, I thought that the usual binary-binary units are not appropriate since the images are gray scale and not binarized. So I tried to follow the code prof. Hinton provided on his website to write my implementation of a linear-to-binary rbm. In my idea this should be the first layer of the autoencoder: the following ones can be binary-to-binary.
So these are some lines of the code I wrote (the variables' names are quite self-explicative):
poshidprobs = sigmoid(data*vishid + repmat(hidbiases, numcases, 1));
batchposhidprobs(:,:,batch)=poshidprobs;
posprods = data' * poshidprobs; % a sort of correlation
poshidact = sum(poshidprobs);
posvisact = sum(data);
%%%%%%%%%END OF POSITIVE PHASE %%%%%%%%%%%%%%%%
poshidstates = poshidprobs > rand(numcases,numhid);
%%%%%%%%%START NEGATIVE PHASE %%%%%%%%%%%%%%%%%
% linear reconstruction
negdata = poshidstates*vishid' + repmat(visbiases,numcases,1);
% hidden units update
neghidprobs = sigmoid(negdata*vishid + repmat(hidbiases,numcases,1));
negprods = negdata'*neghidprobs;
neghidact = sum(neghidprobs);
negvisact = sum(negdata);
%%%%%%%%%END OF NEGATIVE PHASE %%%%%%%%%%%%%%%%%
err= sum(sum( (data-negdata).^2 ));
errsum = err + errsum;
%%%%%%%%%UPDATE WEIGHTS AND BIASES %%%%%%%
vishidinc = momentum*vishidinc + ...
epsilonw*( (posprods-negprods)/numcases - weightcost*vishid);
visbiasinc = momentum*visbiasinc + (epsilonvb/numcases)*(posvisact-negvisact);
hidbiasinc = momentum*hidbiasinc + (epsilonhb/numcases)*(poshidact-neghidact);
vishid = vishid + vishidinc;
visbiases = visbiases + visbiasinc;
hidbiases = hidbiases + hidbiasinc;
Does it make sense to you? Can you spot any mistake? The reason why I am asking is because the reconstruction error stays very high, like 500000 or so. So I was asking myself (and the community) whether this is normal or it indicates some problems and mistakes.
댓글 수: 0
답변 (0개)
참고 항목
카테고리
Help Center 및 File Exchange에서 Mathematics and Optimization에 대해 자세히 알아보기
제품
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!