RBM - linear to binary layer

조회 수: 5 (최근 30일)
minomic
minomic 2015년 5월 5일
편집: minomic 2015년 5월 5일
I am working in Matlab and developing a project which is focused on deep learning. Since I have to apply this technique to images, I thought that the usual binary-binary units are not appropriate since the images are gray scale and not binarized. So I tried to follow the code prof. Hinton provided on his website to write my implementation of a linear-to-binary rbm. In my idea this should be the first layer of the autoencoder: the following ones can be binary-to-binary.
So these are some lines of the code I wrote (the variables' names are quite self-explicative):
poshidprobs = sigmoid(data*vishid + repmat(hidbiases, numcases, 1));
batchposhidprobs(:,:,batch)=poshidprobs;
posprods = data' * poshidprobs; % a sort of correlation
poshidact = sum(poshidprobs);
posvisact = sum(data);
%%%%%%%%%END OF POSITIVE PHASE %%%%%%%%%%%%%%%%
poshidstates = poshidprobs > rand(numcases,numhid);
%%%%%%%%%START NEGATIVE PHASE %%%%%%%%%%%%%%%%%
% linear reconstruction
negdata = poshidstates*vishid' + repmat(visbiases,numcases,1);
% hidden units update
neghidprobs = sigmoid(negdata*vishid + repmat(hidbiases,numcases,1));
negprods = negdata'*neghidprobs;
neghidact = sum(neghidprobs);
negvisact = sum(negdata);
%%%%%%%%%END OF NEGATIVE PHASE %%%%%%%%%%%%%%%%%
err= sum(sum( (data-negdata).^2 ));
errsum = err + errsum;
%%%%%%%%%UPDATE WEIGHTS AND BIASES %%%%%%%
vishidinc = momentum*vishidinc + ...
epsilonw*( (posprods-negprods)/numcases - weightcost*vishid);
visbiasinc = momentum*visbiasinc + (epsilonvb/numcases)*(posvisact-negvisact);
hidbiasinc = momentum*hidbiasinc + (epsilonhb/numcases)*(poshidact-neghidact);
vishid = vishid + vishidinc;
visbiases = visbiases + visbiasinc;
hidbiases = hidbiases + hidbiasinc;
Does it make sense to you? Can you spot any mistake? The reason why I am asking is because the reconstruction error stays very high, like 500000 or so. So I was asking myself (and the community) whether this is normal or it indicates some problems and mistakes.

답변 (0개)

카테고리

Help CenterFile Exchange에서 Mathematics and Optimization에 대해 자세히 알아보기

태그

제품

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by