File Exchange

image thumbnail

Radial Basis Function Neural Networks (with parameter selection using K-means)

version 1.0.0.0 (3.52 KB) by Alireza
RBF Neural Networks (center and distribution of activation functions are selected using K-means)

30 Downloads

Updated 07 Sep 2015

View License

A RBFNN is an artificial neural network that uses radial basis functions as activation functions. Figure shows the structure of the RBFNN. The RBFNN is three layered feed-forward neural network. The first layer is linear and only distributes the input signal, while the next layer is nonlinear and uses Gaussian functions. The third layer linearly combines the Gaussian outputs. Only the tap weights between the hidden layer and the output layer are modified during training.
RBFNN have 5 parameters for optimization:
1- The weights between the hidden layer and the output layer.
2- The activation function.
3- The center of activation functions.
4- The distribution of center of activation functions.
5- The number of hidden neurons.
The weights between the hidden layer and the output layer are calculated by using Moore-Penrose generalized pseudo-inverse. This algorithm overcomes many issues in traditional gradient algorithms such as stopping criterion, learning rate, number of epochs and local minima. Due to its shorter training time and generalization ability, it is suitable for real-time applications.
The radial basis function selected is usually a Gaussian kernel for pattern recognition application.
Generally the center and distribution of activation functions should have characteristic similar to data. Here, the center and width of Gaussians are selected using Kmeans clustering algorithm.
Based on universal approximation theory center and distribution of activation functions are not deterministic if the numbers of hidden neurons being sufficient enough, one can say that the single hidden layer feed-forward network with sufficient number of hidden neurons can approximate any function to any arbitrary level of accuracy.
---------------------------------------------------------------------------------
This code is used in the following papers:
1. A. Asvadi, M. Karami, Y. Baleghi, “Efficient Object Tracking Using Optimized K-means Segmentation and Radial Basis Function Neural Networks,” International Journal of Information and Communication Technology Research (IJICT), vol. 4, no. 1, pp. 29-39, December 2011.
2. A. Asvadi, M. Karami, Y. Baleghi, H. Seyyedi, “Improved Object Tracking Using Radial Basis Function Neural Networks,” in: Proceedings of 7th Iranian Machine Vision and Image Processing (MVIP2011), Tehran, Iran, November 2011.
---------------------------------------------------------------------------------
"mvnrnd" from "Statistics and Machine Learning Toolbox" is used for generating sample data points.

Comments and Ratings (14)

tina nini

How can I use this code "Radial Basis Function neural networks with parameter selection using K-means" for facial recognition ? I want to train the network first with a set of trainnind data then simulate it with a set of test data ? please help
If this code does not work for that, can anyone tell me what neural network with radial basis function can work for my problem? thank you

Dear Mr Alireza ,
how can i use your code "Radial Basis Function Neural Networks with parameter selection using K-means" for regression task ?

Regards

Alan Yu

There is a problem in rbfn_test.m. If the values of output Y are close, then it would produce the same classification(wrong result).
My code to fix the problem is as follows.

function Y = rbfn_test(F, W, K, MU, SIGMA)
%RBFN_TEST Summary of this function goes here
% Detailed explanation goes here

%% Test RBF
N = size(F,1); % n = Number of Pixels
H = zeros(N,K);
for k = 1:K
h = RBFKernel(F,MU,SIGMA,N,k);
H(:,k) = h;
end
Y = H*W; % Output
[M,I] = max(Y,[],2);
IV = ind2vec(I');
Y = IV';

end

Besides, you use the diagonal matrices to represent the kernel widths SIGMA(rbfn_train.m). Is there any theory behind this formula?

Alan Yu

Good work!
Can you give more explanation how to find kernel function of RBF? That is, give more description on RBFKernel.m.

I'm learning Matlab and RBF now. My question is: how can I use this code with data from a csv, consisting of three columns?
Thanks!

Alireza

@pradeep s v it is due to random initialization of the k-means clustering algorithm

pradeep s v

accuracy is varying for every time running without any change

usama mir

sir this code give me an eror in this line [data1,Y1,Xt1]=generate_two_moons(rad,width,dist,(num_samp./2),seed,0)
random_data=[data1,Y1]';random_seq = randperm(num_samp);

it said Error in Untitled (line 15)
[data1,Y1,Xt1]=generate_two_moons(rad,width,dist,(num_samp./2),seed,0)

please solve this issue

Fadya jaw

num_tr = 3000;
num_te = 500;
num_hd = 20;
num_ou = 1;
num_in = 2;
num_hd = 30;
num_samp = num_tr+num_te;
width = 4;
dist = -4;
seed=3000;
rand('seed',seed);
[data1,Y1,Xt1]=half double_moons(rad,width,dist,(num_samp./2),)
random_data=[data1,Y1]';random_seq = randperm(num_samp);random_data = random_data(:,random_seq);
X = random_data(1:2,1:num_tr)';
[indx,centers] = kmeans(X,num_hd);
pos_vector = indx';
c = centers';
%Calculating the variance for each center
for i = 1 : num_hd;

cluster_number=(i);
variance = 0;
counter2=0;
for j = 1 : num_tr;
if cluster_number==pos_vector(j)
sssss = sqrt(sum((random_data(1:2,j)'-c(:,i)').^2))
variance = sssss + variance
counter2=counter2+1;
end
end
sigma(i) = variance/counter2;
end

%Parameters For RLS
w = ones(num_ou,num_hd);
lambda=0.9
P= (lambda)^(-1)*eye(num_hd);
spread = sigma;
IN = random_data(1:2,1:num_tr);
T = random_data(3,1:num_tr);

P= (lambda)^(-1)*eye(num_hd);
w = ones(num_ou,num_hd);
num_train_min=10;
for epoch =1:50;


for m = 1:(num_tr-num_tr./10);

for i = 1:num_hd;
g(i,:) = exp(-(x-c(:,i))'*(x-c(:,i))/(2*spread(i)^2));
end

alpha = d - w*g;
w_diffrence = ((P*g)*alpha' )';
if (norm(w_diffrence) < 0.01) && (m >= num_train_min)

break;
end
w = w + w_diffrence ;
P=P-((P*g/(g'*P*g))*g'*P) ;
end

end

Fadya jaw

%k-mean/rls

num_tr = 3600;
num_te = 400;
num_hd = 20;
num_ou = 1;
num_in = 2;
num_hd = 30;
num_samp = num_tr+num_te;
rad = 10;
width = 6;
dist = -6;
seed=30000000;
rand('seed',seed);
[data1,Y1,Xt1]=generate_two_moons(rad,width,dist,(num_samp./2),seed,0)
random_data=[data1,Y1]';random_seq = randperm(num_samp);random_data = random_data(:,random_seq);
X = random_data(1:2,1:num_tr)';
[indx,centers] = kmeans(X,num_hd);
pos_vector = indx';
c = centers';
%Calculating the variance for each center
for i = 1 : num_hd;

cluster_number=(i);
variance = 0;
counter2=0;
for j = 1 : num_tr;
if cluster_number==pos_vector(j)
sssss = sqrt(sum((random_data(1:2,j)'-c(:,i)').^2))
variance = sssss + variance
counter2=counter2+1;
end
end
sigma(i) = variance/counter2;
end

%Parameters For RLS
w = ones(num_ou,num_hd);
lambda=0.9
P= (lambda)^(-1)*eye(num_hd);
spread = sigma;
IN = random_data(1:2,1:num_tr);
T = random_data(3,1:num_tr);

P= (lambda)^(-1)*eye(num_hd);
w = ones(num_ou,num_hd);
num_train_min=10;
for epoch =1:50;


for m = 1:(num_tr-num_tr./10);

for i = 1:num_hd;
g(i,:) = exp(-(x-c(:,i))'*(x-c(:,i))/(2*spread(i)^2));
end

alpha = d - w*g;
w_diffrence = ((P*g)*alpha' )';
if (norm(w_diffrence) < 0.01) && (m >= num_train_min)

break;
end
w = w + w_diffrence ;
P=P-((P*g/(g'*P*g))*g'*P) ;
end

end

Alireza

Thanks Charles for mentioning that. Yes, mvnrnd is used for generating sample data points for classification.

You may want to mention that your set of functions requires mvnrnd which is part of the Statistics and Machine Learning Toolbox.

Updates

1.0.0.0

dependency

1.0.0.0

description modified

1.0.0.0

title modified

1.0.0.0

dependency

1.0.0.0

description modified

1.0.0.0

File changed

1.0.0.0

tags added

1.0.0.0

description modified

1.0.0.0

Description added

1.0.0.0

Image added

MATLAB Release Compatibility
Created with R2011a
Compatible with any release
Platform Compatibility
Windows macOS Linux