This is one of those question that are almost impossible to answer, because your goal is so fuzzy, too unclear to write code to solve. You want a set of 6 disctint values from the set, randomly chosen, such that they are not too close to the mean, yet are not too far away. And worse, the real problem that you have is not the one you show, and is of course much larger.
The problem is that only you know what you mean by those fuzzy, loose descriptions.
So what you need to do is to quantify how you would know IF a set of numbers was poorly chosen. What does that mean to YOU?
Once you can do that, then there is a simple algorithm you can use.
Use randperm to choose a set of 6 random indices into that matrix. Then apply your measure to the set of values as chosen. Are thay adequate? If so, then you are done. If not, then choose another set of 6 random indices. Repeat until you are happy, or better yet, until your measure of gooodness is satisfied. But remember, only you can write that function that identifies if some set is good, because only you knows what you mean by not too close, not too far.
You may wonder how randperm can choose 6 sets of subscripts. But in reality, you have just 24 numbers there to choose from. For example, if we use randperm to choose 6 numbers from the integers 1:24, without replacement, we might get this:
set = randperm(24,6)
21 9 15 19 13 24
MATLAB can use this single indices to index into a 2-dimensional matrix, as such...
matrix = [0.495172050518114 0.499673696800097 0.504165848780560 0.508647518680314
0.498924748106845 0.503460510444308 0.507986706526993 0.512502341089754
0.502655617713492 0.507225297666612 0.511785339830220 0.516334741496754
0.506363747185978 0.510967138022487 0.515560819970764 0.520143782926196
0.510048224444374 0.514685111140220 0.519312218302613 0.523928528477073
0.513708137847587 0.518378297090918 0.523038606626558 0.527688041699157];
0.51633 0.50723 0.51179 0.50865 0.50417 0.52769
Or, you can convert them into a list of row and column indices.
[rowind,colind] = ind2sub([6,3],set)
3 3 3 1 1 6
4 2 3 4 3 4
In the end though, you will be the one who needs to write the algorithm to define if the set of numbers as chosen was any good, in your opinion.