create image from timeseries data

Hi,

I have time stamp (in microseconds) of the image data stored in array AA and arrays x,y and I contains x,y co-ordinats and intensity. I need to accumulate the data over 200 microseconds to create each image.

x pixel size - 1280;

y pixel size - 720;

[DGM]

The only thing in the mat file is AA.

[Turbulence Analysis]

I couldn't attached it because of the big size. Now, limited the time and attached the same here.

Answer 1

[Mathieu NOE]

hum , maybe it's because image processing is not my everyday activity but unless I have misunderstood the project you want to create images of size 1280 * 720 (= 921600 pixels) but only specifying the intensity for 200 of them for each image.

so the remaining portion of the image is what ? I decided to init each image with zero valued array , but I may be completely off topic

this is what I can offer, if that makes sense to you

load('matlab1.mat')

data = cd_data.AA; 
x = cd_data.x; 
y = cd_data.y; 
intensity = cd_data.I; 
samples = numel(data);

x = x+1 ; %  x range must be changed from (0:1279) to (1:1280)
y = y+1 ; %  x range must be changed from (0:719) to (1:720)

dt = 1; % sample rate (micro s)
t = dt*(0:samples-1);  %  time vector 


%% home made solution (you choose the amount of overlap)
buffer_size = 200; % how many samples 
overlap = 0; % overlap expressed in samples

shift = buffer_size-overlap;    % nb of samples between 2 contiguous buffers  
nb_of_loops = fix((samples-buffer_size)/shift +1);
im_empty = zeros(1280,720);

    for k=1:nb_of_loops
        start_index = 1+(k-1)*shift;
        stop_index = min(start_index+ buffer_size-1,samples);
        xx = x(start_index:stop_index); 
        yy = y(start_index:stop_index); 
        Intens = intensity(start_index:stop_index);
        
        im = im_empty; % init im with zeros
        % now use the data
        for m = 1:buffer_size
            im(xx(m),yy(m),k) = Intens(m);
        end
        % display (optionnal)
        imshow(im(:,:,k))
        
    end

[Turbulence Analysis]

Thanks!

Actually the resultant size for all the imags are still 1280 * 780. All I want to do is accumulate the events from 200 microseconds to generate one image.

For e.g. array AA contains time stamps, here, accumulate 844304 microseconds to 844504 to generate one image and next image from 844505 - 844705 and so on...

[Mathieu NOE]

it's exactly what I have done

[Turbulence Analysis]

But unfortunately, from the first accumulation i.e. (844304 - 844504) I have just got the blank image as attached.

[Mathieu NOE]

ok , first I was not aware that on your side the time index start at 844304 , on my side it will be 1

so running my code only for the first iteration will generate this image (attached also)

    for k=1:1%nb_of_loops
        start_index = 1+(k-1)*shift;
        stop_index = min(start_index+ buffer_size-1,samples);
        xx = x(start_index:stop_index); 
        yy = y(start_index:stop_index); 
        Intens = intensity(start_index:stop_index);
        
        im = im_empty; % init im with zeros
        % now use the data
        for m = 1:buffer_size
            im(xx(m),yy(m),k) = Intens(m);
        end
        % display (optionnal)
        imshow(im(:,:,k))
        
    end

it's not completely blank , the non zero elements are exactly 200 in this first iteration (buffer), but only 200 for 921600 pixels that is only 0.0217 % of the entire image

[Turbulence Analysis]

I just started playing around with the code which was originally created (long ago) by the supplier of my device. With this I am getting the desired image (also attached here), unfortunately, for one accumulation itselft it takes much longer time.

Attached is the data for just one accumulation (i.e. 844304 - 844504)

Here cd_img is my final output which contains the image

clear mov frame_idx last_frame_ts cd_img
width = 1280;
height = 720;
acc_time = 200;
framerate = 10000;
frame_time = 100 % in microseconds
% Initialize state storage
cd_img = 0.5*ones(height, width);
ts_img = zeros(height, width);
last_frame_ts = 0;
frame_idx = 1;

%% Build a figure for movie display and store the first empty frame
figure();
image(cd_img);
colormap jet;
colormap_size = size(colormap, 1);
frame_idx = frame_idx+1;

%%

for i=1:length(cd_data.ts)
    % Get timestamp of current event
    cur_ts = cd_data.ts(i);
    % Check if images should be generated
    while (cur_ts > last_frame_ts + frame_time)
        % Update frame by removing old events
        last_frame_ts = last_frame_ts + frame_time;
        cd_img(ts_img < last_frame_ts - acc_time) = 0.5;
        % scale and display image
        ax = gca;
        image(colormap_size*cd_img);
                set(gca,'TickLabelInterpreter','latex')
    set(gca, 'FontSize', 20);
    set(gca, 'fontweight', 'bold');
    xlabel('x(pixel)','fontweight','bold','FontSize',20,'Interpreter','latex');
    ylabel('y(pixel)','fontweight','bold','FontSize',20,'Interpreter','latex');
        %mov(frame_idx) = getframe();

        %writeVideo(mov(frame_idx))
        frame_idx = frame_idx+1;
    end
    % Add event to state
    if cd_data.p(i) == 1
        % Put a white dot for ON events
        cd_img(cd_data.y(i), cd_data.x(i)) = 1;
    else
        % Put a black dot for OFF events
        cd_img(cd_data.y(i), cd_data.x(i)) = 0;
    end
    ts_img(cd_data.y(i), cd_data.x(i)) = cur_ts;
%         thisBaseFileName = sprintf('SS%4.5d.bmp', i); % Base output file name.
%     saveas(gcf,thisBaseFileName)
%clf
% i
end

Answer 2

[Turbulence Analysis]

Actually your code sums up first 200 rows, but not 200 micro seconds that's why intensity appeared only in the small portion of the image. What I looking for is summing up data correponds to 200 microseconds, which means in the recently attached data set this corresponds to row 1 - row 6105 ( i.e. 844504 - 844304 = 200 microseconds).

[Mathieu NOE]

ok, I was missing this information (or I didn't read carefully somehow) , so instaed of taking buffers of 200 samples , you simply need to take now 6105 samples (no code modification need)

and yes the result looks now like a flow animation !!

% I have time stamp (in microseconds) of the image data stored in array AA and arrays x,y and I contains x,y co-ordinats and intensity. I need to accumulate the data over 200 microseconds to create each image. 
% x pixel size - 1280;
% y pixel size - 720;

% NB : 200 microseconds = 6105 samples 

load('matlab1.mat')

data = cd_data.AA; 
x = cd_data.x; 
y = cd_data.y; 
intensity = cd_data.I; 
samples = numel(data);

x = x+1 ; %  x range must be changed from (0:1279) to (1:1280)
y = y+1 ; %  x range must be changed from (0:719) to (1:720)

dt = 1; % sample rate (micro s)
t = dt*(0:samples-1);  %  time vector 

buffer_size = 6105; % how many samples 
overlap = 0; % overlap expressed in samples

shift = buffer_size-overlap;    % nb of samples between 2 contiguous buffers  
nb_of_loops = fix((samples-buffer_size)/shift +1);
im_empty = zeros(1280,720);

    for k=1:nb_of_loops
        start_index = 1+(k-1)*shift;
        stop_index = min(start_index+ buffer_size-1,samples);
        xx = x(start_index:stop_index); 
        yy = y(start_index:stop_index); 
        Intens = intensity(start_index:stop_index);
        
        im = im_empty; % init im with zeros
        % now use the data
        for m = 1:buffer_size
            im(xx(m),yy(m),k) = Intens(m);
        end
        % display (optionnal)
        imshow(im(:,:,k))
        
    end

[Turbulence Analysis]

Hi,

Just a quick follow up, hiow to identify the buffer size, as this keeps changing.

For instance, in the attached mat file, for the first 200 micro seconds the buffer size is 4513 (i.e. rows 1 to 4513 --> 69142-68942 = 200) and for next 200 micro seconds (i.e. 69143 to 69343) is reprsented by rows 4514 - 8836, the buffer size here is 4322

[Mathieu NOE]

we coud certainly make the code more robust but we need the time stamps values

I don't see any new mat file attached

[Turbulence Analysis]

Sorry, the mat file is attached here

first 200 micro seconds the buffer size is 4533 (i.e. rows 1 to 4533 --> 69142-68942 = 200) and for next 200 micro seconds (i.e. 69143 to 69343) is reprsented by rows 4534 to 8829, the buffer size here is 4295

[Mathieu NOE]

hello again

try this :

% x pixel size : 1280;
% y pixel size : 720;
% first 200 micro seconds the buffer size is 4533 (i.e. rows 1 to 4533 --> 69142-68942 = 200) 
% and for next 200 micro seconds (i.e. 69143 to 69343) is reprsented by rows 4534 to 8829, the buffer size here is 4295

load('matlab.mat')
x = cd_data.x; 
y = cd_data.y; 
intensity = cd_data.p; 
t = cd_data.ts;  % time vector
samples = numel(t);

buffer_duration = 200; % microseconds
nb_of_loops = fix((t(end) - t(1))/buffer_duration);

start_index(1) = 1; % first 200 microsecond buffer starts right at the 1st sample
for k=1:nb_of_loops
    stop_index(k) = find((t - t(1))==(k)*buffer_duration,1,'last');
end
start_index(2:nb_of_loops) = stop_index(1:nb_of_loops-1)+1; % second to last value of start_index is simply former stop_index + 1

buffer_size = stop_index - start_index +1;

x = x+1 ; %  x range must be changed from (0:1279) to (1:1280)
y = y+1 ; %  x range must be changed from (0:719) to (1:720)

im = zeros(1280,720,nb_of_loops); % init im with zeros

    for k=1:nb_of_loops
        xx = x(start_index(k):stop_index(k)); 
        yy = y(start_index(k):stop_index(k)); 
        Intens = intensity(start_index(k):stop_index(k));
        % now use the data
        for m = 1:buffer_size(k)
            im(xx(m),yy(m),k) = Intens(m);
        end
        % display (optionnal)
        imshow(im(:,:,k))
        
    end

[Turbulence Analysis]

Perfect! Thank you very much!

[Turbulence Analysis]

Hi Again,

Sorry for the trouble.

While implementing this for entire time series(data here - https://drive.google.com/file/d/1sGwoTBu3WVqOXCPIAyh0lNskyT8joCZS/view?usp=sharing), unfortunately stopindex in the code throws below error after row no. 2877499

Unable to perform assignment because the left and right sides have a different number of elements.

[Mathieu NOE]

hello again

I see the point

there are some times when the time vector has no time stamp corresponding to exactly a 200 microsecond spaced grid so that will throw this error message

what I have done is to take the previous time stamp available , which , I think , is a reasonnable compromise

so this line

stop_index(k) = find((t - t(1))==(k)*buffer_duration,1,'last');

get's replaced by :

stop_index(k) = find((t- t(1) - k*buffer_duration)<=0,1,'last');

just FYI, as the number of images is now quite high (9868) , I had a memory issue due to the initialization of im

im = zeros(1280,720,nb_of_loops); % init im with zeros

Error using zeros

Requested 1280x720x9868 (67.8GB) array exceeds maximum array size preference.

Creation of arrays greater than this limit may take a long time and cause MATLAB to

become unresponsive.

maybe you have not the same memory limitations on your PC, or somehow we have to find away to avoid building huge arrays , either by splitting the data file in smaller chunks , or generate only one picture at a time an build an animation incrementaly

full code :

% x pixel size : 1280;
% y pixel size : 720;
% first 200 micro seconds the buffer size is 4533 (i.e. rows 1 to 4533 --> 69142-68942 = 200) 
% and for next 200 micro seconds (i.e. 69143 to 69343) is reprsented by rows 4534 to 8829, the buffer size here is 4295

load('matlab.mat')
x = cd_data.x; 
y = cd_data.y; 
intensity = cd_data.p; 
t = cd_data.ts;  % time vector
samples = numel(t);

buffer_duration = 200; % microseconds
nb_of_loops = fix((t(end)-t(1))/buffer_duration);

start_index(1) = 1; % first 200 microsecond buffer starts right at the 1st sample
for k=1:nb_of_loops
    stop_index(k) = find((t- t(1) - k*buffer_duration)<=0,1,'last');
end
start_index(2:nb_of_loops) = stop_index(1:nb_of_loops-1)+1; % second to last value of start_index is simply former stop_index + 1

buffer_size = stop_index - start_index +1;

x = x+1 ; %  x range must be changed from (0:1279) to (1:1280)
y = y+1 ; %  x range must be changed from (0:719) to (1:720)

im = zeros(1280,720,nb_of_loops); % init im with zeros

    for k=1:nb_of_loops
        xx = x(start_index(k):stop_index(k)); 
        yy = y(start_index(k):stop_index(k)); 
        Intens = intensity(start_index(k):stop_index(k));
        % now use the data
        for m = 1:buffer_size(k)
            im(xx(m),yy(m),k) = Intens(m);
        end
        % display (optionnal)
        imshow(im(:,:,k))
        
    end

[Turbulence Analysis]

Thanks very much again!

I do have the limiation on my PC. Is there any way to get rid of this?

[Mathieu NOE]

I tried to reduce the amount of memory use , so first thing I tried is to change from double precision to integer to store the im array.

here I opted for logical class but I could have done with uint8 or int8 for the same result

try this new code :

% x pixel size : 1280;
% y pixel size : 720;
% first 200 micro seconds the buffer size is 4533 (i.e. rows 1 to 4533 --> 69142-68942 = 200) 
% and for next 200 micro seconds (i.e. 69143 to 69343) is reprsented by rows 4534 to 8829, the buffer size here is 4295

load('matlab.mat')
x = cd_data.x; 
y = cd_data.y; 
intensity = cd_data.p; 
t = cd_data.ts;  % time vector
samples = numel(t);

buffer_duration = 200; % microseconds
nb_of_loops = fix((t(end)-t(1))/buffer_duration);

start_index(1) = 1; % first 200 microsecond buffer starts right at the 1st sample
for k=1:nb_of_loops
    stop_index(k) = find((t- t(1) - k*buffer_duration)<=0,1,'last');
end
start_index(2:nb_of_loops) = stop_index(1:nb_of_loops-1)+1; % second to last value of start_index is simply former stop_index + 1

buffer_size = stop_index - start_index +1;

x = x+1 ; %  x range must be changed from (0:1279) to (1:1280)
y = y+1 ; %  x range must be changed from (0:719) to (1:720)

im = zeros(1280,720,nb_of_loops,'logical'); % init im with logical zeros

    for k=1:nb_of_loops
        xx = x(start_index(k):stop_index(k)); 
        yy = y(start_index(k):stop_index(k)); 
        Intens = intensity(start_index(k):stop_index(k)); % range : -1/+1
        Intens = logical((Intens+1)*0.5);  % change range to 0/+1 and converted to logical
        % now use the data
        for m = 1:buffer_size(k)
            im(xx(m),yy(m),k) = Intens(m);
        end
        % display (optionnal)
        imshow(im(:,:,k))
        
    end

[Turbulence Analysis]

Thanks very much! It works perfectly.

[Turbulence Analysis]

Sorry for the trouble again!

Just a one issue, I need to differentiate (-1, 1) on the image (see attached image 2), however, the image generated with the present code i.e. logical (see image 1) that's not the case.

[Mathieu NOE]

hello again

ok, here is it :

I changed from logical to int8 class (same memory usage) , so you have both levels displayed

% x pixel size : 1280;
% y pixel size : 720;
% first 200 micro seconds the buffer size is 4533 (i.e. rows 1 to 4533 --> 69142-68942 = 200) 
% and for next 200 micro seconds (i.e. 69143 to 69343) is reprsented by rows 4534 to 8829, the buffer size here is 4295

load('matlab.mat')
x = cd_data.x; 
y = cd_data.y; 
intensity = int8(cd_data.p); 
t = cd_data.ts;  % time vector
samples = numel(t);

buffer_duration = 200; % microseconds
nb_of_loops = fix((t(end)-t(1))/buffer_duration);

start_index(1) = 1; % first 200 microsecond buffer starts right at the 1st sample
for k=1:nb_of_loops
    stop_index(k) = find((t- t(1) - k*buffer_duration)<=0,1,'last');
end
start_index(2:nb_of_loops) = stop_index(1:nb_of_loops-1)+1; % second to last value of start_index is simply former stop_index + 1

buffer_size = stop_index - start_index +1;

x = x+1 ; %  x range must be changed from (0:1279) to (1:1280)
y = y+1 ; %  x range must be changed from (0:719) to (1:720)

im = zeros(1280,720,nb_of_loops,'int8'); % init im with integer zeros

    for k=1:nb_of_loops
        xx = x(start_index(k):stop_index(k)); 
        yy = y(start_index(k):stop_index(k)); 
        Intens = intensity(start_index(k):stop_index(k)); % range : -1/+1
        % now use the data
        for m = 1:buffer_size(k)
            im(xx(m),yy(m),k) = Intens(m);
        end
        % display (optionnal)
        imagesc(im(:,:,k))
        pause(0.01)
        
    end

[Turbulence Analysis]

Thank you very much for your kind support!

[Mathieu NOE]

as always, my pleasure !