Problem with serial port communication

Hi,

I'm interfacing ALtera MAX10 fpga development kit with Matlab and trying to acquire data continuously. I can send command to the fpga board through Matlab regarding how many samples of data I want to capture (Also I'm increasing the buffer size according to the number of samples to be captured). Till 30000 samples I am getting the data correctly. Whenever I'm increasing the sample count to 50000, Matlab is throwing an error that the buffer size cannot be increased to this amount. I want to acquire data continuously say for 5 minutes ( corresponding to 300,000 samples ). Can anyone please kindly help me resolve this issue ? Note that baud rate is set to 115200 bps.

Thanks, Swarnava Pramanik

Answer 1

[Walter Roberson]

Use a BytesAvailableFcn callback to save the data from time to time.

[Swarnava Pramanik]

Hi Walter,

Thanks for your reply. Could you please kindly be more elaborate on this ? I'm using serialObj.BytesAvailable to check if there are bytes in the buffer or not, if there is data in the buffer keeps on reading. I'm also using flushinput(serialObj) to empty the buffer.

Thanks,

Swarnava Pramanik

[Walter Roberson]

function get_data_function = get_some_data(s, num_to_get)

    chunk = 10000;         %at a time

    %shared variables must be initialized
    data_buffer = cell(1, ceil(num_to_get / chunk));
    num_read_so_far = 0;
    buffer_idx = 0;
    num_needed = num_to_get;

    s.BytesAvailableFcnCount = chunk;   %at a time
    s.InputBufferSize = 30000;          %large enough to give time for interrupt to be processed
    s.BytesAvailableFcnMode = 'byte';
    s.BytesAvailableFcn = @store_some_data;

    get_data_function = @return_the_data;

    fopen(s);

   function store_some_data(src, evt)
     %need the below on the same line to be sure there is no interrupt
     num_this_time = src.BytesAvailable; buffer_idx = buffer_idx = 1; data_buffer{buffer_idx} = fread(src, [1 num_this_time]); num_read_so_far = num_read_so_far + num_this_time; if num_read_so_far >= num_needed; fclose(s); end
   end

    function data = return_the_data
      %need the below on the same line to be sure there is no interrupt
      %do not reset num_read_so_far while the stream is live
      %or you would keep reading and keep reading... 
      data_buffer_copy = data_buffer; data_buffer = cell(1, ceil(num_to_get / chunk)); data = horzcat(data_buffer_copy{1:buffer_idx}); buffer_idx = 0; 
    end

   end

To use this, create and configure a serial object but do not fopen() it. Then call get_some_data() passing in the serial object and the number of bytes you want to read. The routine will set up the conditions and start remembering data, and will return a function handle. You can tell when the acquisition is finished because the serial object state will go to closed (you could improve that interface if you wanted.)

Eventually, call the function whose handle was returned by get_some_data(), and it will assemble the data chunks into one continuous array and return it, and reset the buffering. This function handle is written to not assume that all of the data has already been acquired -- it could be simplified if that could be assumed.

The code is intended to be self contained, doing its own buffer management, using shared variables and shared functions. The awkwardness of returning a function handle that fetches the data instead of returning the data itself is there so that you can do other activities while the data is collected. You can call the function handle multiple times during data collection, each time returning whatever has accumulated since the last time it was called; if you get no bytes back then the request count has been reached.

[Swarnava Pramanik]

Hi Walter,

Thanks a lot for your help. But why I'm getting this error

" Warning: The BytesAvailableFcn is being disabled. To enable the callback property either connect to the hardware with FOPEN or set the BytesAvailableFcn property."

I'm adding my code over here.

   if(trigger)
            size_data = 15000;
            s.BytesAvailableFcnCount = size_data;
            s.InputBufferSize = 30000;
            s.BytesAvailableFcnMode = 'byte';
            s.BytesAvailableFcn = {@store_plot_data,size_data};
            fopen(s);
            s.ReadAsyncMode = 'continuous';
            readasync(s);
            fprintf(s, samples_cat);
            pause(2);

        end

        while (1)
            try

                  store_plot_data(s,size_data);

            catch ERR_MSG

                disp('Error Reading Data! Check Unit')

            end

        end
        fclose(s);
        delete(s);

    end
  end

function  store_plot_data(serialObject,chunk)
 A = fscanf(serialObject, '%s', chunk);
if( strcmp(A(end), ',') == 1 )
   data1 = textscan(A, '%f', 'Delimiter', ',');
   data_draw = [data_draw, data1{1}'];
end

Could you please kindly help me resolve this issue ?

Thanks,

[Walter Roberson]

Three parameters are being passed to your callback but your code expects two. The callback fails so it gets disabled.

Do not do that stuff with async mode or looping calling the callback. When you use a bytes available callback it will be called automatically.

[Swarnava Pramanik]

Hi Walter,

Thanks for your response. But I'm not able to follow you as I haven't used callback function before. I want my callback function to accept the serial object and the size_data. Hence I've written my function in that way. So when you are saying that three parameters are being passed to the callback function, I'm getting bit confused over here. Would you mind clarifying this doubt ?

I've modified this :

     store_plot_data(s,evt,size_data);
function store_plot_data(serialObject,~,chunk)

Still I'm getting the same error

Thanks,

[Walter Roberson]

Every callback function (except for a few having to do with constraining resizing ROIs or resizing figures) is automatically passed two parameters. The first of those is the handle to the object that the callback is about; the second of those is a structure with more information. There are a lot of callback varieties for which that structure is empty, and when it is not empty the fields in the structure depend on exactly what kind of callback it is -- but the parameter exists anyhow. When you coded

{@store_plot_data,size_data}

that means that store_plot_data should be called with the first two parameters being the two automatic ones, and as well a third parameter will be passed which is the value of size_data. So your callback declaration should be

function  store_plot_data(serialObject, event, chunk)

[Swarnava Pramanik]

Hi Walter,

Thanks for clearing my doubt but using the function declaration as you have mentioned, it's still throwing the same warning and goes into the catch block,

    store_plot_data(s,evt,size_data);
    function store_plot_data(serialObject,event,chunk)

Thanks,

[Walter Roberson]

Do not do that stuff with async mode or looping calling the callback.

Your callback is failing because it references the undefined variable data_draw . I could do that because I set up nested functions with shared variables, so the variable data_buffer in my code was "in scope" in my store_some_data routine. Notice that in my code, I used the coding structure

   function A
      variable = value;
      function B
        reference to variable
      end
    end

Notice that each "function" has an "end" statement and that function B is completely defined inside function A. Those are conditions to set up nested functions and shared variables. Your code, though, does not use nested functions and has no "end" corresponding to each "function" statement, so data_draw simply does not exist inside the second function at the time you try to append data on to it. (You also do not have any way of getting at the draw_data when you are ready to use it.)

Also:

When you are trying to store a lot of data under a time constraint, when you can you should avoid inefficient operations like textscan, and instead just file the raw data away to be read through later. You get the fastest response time when you do the least "work" in your callback routine.

One of the operations that is really really really inefficient is growing arrays all the time. If you look at my code, I allocated a complete cell array and I store the received chunks in the cell array while the data is being collected. Storing a variable into an existing cell array location is very efficient. I do not put the cell arrays back together into a single array until asked to do so, presumably after all of the data has been collected.

Also:

You are asking to read chunks of data by number of bytes, but then you assume that you can textscan() the chunk with %f floating point format. But because you are reading by number of bytes rather than line by line, you run the strong risk that one of the text-formatted numbers is split between two buffers. You do not handle that case at all, but it is very likely to happen.

For efficiency it would be even better if your device could send you binary formatted numbers rather than text formatted numbers: then you would not have to do any conversion and there would be less data to transfer.

[Swarnava Pramanik]

Hi Walter,

Thanks a lot for this insight. I'll implement the cell array later as of now my priority is to get the whole data. I got passed that warning but now my callback function is getting executed only once and stop executing.

Here is my whole code

 function data_from_serial_port = serial_port_read
data_draw = 0;
oldSerial = instrfind('Port', 'COM4');
if (~isempty(oldSerial))
    delete(oldSerial)
end
baud = 115200;
% Testing whether it reads or not
s = serial('COM4');
set(s,'BaudRate',baud);
  end
samples = '*IDN?';
fopen(s);
fprintf(s, samples);
output = fgetl(s);
fclose(s);
delete(s);
s = serial('COM4');
set(s,'BaudRate',baud);
prompt = 'Enter the number of samples you need to capture: ';
samples = input(prompt,'s');
if(strcmp(samples,'LIA_STATE') == 1)
    set(s,'Timeout',0.001);
    buff_size = 2;
    s.InputBufferSize = buff_size;
    %         s.OutputBufferSize = buff_size;
    fopen(s);
    s.ReadAsyncMode = 'continuous';
    readasync(s);
    fprintf(s, samples);
    mode = fgetl(s);
    pause(1);
    disp('Ignore the above warning..');
    fprintf('LIA Mode is : %c\n', mode)
    fclose(s);
   delete(s);
else
prompt2 = 'Enter "Cap" to start acquiring data: ';
if(strcmp(samples,'') == 1)
        disp('Starting acquiring data with default 512 samples');
elseif(strcmp(samples,'DATA?') == 1
        samples = '512';
end
comma = ' ';
    samples_cat = ['N_Sample',comma,samples];
    decision = lower(input(prompt2,'s'));
if(strcmp(decision,'cap')) == 1
 trigger = 1;
end
if(trigger)
s.BytesAvailableFcn = {@store_plot_data,size_data};
            s.BytesAvailableFcnCount = size_data;
            s.InputBufferSize = 30000;
            s.BytesAvailableFcnMode = 'byte';
            fopen(s);
            fprintf(s, samples_cat);
            pause(2);
end
function store_plot_data(serialObject,event,chunk)
data_draw = 0;
A = fscanf(serialObject, '%s', chunk);
   data1 = textscan(A, '%f', 'Delimiter', ',');
   data_draw = [data_draw, data1{1}'];
if( s.BytesAvailable == 0 )
    data_from_serial_port = data_draw;
end
end
end

[Walter Roberson]

Unfortunately I have some preparations to do now for a meeting; it might be several hours before I can get back to you on this.

[Swarnava Pramanik]

Hi Walter,

That's absolutely fine for me,

In the mean time I'll also try to see if I can get any desired result from my code and will post any update over here.

I really appreciate your help. Thanks again,

Thanks,

[Swarnava Pramanik]

Hi Walter,

I'm able to read the data from the buffer continuously thanks to you but am not able to save the data when I'm calling the serial_port_read function from my script, not sure why. Could you please kindly have a look at my code posted above and let me know where I'm going wrong ?

Thanks,

[Swarnava Pramanik]

Hi Walter,

Could you please kindly help me in another problem. I'm able to continuously get the data from my fpga board and at the same time I'm processing my data. It's happening so that my board is sending suppose 12000 samples but I'm only able to receive 11770 samples (losing around 300 samples corresponding to 0.3 seconds of data ). I'm attaching my code also

    function store_plot_data(serialObject,event,chunk)

        strRecv = fscanf(serialObject, '%s', chunk);

        if( strcmp(strRecv(end), ',') == 1)

            data_draw = [ data_draw dataStringTail strRecv ];
            dataStringTail = [];

        elseif( ( isempty(dataStringTail) == 0 ) && ( strcmp(strRecv(end), ',') == 0))

            lastCommaIndex =  max(strfind(strRecv,','));

            % dataStringTail = strRecv(lastCommaIndex : end);

            data_draw = [data_draw dataStringTail strRecv];
            dataStringTail = [];

        else

            lastCommaIndex =  max(strfind(strRecv,','));

            dataStringTail = strRecv(lastCommaIndex : end);

            data_draw = [data_draw strRecv(1 : lastCommaIndex - 1)];

        end
  end

Please note that "data_draw", "dataStringTail" are global variables.

Thanks,

[Swarnava Pramanik]

Hi Walter,

The callback function to the BytesAvailableFcn occurs only when the number of bytes specified by the BytesAvailableFcnCount is available in the input buffer. Suppose if the input buffer doesn't have the number of bytes specified by the BytesAvailableFcnCount property then the callback function is not executed. This is why I'm loosing some sample. Is there anyway to get around this ?

Thanks,