How to detect the specific area from an oscilloscope signal

Hello,

I don't have any experience with MATLAB or coding, therefore I ask you about the general idea and method so that I can solve that easier.

I have two types of patterns, as shown in the attachment, the first type has the unique signal before coming back to the noise level for 0.15-0.2s. While the second one has typically longer time,

Each type has about hundreds files of similar pattern. So how can I create a a program to detect if it is type 1 or type 2 when inserting a random file (.csv format) in 2 groups?

[Mathieu NOE]

hello

you have some data available ?

does the spike always occurs in type 1 or is the quiet zone duration the only way to differenciate the two types ?

[Duc Khanh]

Hi,

I have around 5 files for each, I attached one for each type. I think only type 1 has a very high peak while type 2 does not. Beside the quiet zone duration, I think the signal peak is another way but my boss prefer the quiet zone stuff.

Thanks in advance.

Answer 1

[Star Strider]

Perhaps something like this —

jpgs = dir('*.jpg');
csvs = dir('*.csv');

for k = 1:numel(jpgs)
    figure
    imshow(imread(jpgs(k).name))
end

for k = 1:numel(csvs)
    T{k,:} = readtable(csvs(k).name);
    Trm = rmmissing(T{k});
    [eu,ed] = envelope(Trm{:,2}, 2, 'peak');
    Lv = ed >= -0.2;
    [ts,te]  = bounds(Trm{Lv,1});
    tdur = te-ts;
    thrshld = 0.2;
    Type = 1;
    if tdur > thrshld
        Type = 2;
    end
    figure
    plot(Trm{:,1}, Trm{:,2}, 'DisplayName','Signal')
    hold on
    plot(Trm{:,1}, ed, '-r', 'DisplayName','Lower Envelope')
    plot(Trm{:,1}, Lv*0.5, '-k', 'DisplayName','Decision Logical Vector')
    hold off
    grid
    ylim([-1 1])
    xlabel('Time (s)')
    ylabel('Voltage (V)')
    if Type == 2
        ylabel('B')
    end
    title(["Type = "+Type+", Gap Duration = "+tdur+" s", "Filename: "+csvs(k).name])
    legend('Location','best')
end

You can remove many (if not all) of the plot calls. I kept them in to demonstrate how the code works.

EDIT — Added the file name to the title text.

.

[Duc Khanh]

God, thank you so much Mr. Star Strider. Can I ask you a little bit more? Your code will automatically detect the gap area regardless of its position inside the graph?

[Star Strider]

My pleasure!

It should, and will for signals such as the ones you posted as examples.

Another option is to use the interp1 function —

csvs = dir('*.csv');

for k = 1:numel(csvs)
    T{k,:} = readtable(csvs(k).name);
    Trm = rmmissing(T{k});
    [eu,ed] = envelope(Trm{:,2}, 2, 'peak');
    % Lv = ed >= -0.2;
    lvl = -0.2;
    idx = find(diff(sign(ed-lvl)));
    for k  = 1:numel(idx)
        idxrng = idx(k)-1 : idx(k)+1;
        tv(k) = interp1(ed(idxrng), Trm{idxrng,1}, lvl);
    end
    tdur = diff(tv)
    % [ts,te]  = bounds(Trm{Lv,1});
    % tdur = te-ts;
    thrshld = 0.2;
    Type = 1;
    if tdur > thrshld
        Type = 2;
    end
    figure
    plot(Trm{:,1}, Trm{:,2}, 'DisplayName','Signal')
    hold on
    plot(Trm{:,1}, ed, '-r', 'DisplayName','Lower Envelope')
    plot(tv, ones(size(tv))*lvl, '-sk', 'LineWidth',2.5, 'DisplayName','Decision Points')
    hold off
    grid
    ylim([-1 1])
    xlabel('Time (s)')
    ylabel('Voltage (V)')
    if Type == 2
        ylabel('B')
    end
    title(["Type = "+Type+", Gap Duration = "+tdur+" s", "Filename: "+csvs(k).name])
    legend('Location','best')
end

tdur = 1.7686

tdur = 0.1655

The interp1 funciton give a slightly different result (and may be more accurate) compared to the logical comparison approach. I offer it as alternative. I left the original code assignments in (commented out) to illustrate the differences. You can of course delete them. It sould be straightforward to create this as a funciton, with or without the figures.

That would go something like this —

for k = 1:numel(csvs)
    filename{k,:} = csvs(k).name;
    [Type(k,:),tdur(k,:)] = csvCheck(filename{k});
end

Result = table(filename, Type, tdur)

Result = 2x3 table
       filename        Type     tdur  
    _______________    ____    _______

    {'tek0094.csv'}     2       1.7686
    {'tek0162.csv'}     1      0.16548

function [Type,tdur] = csvCheck(csvfile)
    T1 = readtable(csvfile);
    Trm = rmmissing(T1);
    [eu,ed] = envelope(Trm{:,2}, 2, 'peak');
    lvl = -0.2;
    idx = find(diff(sign(ed-lvl)));
    for k  = 1:numel(idx)
        idxrng = idx(k)-1 : idx(k)+1;
        tv(k) = interp1(ed(idxrng), Trm{idxrng,1}, lvl);
    end
    tdur = diff(tv);
    thrshld = 0.2;
    Type = 1;
    if tdur > thrshld
        Type = 2;
    end
end

.

[Star Strider]

If my Answer helped you solve your problem, please Accept it!

.

[Duc Khanh]

I did it. Thank you very much for your helpful answer.

Best regards!

[Duc Khanh]

I have just tried it. It runs smoothly for the whole folder but why is the file name so strange? And what could I change when the Decision Logical Vector is narrower than it is supposed to be? (Image 2)

EDIT- The file name was fixed by myself.

[Star Strider]

As always, my pleasure!

Thank you!

figure
imshow(imread('Mr. Starstrider question 1.png'))

figure
imshow(imread('Mr. Starstrider question 2.png'))

I suggest that you use the second version in my later Comment rather than the first version. It is more robust, and likely more accurate. Also, while I chose the decision value (the ‘lvl’ variable) to be -0.2, you can define it to be whatever you want, so long as it is greater than the lowest value of the envelope (red curve) in the ‘gap’ region.

Please attach the files for the two plot images. I can then see what the problem might be.

.

[Duc Khanh]

I already tried both method and it seems like they gave similar answers, Mr. Starstrider.

As you can see, the decision vector/decision points are narrower than the purple line (which is more accurate to us).

I attached the image with the first method to the right and the second one (interp 1) to the left, the csv file was also attached.

Thank you very much.

[Star Strider]

The problem with that file is to determine what you want to measure.

I changed the ‘tdur’ assignment to:

[tdur,ix] = min(tv(end) - tv(1:end-1)); % Change This As Necessary To Provide The Result You Want

This measures the shortest distance between the end of ‘tv’ and the earlier elements. Change that if you want to measure a different distance.

For example to measure the maximum distance:

[tdur,ix] = max(tv(end) - tv(1:end-1)); % Change This As Necessary To Provide The Result You Want

As it is here, that should not change the distance measures of the previous files.

I included all the files to check, using the min version of ‘tdur’. The plots also adapt to choosing either the min or max values. (I ran both to check that.)

I also found a bug that I should have caught earlier. (I do not know the reason it did not cause problems originally.) I used ‘k’ in the file loop counter and also in the interrpolation loop, and that caused problems, since the interpolation loop changes the counter value in the end. (It should not have caused problems in the original version without the interpolation loop.) I solved that by creating ‘k1’ for the file loop and ‘k2’ for the interpolation loop. It now works correctly for all the files.

The ‘csvCheck’ function uses the same logic, so change min to max in it as well if you change it in my oiriginal code. (It does everything my original code does except the plots, and produces a ‘Result’ table instead.)

These changes work for all the provided files —

figure
imshow(imread('Mr. Starstride...omparison.png'))

csvs = dir('*.csv');

for k1 = 1:numel(csvs)
    T{k1,:} = readtable(csvs(k1).name);
    Trm = rmmissing(T{k1});
    [eu,ed] = envelope(Trm{:,2}, 2, 'peak');
    % Lv = ed >= -0.2;
    lvl = -0.2;
    idx = find(diff(sign(ed-lvl)));
    for k2  = 1:numel(idx)
        idxrng = idx(k2)-1 : idx(k2)+1;
        tv(k2) = interp1(ed(idxrng), Trm{idxrng,1}, lvl);
    end
    [tdur,ix] = min(tv(end) - tv(1:end-1));                 % Change This As Necessary To Provide The Result You Want
    % [ts,te]  = bounds(Trm{Lv,1});
    % tdur = te-ts;
    thrshld = 0.2;
    Type = 1;
    if tdur > thrshld
        Type = 2;
    end
    figure
    plot(Trm{:,1}, Trm{:,2}, 'DisplayName','Signal')
    hold on
    plot(Trm{:,1}, ed, '-r', 'DisplayName','Lower Envelope')
    plot(tv([ix end]), ones(size(tv([ix end])))*lvl, '-sk', 'LineWidth',2.5, 'DisplayName','Decision Points')
    hold off
    grid
    ylim([-1 1]*2)
    xlabel('Time (s)')
    ylabel('Voltage (V)')
    if Type == 2
        ylabel('B')
    end
    title(["Type = "+Type+", Gap Duration = "+tdur+" s", "Filename: "+csvs(k1).name])
    legend('Location','best')
end

for k1 = 1:numel(csvs)
    filename{k1,:} = csvs(k1).name;
    [Type(k1,:),tdur(k1,:)] = csvCheck(filename{k1});
end

Result = table(filename, Type, tdur)

Result = 3x3 table
       filename        Type      tdur  
    _______________    ____    ________

    {'tek0094.csv'}     2        1.7686
    {'tek0162.csv'}     1       0.16548
    {'tek0166.csv'}     1      0.083154

function [Type,tdur] = csvCheck(csvfile)
    T1 = readtable(csvfile);
    Trm = rmmissing(T1);
    [eu,ed] = envelope(Trm{:,2}, 2, 'peak');
    lvl = -0.2;
    idx = find(diff(sign(ed-lvl)));
    for k  = 1:numel(idx)
        idxrng = idx(k)-1 : idx(k)+1;
        tv(k) = interp1(ed(idxrng), Trm{idxrng,1}, lvl);
    end
    [tdur,ix] = min(tv(end) - tv(1:end-1));                 % Change This As Necessary To Provide The Result You Want
    thrshld = 0.2;
    Type = 1;
    if tdur > thrshld
        Type = 2;
    end
end

.

[Duc Khanh]

Thank you Mr. Star Strider, I managed to modify your code for specific files, it was quite hard at first because I know nothing about coding but your explanation helps me a lot.

From what I learnt, for example, if I want to detect anything lower than 0.5, I will write lvl = -0.5. And tv is the distance from that signal starts until it return back to the normal noise, isnt it?

[Star Strider]

As always, my pleasure!

Thank you!

I will always help to change my code if you want to change it. Please just ask.

The ‘lvl’ value is the level where the lower signal envelope (the red line in my code) first crosses that value until it again crosses it on the other side of the gap. So it has to be some value greater than the lowest value of the envelope. (I arbitrarily chose -0.2 because it worked for alll the files.)

You can get the minimum value of the envelope with:

ed_min = min(ed(1:20));

calculating the miniumum of the first 20 values of the envelope, for example. You might need to do the same thing on the other end as well, if the envelope amplitudes are not the the same before and after the gap (as with tek0094.csv):

ed_min(1) = min(ed(1:20));

ed_min(2) = min(ed(end-19:end));

That will return a vector of the two values. Then choose ‘lvl’ to be the value you want. In that instance, it has to be greater than the highest value of ‘ed_min’, that being ‘ed_min(2)’ or equivalently ‘max(ed_min)’.

Change the index vectors as necessary to produce the result you want.

‘And tv is the distance from that signal starts until it return back to the normal noise, isnt it?’

The ‘tv’ vector are all the values where the lower envelope ‘ed’ crosses the ‘lvl’ value. The ‘tdur’ value is the time between the minimum (or maximum) values of ‘tv’ (depending on how you want to define it) if there are more than two crossings, as with the tek0166.csv file. If there are only two crossings, ‘tdur’ is simply the time difference between them.

.