Extract info from .json file by matlab

I need to parse a json file and extract many objects and arrays from that file. Then I am going to plot those extracted information.

I used jsondecode but it only created one big char. Shoud I use that?

Answer 1

[Neeraj Rajpurohit]

fileName = 'filename.json'; % filename in JSON extension
fid = fopen(fileName); % Opening the file
raw = fread(fid,inf); % Reading the contents
str = char(raw'); % Transformation
fclose(fid); % Closing the file
data = jsondecode(str); % Using the jsondecode function to parse JSON from string

[Neeraj Rajpurohit]

Try the above code, hope it helps!

[Stefan Grandl]

fileName = 'filename.json'; % filename in JSON extension
str = fileread(fileName); % dedicated for reading files as text
data = jsondecode(str); % Using the jsondecode function to parse JSON from string

...this version is even more compact. ;-)

--> fileread() instead of fread()

[Vairamani Kanagavel]

tried and working fine for reading .json files

Answer 2

[Navya Seelam]

Hi,

In order to plot the data use str2num to covert the string to numeric data type. If the data in JSON was array of numbers rather than strings then the output of jsondecode would have been array of double.

For more details on conversion of JSON datatypes to MATLAB data types refer this link

Answer 3

[Arvind Gauns]

I have a bunch of JSON files to get infornmation from. Eah file has got three structs within it.

I want to run a batch process to read all the 50 files and seperate the three structs and compile them into three csv files.

Can you help me with it ? I am sharing my code herebelow. Thank you in advance.

[file_list, path_n] = uigetfile('*dark.pico', 'Grab all the files', 'MultiSelect','on');

if iscell (file_list)== 0
    file_list = (file_list)
end

T = array2table(file_list);

for i = 1:length(file_list);
    
    
    loadfile = loadjson('{1}');
    s = struct(loadfile);
    
    x = s.Spectra(1).Pixels(:);
    T1 = table(x);
    x1 = s.Spectra(1).Metadata().Datetime;
    x2 = s.Spectra(1).Metadata().Direction;
    writetable(T1,'filename1.csv')
    
    y = s.Spectra(2).Pixels(:);
    T2 = table(y);
    y1 = s.Spectra(2).Metadata().Datetime;
    y2 = s.Spectra(2).Metadata().Direction;
    writetable(T2,'filename2.csv')
    
    z = s.Spectra(3).Pixels(:);
    T3 = table(z);
    z1 = s.Spectra(3).Metadata().Datetime;
    z2 = s.Spectra(3).Metadata().Direction;
    writetable(T3,'filename3.csv')
    
end

[Walter Roberson]

[file_list, path_n] = uigetfile('*dark.pico', 'Grab all the files', 'MultiSelect','on');

if ischar(file_list); file_list = {file_list}; end  %only one selected
if ~iscell(file_list); return; end   %user cancel

T = array2table(file_list);

for i = 1:length(file_list);
    
    this_file = fullfile(path_n, filelist{i});
    loadfile = loadjson(this_file);
    s = struct(loadfile);
    
    x = s.Spectra(1).Pixels(:);
    T1 = table(x);
    x1 = s.Spectra(1).Metadata().Datetime;
    x2 = s.Spectra(1).Metadata().Direction;
    writetable(T1,'filename1.csv')
    
    y = s.Spectra(2).Pixels(:);
    T2 = table(y);
    y1 = s.Spectra(2).Metadata().Datetime;
    y2 = s.Spectra(2).Metadata().Direction;
    writetable(T2,'filename2.csv')
    
    z = s.Spectra(3).Pixels(:);
    T3 = table(z);
    z1 = s.Spectra(3).Metadata().Datetime;
    z2 = s.Spectra(3).Metadata().Direction;
    writetable(T3,'filename3.csv')
    
end

However:

You do not use x1, x2, y1, y2, z1, z2 after you create them.

You are always overwriting all of the csv file content.

Instead of using tables for that purpose, you should consider using writematrix().

[Arvind Gauns]

Thank you for the solution.

using the code I could extract the values from first JSON file. I still have 70 with me.

how to incorporate that into the code? further i want the table to be consisting of 71 columns

[Walter Roberson]

@Arvind Gauns

No, that code extracts values from all of the selected files.

The problem is that you always write the results to the same place.

[Walter Roberson]

The code could be much shorter if only we had some assurance that all of the files have the same number of rows and columns -- or, alternatively, if it was acceptable to pad short rows with 0 instead of nan. It would also help to know that Datetime and Direction are columns.

Nearly all of the work here is in padding out content to make sure that we can put everything into rectangular arrays. The code would be much shorter if only we knew everything was the same size...

[file_list, path_n] = uigetfile('*dark.pico', 'Grab all the files', 'MultiSelect','on');

if ischar(file_list); file_list = {file_list}; end  %only one selected
if ~iscell(file_list); return; end   %user cancel

T = array2table(file_list);
x = []; x1 = []; x2 = [];
y = []; y1 = []; y2 = [];
z = []; z1 = []; z2 = [];

for i = 1:length(file_list);
    
    this_file = fullfile(path_n, filelist{i});
    loadfile = loadjson(this_file);
    s = struct(loadfile);
    
    tx = s.Spectra(1).Pixels(:);
    tx1 = s.Spectra(1).Metadata().Datetime;
    tx2 = s.Spectra(1).Metadata().Direction;
    
    ntx = size(tx,1); ntx1 = size(tx1,1); ntx2 = size(tx2,1);
    x(size(x,1)+1:ntx, 1:end) = nan;
    x1(size(x1,1)+1:ntx1, 1:end) = nan;
    x2(size(x2,1)+1:ntx2, 1:end) = nan;
    nx = size(x,1); nx1 = size(x1,1); nx2 = size(x2,1);
    tx(end+1:ntx) = nan; tx1(end+1:ntx1) = nan; tx2(end+1:ntx2) = nan;
    x(:,end+1:end+size(tx,2)) = tx;
    x1(:,end+1:end+size(tx1,2)) = tx1;
    x2(:,end+1:end+size(tx2,2)) = tx2;

    ty = s.Spectra(2).Pixels(:);
    ty1 = s.Spectra(2).Metadata().Datetime;
    ty2 = s.Spectra(2).Metadata().Direction;
    
    nty = size(ty,1); nty1 = size(ty1,1); nty2 = size(ty2,1);
    y(size(y,1)+1:nty, 1:end) = nan;
    y1(size(y1,1)+1:nty1, 1:end) = nan;
    y2(size(y2,1)+1:nty2, 1:end) = nan;
    ny = size(y,1); nx1 = size(y1,1); ny2 = size(y2,1);
    ty(end+1:nty) = nan; ty1(end+1:nty1) = nan; ty2(end+1:nty2) = nan;
    y(:,end+1:end+size(ty,2)) = ty;
    y1(:,end+1:end+size(ty1,2)) = ty1;
    y2(:,end+1:end+size(ty2,2)) = ty2;

    tz = s.Spectra(3).Pixels(:);
    tz1 = s.Spectra(3).Metadata().Datetime;
    tz2 = s.Spectra(3).Metadata().Direction;
    
    ntz = size(tz,1); ntz1 = size(tz1,1); ntz2 = size(tz2,1);
    z(size(z,1)+1:ntz, 1:end) = nan;
    z1(size(z1,1)+1:ntz1, 1:end) = nan;
    z2(size(z2,1)+1:ntz2, 1:end) = nan;
    nz = size(z,1); nz1 = size(z1,1); nz2 = size(x2,1);
    tz(end+1:ntz) = nan; tz1(end+1:ntz1) = nan; tz2(end+1:ntz2) = nan;
    z(:,end+1:end+size(tz,2)) = tz;
    z1(:,end+1:end+size(tz1,2)) = tz1;
    z2(:,end+1:end+size(tz2,2)) = tz2;

end

xr = max([size(x,1), size(x1,1), size(x2,1)]);
x(end+1:xr, 1:end) = nan;
x1(end+1:xr, 1:end) = nan;
x2(end+1:xr, 1:end) = nan;
Tx = table(x, x1, x2);
writetable(Tx, 'filename1.csv');

yr = max([size(y,1), size(y1,1), size(y2,1)]);
y(end+1:yr, 1:end) = nan;
y1(end+1:yr, 1:end) = nan;
y2(end+1:yr, 1:end) = nan;
Ty = table(y, y1, y2);
writetable(Ty, 'filename2.csv');

zr = max([size(z,1), size(z1,1), size(z2,1)]);
z(end+1:zr, 1:end) = nan;
z1(end+1:zr, 1:end) = nan;
z2(end+1:zr, 1:end) = nan;
Tz = table(z, z1, z2);
writetable(Tz, 'filename3.csv');

Did I mention how much of the work is needed because we need to assume that nothing is the same size as anything else, since we are not given assurances otherwise?

[Arvind Gauns]

Thank you for the code. It actually reduced a tonnes of my manyal work.

i had a small query in the lower section

• tw1 is date and time
• tw2 is characteristics od object (character)

i have to also extract these details from metadata. when the above variables are assigned, I get the value of the first file. but i dont get the values of the subsequent files. how can i midify the code for the same ?

w = s.Spectra(1).Pixels(:);
    tw1 = s.Spectra(1).Metadata().Datetime;
    tw2 = s.Spectra(1).Metadata().Direction;
 ntw1 = size(tw1,1) ; ntw2 = size(tw2,1); ntw = size(tw,1);
    w(size(w,1)+1:ntw, 1:end) = nan;
    w1(size(w1,1)+1:ntw1, 1:end) = nan;
    w2(size(w2,1)+1:ntw2, 1:end) = nan;
nw1 = size(w1,1); nw2 = size(w2,1); nw =  size(w,1);
    tw1(end+1:ntw1) = nan; tw2(end+1:ntw2) = nan; tw(end+1:ntw)=nan; 
    w(:,end+1:end+size(tw,2)) = tw;
    w1(:,end+1:end+size(tw1,2)) = tw1;
    w2(:,end+1:end+size(tw2,2)) = tw2;

[Walter Roberson]

In the below, you might need to adjust the initialization of Invalid_Direction and Empty_Direction .

Empty_Direction needs to be an empty object with no rows and no columns. Depending on exactly what is being stored in the file, it might potentially be '' or cell(0,0) or strings(0,0)

Invalid_Direction needs to be the appropriate datatype matching Empty_Direction, but with content that signals that the row is unused, padding. It should not typically be blanks, because blanks too often is valid, and you need to distinguish between valid content and padding that exists only to make everything work out to be the same length.

Caution: if the Direction comes out as array of char (so, fixed width) and that it has more than one column, then the code that assigns in Invalid_Direction to pad out to indicate unused rows, needs to be modified to repmat() to the appropriate number of rows, such as

y2(size(y2,1)+1:nty2, 1:end) = repmat(Invalid_Direction, nty2-size(y2,1), 1);

This is not needed if Invalid_Direction is a scalar character that can be repeated for every column (such as '?') . It is also not needed if Invalid_Direction is a cell array of character vector, or a scalar string() object.

[file_list, path_n] = uigetfile('*dark.pico', 'Grab all the files', 'MultiSelect','on');

if ischar(file_list); file_list = {file_list}; end  %only one selected
if ~iscell(file_list); return; end   %user cancel

Invalid_Direction = '?';
Empty_Direction = '';

T = array2table(file_list);
x = []; x1 = NaT(0,0); x2 = Empty_Direction;
y = []; y1 = NaT(0,0); y2 = Empty_Direction;
z = []; z1 = NaT(0,0); z2 = Invalid_Direction;



for i = 1:length(file_list);
    
    this_file = fullfile(path_n, filelist{i});
    loadfile = loadjson(this_file);
    s = struct(loadfile);
    
    tx = s.Spectra(1).Pixels(:);
    tx1 = s.Spectra(1).Metadata().Datetime;
    tx2 = s.Spectra(1).Metadata().Direction;
    
    ntx = size(tx,1); ntx1 = size(tx1,1); ntx2 = size(tx2,1);
    x(size(x,1)+1:ntx, 1:end) = nan;
    x1(size(x1,1)+1:ntx1, 1:end) = NaT;
    x2(size(x2,1)+1:ntx2, 1:end) = Invalid_Direction;
    nx = size(x,1); nx1 = size(x1,1); nx2 = size(x2,1);
    tx(end+1:ntx) = nan; tx1(end+1:ntx1) = nan; tx2(end+1:ntx2) = Invalid_Direction;
    x(:,end+1:end+size(tx,2)) = tx;
    x1(:,end+1:end+size(tx1,2)) = tx1;
    x2(:,end+1:end+size(tx2,2)) = tx2;

    ty = s.Spectra(2).Pixels(:);
    ty1 = s.Spectra(2).Metadata().Datetime;
    ty2 = s.Spectra(2).Metadata().Direction;
    
    nty = size(ty,1); nty1 = size(ty1,1); nty2 = size(ty2,1);
    y(size(y,1)+1:nty, 1:end) = nan;
    y1(size(y1,1)+1:nty1, 1:end) = NaT;
    y2(size(y2,1)+1:nty2, 1:end) = Invalid_Direction;
    ny = size(y,1); nx1 = size(y1,1); ny2 = size(y2,1);
    ty(end+1:nty) = nan; ty1(end+1:nty1) = NaT; ty2(end+1:nty2) = Invalid_Direction;
    y(:,end+1:end+size(ty,2)) = ty;
    y1(:,end+1:end+size(ty1,2)) = ty1;
    y2(:,end+1:end+size(ty2,2)) = ty2;

    tz = s.Spectra(3).Pixels(:);
    tz1 = s.Spectra(3).Metadata().Datetime;
    tz2 = s.Spectra(3).Metadata().Direction;
    
    ntz = size(tz,1); ntz1 = size(tz1,1); ntz2 = size(tz2,1);
    z(size(z,1)+1:ntz, 1:end) = nan;
    z1(size(z1,1)+1:ntz1, 1:end) = NaT;
    z2(size(z2,1)+1:ntz2, 1:end) = Invalid_Direction;
    nz = size(z,1); nz1 = size(z1,1); nz2 = size(x2,1);
    tz(end+1:ntz) = nan; tz1(end+1:ntz1) = NaT; tz2(end+1:ntz2) = Invalid_Direction;
    z(:,end+1:end+size(tz,2)) = tz;
    z1(:,end+1:end+size(tz1,2)) = tz1;
    z2(:,end+1:end+size(tz2,2)) = tz2;

end

xr = max([size(x,1), size(x1,1), size(x2,1)]);
x(end+1:xr, 1:end) = nan;
x1(end+1:xr, 1:end) = NaT;
x2(end+1:xr, 1:end) = Invalid_Direction;
Tx = table(x, x1, x2);
writetable(Tx, 'filename1.csv');

yr = max([size(y,1), size(y1,1), size(y2,1)]);
y(end+1:yr, 1:end) = nan;
y1(end+1:yr, 1:end) = NaT;
y2(end+1:yr, 1:end) = Invalid_Direction;
Ty = table(y, y1, y2);
writetable(Ty, 'filename2.csv');

zr = max([size(z,1), size(z1,1), size(z2,1)]);
z(end+1:zr, 1:end) = nan;
z1(end+1:zr, 1:end) = NaT;
z2(end+1:zr, 1:end) = Invalid_Direction;
Tz = table(z, z1, z2);
writetable(Tz, 'filename3.csv');

[Arvind Gauns]

Mr. @Walter Roberson thank you very much for the exact solution for my problem. I workled on the solution provided and came up with the latest code.

Now i am also geting most the results I am looking for. But the issue I am facing is that, first few of my readings doesnt have the 'z' component. ( to tell you a little in the background, the JSON file has 4 arrays, which will be utilised to obtain different readings. first few of them only have 3 arrays so the 4th arrey in the successing files arent retrieved in matrix format). Either i want NAN values or 0 in place of the blank values or a filter which will (1. select the files with 4 arrays & 2. keep the list of the files). can that be added ?

clear all
clc

[file_list, path_n] = uigetfile('*dark.pico', 'Grab all the files', 'MultiSelect','on');

if ischar(file_list); file_list = {file_list}; end  %only one selected
if ~iscell(file_list); return; end   %user cancel

Invalid_Direction = '?';
Empty_Direction = '';

T = array2table(file_list);

w = []; x = []; y = []; z = [];
w1 = Empty_Direction; x1 = Empty_Direction; y1 = Empty_Direction; z1 = Empty_Direction;
w2 = Empty_Direction; x2 = Empty_Direction; y2 = Empty_Direction; z2 = Empty_Direction;
w3 = []; x3 = []; y3 = []; z3 = [];

for i = 1:length(file_list)
    
    this_file = fullfile(path_n, file_list{i});
    loadfile = loadjson(this_file);
    s = struct(loadfile);
    
    
    tw = s.Spectra(1).Pixels(:);
    tw1 = s.Spectra(1).Metadata().Datetime;
    tw2 = s.Spectra(1).Metadata().Direction;
    tw3 = s.Spectra(1).Metadata().IntegrationTime;
    
 ntw1 = size(tw1,1) ; ntw2 = size(tw2,1); ntw3 = size(tw3,1); ntw = size(tw,1);
    w(size(w,1)+1:ntw, 1:end) = nan;
    w1(size(w1,1)+1:ntw1, 1:end) = nan;
    w2(size(w2,1)+1:ntw2, 1:end) = nan;
    w3(size(w3,1)+1:ntw2, 1:end) = nan;
    nw1 = size(w1,1); nw2 = size(w2,1); nw3 = size(w3,1); nw =  size(w,1);
    tw1(end+1:ntw1) = nan; tw2(end+1:ntw2) = nan; tw3(end+1:ntw3) = nan; tw(end+1:ntw)=nan; 
    w(:,end+1:end+size(tw,2)) = tw;
    w1(:,end+1:end+size(tw1,2)) = tw1;
    w2(:,end+1:end+size(tw2,2)) = tw2;
    w3(:,end+1:end+size(tw3,2)) = tw3;
    
    
    tx = s.Spectra(2).Pixels(:);
    tx1 = s.Spectra(2).Metadata().Datetime;
    tx2 = s.Spectra(2).Metadata().Direction;
    tx3 = s.Spectra(2).Metadata().IntegrationTime;
    
ntx1 = size(tx1,1) ; ntx2 = size(tx2,1); ntx3 = size(tx3,1); ntx = size(tx,1);
    x(size(x,1)+1:ntx, 1:end) = nan;
    x1(size(x1,1)+1:ntx1, 1:end) = nan;
    x2(size(x2,1)+1:ntx2, 1:end) = nan;
    x3(size(x3,1)+1:ntx2, 1:end) = nan;
    nx1 = size(x1,1); nx2 = size(x2,1); nx3 = size(x3,1); nx =  size(x,1);
    tx1(end+1:ntx1) = nan; tx2(end+1:ntx2) = nan; tx3(end+1:ntx3) = nan; tx(end+1:ntx)=nan; 
    x(:,end+1:end+size(tx,2)) = tx;
    x1(:,end+1:end+size(tx1,2)) = tx1;
    x2(:,end+1:end+size(tx2,2)) = tx2;
    x3(:,end+1:end+size(tx3,2)) = tx3;
    
    
    ty = s.Spectra(3).Pixels(:);
    ty1 = s.Spectra(3).Metadata().Datetime;
    ty2 = s.Spectra(3).Metadata().Direction;
    ty3 = s.Spectra(3).Metadata().IntegrationTime;
 
nty1 = size(ty1,1) ; nty2 = size(ty2,1); nty3 = size(ty3,1); nty = size(ty,1);
    y(size(y,1)+1:nty, 1:end) = nan;
    y1(size(y1,1)+1:nty1, 1:end) = nan;
    y2(size(y2,1)+1:nty2, 1:end) = nan;
    y3(size(y3,1)+1:nty2, 1:end) = nan;
    ny1 = size(y1,1); ny2 = size(y2,1); ny3 = size(y3,1); ny =  size(y,1);
    ty1(end+1:nty1) = nan; ty2(end+1:nty2) = nan; ty3(end+1:nty3) = nan; ty(end+1:nty)=nan; 
    y(:,end+1:end+size(ty,2)) = ty;
    y1(:,end+1:end+size(ty1,2)) = ty1;
    y2(:,end+1:end+size(ty2,2)) = ty2;
    y3(:,end+1:end+size(ty3,2)) = ty3;
    

    tz = s.Spectra(4).Pixels(:);
    tz1 = s.Spectra(4).Metadata().Datetime;
    tz2 = s.Spectra(4).Metadata().Direction;
    tz3 = s.Spectra(4).Metadata().IntegrationTime;

ntz1 = size(tz1,1) ; ntz2 = size(tz2,1); ntz3 = size(tz3,1); ntz = size(tz,1);
    z(size(z,1)+1:ntz, 1:end) = nan;
    z1(size(z1,1)+1:ntz1, 1:end) = nan;
    z2(size(z2,1)+1:ntz2, 1:end) = nan;
    z3(size(z3,1)+1:ntz2, 1:end) = nan;
    nz1 = size(z1,1); nz2 = size(z2,1); nz3 = size(z3,1); nz =  size(z,1);
    tz1(end+1:ntz1) = nan; tz2(end+1:ntz2) = nan; tz3(end+1:ntz3) = nan; tz(end+1:ntz)=nan; 
    z(:,end+1:end+size(tz,2)) = tz;
    z1(:,end+1:end+size(tz1,2)) = tz1;
    z2(:,end+1:end+size(tz2,2)) = tz2;
    z3(:,end+1:end+size(tz3,2)) = tz3; 
    
end

wr = max([size(w,1), size(w1,1), size(w2,1),size(w3,1)]);
w(end+1:wr, 1:end) = nan;
w1(end+1:wr, 1:end) = Invalid_Direction;
w2(end+1:wr, 1:end) = Invalid_Direction;
w3(end+1:wr, 1:end) = nan;
w4= table( w./w3(1,:));
%Tw = [w4;w];

writetable(w4, 'QEP01348_downwelling.csv');

xr = max([size(x,1), size(x1,1), size(x2,1),size(x3,1)]);
x(end+1:xr, 1:end) = nan;
x1(end+1:xr, 1:end) = Invalid_Direction;
x2(end+1:xr, 1:end) = Invalid_Direction;
x3(end+1:xr, 1:end) = nan;
x4= table(x./x3(1,:));
% Tx = table(x, x1, x2, x3);
writetable(x4, 'QEP01349_downwelling.csv');

yr = max([size(y,1), size(y1,1), size(y2,1), size(y3,1)]);
y(end+1:yr, 1:end) = nan;
y1(end+1:yr, 1:end) = Invalid_Direction;
y2(end+1:yr, 1:end) = Invalid_Direction;
y3(end+1:yr, 1:end) = nan;
y4= table(y./y3(1,:));
% Ty = table(y, y1, y2, y3);
writetable(y4, 'QEP01348_upwelling.csv');

zr = max([size(z,1), size(z1,1), size(z2,1), size(z3,1)]);
z(end+1:zr, 1:end) = nan;
z1(end+1:zr, 1:end) = Invalid_Direction;
z2(end+1:zr, 1:end) = Invalid_Direction;
z3(end+1:zr, 1:end) = nan;
% Tz = table(z, z1, z2, z3);
z4= table(z./z3(1,:));
writetable(z4, 'QEP01349_upwelling.csv');