필터 지우기
필터 지우기

Coalesce data from 1000 monte Carlo simulations with different data file lengths?

조회 수: 1 (최근 30일)
Hi all,
I'm trying to coalesce data from 1000 monte Carlo simulations of contaminant infiltration into the ground. Each Monte Carlo run generated a file Obs_Node.txt. I want to skip the first 11 lines of this .txt file, then isolate each data column beneath that to save for future analysis. My challenge is that each Monte Carlo run has unique time convergence intervals...ie the length of the data columns are different for each MC iteration's Obs_Node.txt file. In other words, in one Path(i) (ie Monte Carlo iteration) there would be 123456 rows of data for each of the 5 columns and in another Path(i) there may be 65432 rows of data for each of the .txt's 5 data columns. So the ## changes with each Path(i).
My questions are:
1) how do I preallocate when the length of the data changes for each Path(i) at the top of the code?
2) how do I tell Matlab to scan the data when I don't know the length of the data colums ?
I attached an example of Obs_Node.txt for reference....there are only 5 rows in this reference file for simplicity.
num_sim = 1000; %number of moonte carlo simulations
%Obs_Node.out - preallocation of desired output data from each monte carlo run
TIME_INTERVAL = zeros(##,num_sim);
Node_CONC=zeros(##,num_sim);
THETA_T1 = zeros(##,num_sim);
FLUX_T1 = zeros(##,num_sim);
CONC_T1 = zeros(##,num_sim);
%~~~~~~~~~~Coalesce data from Obs_Node.out files~~~~~~~~~~~~
for i=1:num_sim
Obs_Node = fopen(["/Users/apple/Dropbox/Desktop/Simulations/MC_"+num2str(i)+'/Obs_Node.out']); % Open monte carlo output file in Path (i)
skip_lines=11; %skip all the lines until the output data of interest
for k=1:(skip_lines)
x=fgetl(Obs_Node);
end
temp1 = fscanf(Obs_Node,'%f',[5,##]); %scan the matrix of data (QUESTION 2 ABOVE)
TEMP1 = temp1'; % transpose data
time_interval = TEMP1(:,1); % Time intervals - 1st data column
TIME_INTERVAL(:,i) = time_interval(:); % saves each iteration's data
theta_T1 = TEMP1(:,3); % Hydraulic Conductivity - 3rd data column
THETA_T1(:,i) = theta_T1(:); % saves each iteration's data
fclose(Obs_Node);
end

답변 (2개)

Benjamin Thompson
Benjamin Thompson 2022년 10월 24일
With the data import tool you can generate a function like this. It needs some editing to handle some of the odd formatting of your file. Then you can pass [12 Inf] as the datalines argument. The "end" at the end of your file is difficult, so you can either remove the last line from the output after loading the file data, or have a separate tool or function that removes the last row with "end" from files before you read them into MATLAB.

David Hill
David Hill 2022년 10월 24일
opts = detectImportOptions('Obs_Node.txt');
opts.VariableTypes={'double','double','double'};
opts.Delimiter={'\t'};
opts.DataLines=[12 Inf];
t=readmatrix('Obs_Node.txt',opts);
t(:,1)=[];
t(isnan(t(:,1)),:)=[]
t = 5×5
0.0010 -25.0000 0.2827 -382.0000 0 0.0020 -25.0000 0.2827 -244.0000 0 0.0030 -25.0000 0.2827 -170.0000 0 0.0043 -25.0000 0.2827 -114.0000 0 0.0060 -25.0000 0.2827 -75.4000 0
  댓글 수: 1
Wesser
Wesser 2022년 10월 24일
This works conceptually but the file that is being read is actually "Obs_Node.out" and not "Obs_Node.txt". I had changed the file ending for the question above as I couldn't upload the example file without changing the file from .out to .txt. The opts route seems feasible, but how do I get matlab to read the .out file path? It doesn't seem that there is a name-value pair that can access the .out files.
Name-Value Pairs supported with Import Options OPTS:
% See also spreadsheetImportOptions, delimitedTextImportOptions, fixedWidthImportOptions,xmlImportOptions, readtable, readtimetable, readmatrix, readcell

댓글을 달려면 로그인하십시오.

카테고리

Help CenterFile Exchange에서 Logical에 대해 자세히 알아보기

태그

제품


릴리스

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by