Coalesce data from 1000 monte Carlo simulations with different data file lengths?
조회 수: 1(최근 30일)
표시 이전 댓글
댓글: Benjamin Thompson 2022년 10월 24일
I'm trying to coalesce data from 1000 monte Carlo simulations of contaminant infiltration into the ground. Each Monte Carlo run generated a file Obs_Node.txt. I want to skip the first 11 lines of this .txt file, then isolate each data column beneath that to save for future analysis. My challenge is that each Monte Carlo run has unique time convergence intervals...ie the length of the data columns are different for each MC iteration's Obs_Node.txt file. In other words, in one Path(i) (ie Monte Carlo iteration) there would be 123456 rows of data for each of the 5 columns and in another Path(i) there may be 65432 rows of data for each of the .txt's 5 data columns. So the ## changes with each Path(i).
My questions are:
1) how do I preallocate when the length of the data changes for each Path(i) at the top of the code?
2) how do I tell Matlab to scan the data when I don't know the length of the data colums ?
I attached an example of Obs_Node.txt for reference....there are only 5 rows in this reference file for simplicity.
num_sim = 1000; %number of moonte carlo simulations
%Obs_Node.out - preallocation of desired output data from each monte carlo run
TIME_INTERVAL = zeros(##,num_sim);
THETA_T1 = zeros(##,num_sim);
FLUX_T1 = zeros(##,num_sim);
CONC_T1 = zeros(##,num_sim);
%~~~~~~~~~~Coalesce data from Obs_Node.out files~~~~~~~~~~~~
Obs_Node = fopen(["/Users/apple/Dropbox/Desktop/Simulations/MC_"+num2str(i)+'/Obs_Node.out']); % Open monte carlo output file in Path (i)
skip_lines=11; %skip all the lines until the output data of interest
temp1 = fscanf(Obs_Node,'%f',[5,##]); %scan the matrix of data (QUESTION 2 ABOVE)
TEMP1 = temp1'; % transpose data
time_interval = TEMP1(:,1); % Time intervals - 1st data column
TIME_INTERVAL(:,i) = time_interval(:); % saves each iteration's data
theta_T1 = TEMP1(:,3); % Hydraulic Conductivity - 3rd data column
THETA_T1(:,i) = theta_T1(:); % saves each iteration's data
댓글 수: 1
My example works on text files regardless of the extension. So *.out is fine.
With the data import tool you can generate a function like this. It needs some editing to handle some of the odd formatting of your file. Then you can pass [12 Inf] as the datalines argument. The "end" at the end of your file is difficult, so you can either remove the last line from the output after loading the file data, or have a separate tool or function that removes the last row with "end" from files before you read them into MATLAB.
댓글 수: 0
David Hill 2022년 10월 24일
opts = detectImportOptions('Obs_Node.txt');
댓글 수: 1
Find more on Spreadsheets in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!