- store (and use) numeric data as numeric, text data as text (or categorical), etc. This is much tricker with other container types like cell arrays.
- easily find groups based on one or more conditions, then apply functions or operations to those groups (e.g. exactly like your "plot the STDEV of all LED:s with filter F3 and Detector D2 at a certain LED current" example).
- can trivially include meta-data, eg. filenames, timestamps, or any other test parameters.
Strategy on how to save data from multiple runs of an experiment where variables changes
조회 수: 9 (최근 30일)
이전 댓글 표시
I need some guidance on the best way to structure and save data from experiments with many runs and chaning variables.
An example:
Let's say that I try to measure the light intesity from an LED after the light has passed an optical filter. I will do many experiments and change between different combinations of LEDs, filters and detectors, all with a traceable serial number. At every experiment i step the LED current between 0 and 100% in suitable steps and make 1000 samples at every current level so that I can analyze noise and drift at every level. Some combinantions of LED, filter and detector will be run several times. Every experiment that I perform will produce a text-file with the measured data inside.
How should I handle this data so that I easilly can make statistics from it in MATLAB?
Should I save everthing in tables, a large cell array or keep it in the text files? How do I map the LED, filter and detector ID:s to the data?
I would for example like to be able to easilly plot the STDEV of all LED:s with filter F3 and Detector D2 at a certain LED current. Or any other combination of equipment.
Any thoughts on how to save the data and keep mapping to the physical equipment would be much appreciated.
댓글 수: 1
Stephen23
2023년 3월 18일
편집: Stephen23
2023년 3월 18일
Use one table to conduct the statistical analyses.
That does not mean that all intermediate results and whatnot need to be stored there, but using one table has many benefits for the kind of processing that you seem to want to do:
채택된 답변
the cyclist
2023년 3월 18일
There's probably no single best answer here. But if the overall scale of the data is not too large, storing everything in one table is a very appealing way to go, and may be the most "modern" from a MATLAB point of view.
I would specifically recommend storing the data in what is known as the tidy format. Each row is a single observation, and the columns will track all the possible variables, which look to be
- LED type
- filter type
- detector type
- LED current
- sample number
- light intensity
Data are not implicitly coded into column names -- e.g. you do not want columns "Sample1", "Sample2", etc -- even though that sometimes seems more compact. The "long skinny" format is almost alway better for general analysis. That site I linked gives explanation. (It is written from the perspective of R, but the concept is agnostic to language.)
MATLAB operations on table are well designed to do summary and group statistics on data that are laid out like this.
댓글 수: 4
the cyclist
2023년 3월 22일
This forum is not a very good way to teach you from scratch how to use MATLAB in general, or about tables as a data type. It is more suited to targeted questions when you are stuck on a particular task.
You need to immerse yourself in the excellent documentation provided, and its many examples. I would start with the main documentation about tables. From there, there are many specific articles about how to do statistics, deal with missing data, etc. Maybe just try to set up a few of your variables at first, instead of your whole dataset.
Thinking about setting up the data, you asked, "Is it a good idea to make the variables Categorical?". The answer to that is no. If you have measured current, that would not be categorical. But maybe you have a variable for "Detector Type", which would be. The data type should match the variable. (This is one of the advantages of using tables rather than a purely numeric data type, which could not store "Type X" and "Type Y".)
Regarding your second question ... There are often different methods for achieving the same result. (It's not possible to state "the best" way, without more context and insight into your problem.) The exact syntax will depend on the datatype, but if
T(T.Var1 == 'X' & T.Var2 == 'Y' & T.Var3 == 'Z',:).Data;
worked to filter how you wanted, then
T(ismember(T.Var1,{'X1','X2'}) & T.Var2 == 'Y' & T.Var3 == 'Z',:).Data;
should work.
'X12'=='X2'
(That doesn't work because you are compareing a 1x3 array with a 1x2, and MATLAB doesn't know what you mean.) You need to use the strcmp function instead:
strcmp('X12','X2')
Alternatively, you could work with strings instead of character arrays:
"X12"=="X2"
I know, I know. It's a bit confusing and probably a bit overwhelming. Take it a step at a time, and post new, specific questions when you get stuck. Upload a small sample of your actual data/table if you can.
추가 답변 (0개)
참고 항목
카테고리
Help Center 및 File Exchange에서 Logical에 대해 자세히 알아보기
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!