How do I separate raw data in columns, effectively?

I have been trying to take the raw data from a txt. file that came from a beagleboneblack (small application computer) and have had a few problems separating some of the values, how could I affect the file once it is imported so that specific rows are chosen.

I have a lot of data and cannot afford to sit down and pick them one by one. I have made separate file for most of them, but since they overlap in some cases it is impossible for me to see them correctly. here is an example:

Accel (mg), Mag (mGauss) 946808257.344 (-85.9375, 33.203125, 997.0703125), (-27.27272727272727, -357.27272727272725, 1.8181818181818181)] Gyro (dps) 946808257.346 (-2.532958984375, 0.6103515625, 1.0528564453125)] P(mbar), T(C) 946808257.35 1008.09423828125, 27.545833333333334] Accel (mg), Mag (mGauss) 946808257.443 (-83.984375, 30.2734375, 985.3515625), (-23.636363636363633, -351.81818181818181, 4.545454545454545)] Gyro (dps) 946808257.446 (-2.3651123046875, 0.396728515625, 0.9613037109375)] P(mbar), T(C) 946808257.449 1008.3203125, 27.556249999999999] Accel (mg), Mag (mGauss) 946808257.546 (-81.0546875, 29.296875, 989.2578125), (-23.636363636363633, -357.27272727272725, 0.90909090909090906)] Gyro (dps) 946808257.56 (-2.349853515625, 0.6256103515625, 0.701904296875)] P(mbar), T(C) 946808257.577 1007.97705078125, 27.524999999999999]

Thank you for your help!

[dpb]

What is the raw data format? Surely there are records?

I looked up the board manual but obviously this is an app using the board that has its own support somewhere one presumes? Unless it's something developed where you're presently working in which case they should have that info.

[Space Science undergrad]

it comes out of a python programing code set on my satellite components. I can ask my prof if there is another format, but so far this is all I got.

[dpb]

If you open a file in the Matlab editor what does it look like? Is it all on one long line w/ no separation of the various pieces?

Seems like it should be simple enough to hack the Python code to have it spit out something far more easily dealt with as a starting point...

[Space Science undergrad]

the problem is that the python is on a small beaglebone that can only process so much at a time, if I ask him to spit it out differently, it wont calculate fast enough to give me the amount of data needed. When I bring it in editor, it separates it in three distinct columns that repeats itself, but the separation of data is overlapping, which does not let me take the data in a productive context. If you copy and paste my example in a notepad and open it with, you might be able to understand my problem. class now, ill be back this afternoon, thanks for your help.

[dpb]

OK, if look in more detail, it appears there is a soft record marker of a ']' -- assuming that is so then it appears there are sets of three records each as follows...

Accel (mg), Mag (mGauss) 9...7.344 (-85, 33, 997), (-27, 357, 1)]
Gyro (dps) 9...7.346 (-2.532958984375, 0.6103515625, 1.0528564453125)]
P(mbar), T(C) 9...7.35 1008.09423828125, 27.545833333333334]

where I shortened the values of the first "record" to fit on a single line for clarity.

I'm guessing the first value 946808257.344 and so on is a timestamp, maybe, followed by the three-coordinate values of the named variables?

That would make the numbers of the values match up with the labels. If this is so, let know--I'll wait to make certain know what it is we're actually parsing before actually trying to do so...

Answer 1

[dpb]

OK, I just went ahead with the above assumption...one way reading a group of three as a set--

>> fmt='Accel (mg), Mag (mGauss) %f (%f, %f, %f), (%f, %f, %f)] Gyro (dps) %f (%f, %f, %f)] P(mbar), T(C) %f %f, %f]';
>> c=textscanf(fid,fmt,'collectoutput',0);
>> acc=cell2mat(c(2:4))
acc =
-85.9375   33.2031  997.0703
-83.9844   30.2734  985.3516
-81.0547   29.2969  989.2578
>> mag=cell2mat(c(5:7))
mag =
-27.2727 -357.2727    1.8182
-23.6364 -351.8182    4.5455
-23.6364 -357.2727    0.9091
>> gyr=cell2mat(c(9:11))
gyr =
 -2.5330    0.6104    1.0529
 -2.3651    0.3967    0.9613
 -2.3499    0.6256    0.7019
>> P=cell2mat(c(13))
P =
 1.0e+03 *
  1.0081
  1.0083
  1.0080
>> T=cell2mat(c(14))
T =
 27.5458
 27.5562
 27.5250
>>

The various timestamps(?) are in c(1), c(8) and c(12), respectively. The total number of them don't match up with the other values, you'll have to figure out where/how to match them up.

Alternatively, you could instead of defining the format string for the full set of three records do one for each and put the textscan call in a loop and have the timestamps retained with the records instead of sorting them out later. All depends on what you're after and how will use them in the end as to what's more convenient. But, it does get the data variables into usable form automagically.

[Space Science undergrad]

this is exactly what I was looking for, I will make independent graphs with their respective timestamps and create a timeline to see where values match up. Thanks a million for your effort and +1 for automagically!