How do I extract certain data from a table?

I'm working on a tall array, which contains multiple flightplan data (size over 100k rows, 5 columns) and I want to extract just certain flight routes (departureairport --> arrivalairport) and continue to work with them.

The table looks approximately like this:

   flightno.    depart.airp  arr.airp   depart.time   arr.time
   ---------    -----------  --------   -----------   --------

      111          BOS          LAX        ...           ...
      321          JFK          DEN        ...
      121          BOS          JFK
      222          DEN          BOS
      333          BOS          DEN

For the further data analysis I only want to work with flight data departuring from BOS.

Could someone help me on this issue? Thanks

[Birdman]

Can you send the tall array(data)?

[Philipp Henschel]

Not sure if I can share the file. Instead I posted you a few rows out of the tall array

   flighthistoryid    departureairport    arrivalairport    originaldepartureutc    originalarrivalutc 
    _______________    ________________    ______________    ____________________    ___________________

     2.803e+08         'PDX'               'LAS'             2012-11-14 02:55:00     2012-11-14 05:00:00
    2.8031e+08         'RDM'               'SEA'             2012-11-14 02:55:00     2012-11-14 04:02:00
     2.803e+08         'PUW'               'SEA'             2012-11-14 02:55:00     2012-11-14 04:05:00
    2.8029e+08         'MFR'               'PDX'             2012-11-14 02:55:00     2012-11-14 03:56:00
    2.8031e+08         'SLC'               'TWF'             2012-11-14 02:55:00     2012-11-14 03:59:00
    2.8028e+08         'HOU'               'DFW'             2012-11-14 02:55:00     2012-11-14 04:00:00
    2.8025e+08         'ATL'               'BOS'             2012-11-14 02:51:00     2012-11-14 05:25:00
    2.8025e+08         'ATL'               'STL'             2012-11-14 02:53:00     2012-11-14 04:45:00

Answer 1

[dpb]

I'd guess the ID column isn't a float but an integer number; need to ensure you've imported it with sufficient precision to be correct.

Then convert flight airport IDs to categorical arrays...

T(:,1:3)=categorical(T(:,1:3));  % where T is your table variable

That will make selection simple to write--

B=T(T.departure=='BOS',:);

You can write similar things with cell strings, but in general the syntax is more messy and the categorical variable type has some useful builtin utility functions for summaries and the like that can be helpful.

[Philipp Henschel]

Thanks for the help. Unfortunately, when using the code, I received following error message:

Error using categorical (line 277) Conversion from table not supported. Extract data from one or more table variables into an array using dot or brace subscripting. Then convert the array to a categorical array.

Any further ideas?

[dpb]

I'd figured that TMW would've gotten that limitation removed by now... :(

It's the array syntax trying to shorten the multiple variables into a vectorized form, sorry.

Use

T.varN=categorical(T.varN);

for each variable in the table or a loop with indexing for a column at a time with the curly braces indexing on the RHS to return the underlying data in each rather than the table that is the result of "regular" parentheses addressing.

[Philipp Henschel]

Now it's working. Your last comment was the key. Thank you

[dpb]

No problem; glad to help. It takes some time to learn the nuances of what is/isn't allowable inside the table object but overall it's pretty handy for many things. Be sure to read the doc on the categorical data type as well; there are as noted earlier some other facilities with it that can be very helpful depending on just what you want to do with the data. Grouping is one possibility that comes to mind...

[Padmamalini T H]

thanks a ton. i was struggling with the same issue as well

[abdul rehman]

T(:,1:3)=categorical(T(:,1:3)); % where T is your table variable

when I am trying this is says "Conversion from table not supported. Extract data from one or more table variables into an array using dot or brace subscripting. Then convert the array to a categorical array."

[dpb]

Huh. Surprised nobody else had noticed..."air code" (meaning just typed in at the prompt, not tested) tends to have such oversights. The expression needs the curlies ("{}") instead of parentheses to return the base array when using subscript indexing into a table.

T.name for a column variable does return the variable as its storage type; it's easy to forget that the array indexing form returns the table section referred to instead. It's consistent wit MATLAB syntax; just easy to forget/overlook.

[Walter Roberson]

    data = cat(1, image_patches,labels);

That code is overwriting all of data each iteration.

It looks to me as if data will not be a vector, but I do not seem to be able to locate any hellopatches() function so I cannot tell what shape it will be. As you are not doing imresize() I also cannot be sure that all of the images are the same size, so I cannot be sure that data will be the same size for each iteration. Under the circumstances you should be considering saving into a cell array.

Note: please do not post the same query multiple times. I found at least 10 copies of your query :(