Convert python numpy array to double

The plot call below will throw an error because x is not a matlab type. How do you convert a python numpy array to a regular matlab matrix?

x = py.numpy.random.random([4,4]);
plot(x)

Sincerely, Peter

[Geoff Hayes]

Peter - you must be using the new functionality from R2014b which allows for easier to interface with Python. In this example, what does x return?

[Peter]

Geoff, yes I try to.

[Peter]

>> x
x = 
  Python ndarray with properties:

             T: [1x1 py.numpy.ndarray]
          base: [1x1 py.NoneType]
        ctypes: [1x1 py.numpy.core._internal._ctypes]
          data: [1x1 py.buffer]
         dtype: [1x1 py.numpy.dtype]
         flags: [1x1 py.numpy.flagsobj]
          flat: [1x1 py.numpy.flatiter]
          imag: [1x1 py.numpy.ndarray]
      itemsize: 8
        nbytes: 128
          ndim: 2
          real: [1x1 py.numpy.ndarray]
         shape: [1x1 py.tuple]
          size: 16
       strides: [1x1 py.tuple]
      [[ 0.84501423  0.72129285  0.53197632  0.56672641]
       [ 0.353229    0.40170347  0.99848745  0.74374568]
       [ 0.15023563  0.72432396  0.05446134  0.00455388]
       [ 0.85040029  0.65360657  0.35074251  0.08965406]]
  >>

Answer 1

[David Garrison]

Beginning in MATLAB R2018b, Python functions that accept numpy arrays may also accept MATLAB arrays without explicit conversion. When necessary, a numpy array can be created explicitly from a MATLAB array. For example, if you have a supported version of Python that is installed with the numpy library, you can do the following:

>> x = rand(2,2); % MATLAB array

>> y = py.numpy.array(x); % numpy array

y =

Python ndarray:

0.5943 0.8064

0.6133 0.1372

Use details function to view the properties of the Python object.

Use double function to convert to a MATLAB array.

Also beginning in MATLAB R2018b, it is possible to convert numeric numpy arrays returned from Python into MATLAB arrays. For example:

>> y = py.numpy.random.random([int32(2), int32(2)]) % numpy array

y =

Python ndarray:

0.5943 0.8064

0.6133 0.1372

Use details function to view the properties of the Python object.

Use double function to convert to a MATLAB array.

>> x = 2*double(y) % MATLAB array

x =

1.1885 1.6129

1.2266 0.2744

See the MATLAB documentation on Passing Matrices and Multidimensional Arrays for additional Information.

[Badr Hisham]

I got this error

Conversion to double from cell is not possible.

[abraham rodriguez]

@Badr Hisham In latest R2021a, you can pass directly a numpy ndarray to double() and it will convert to a native matlab matrix.

[winkmal]

y = py.numpy.random.random([int32(2), int32(2)])  

gives

Python Error: AttributeError: 'array.array' object has no attribute 'fromstring'

probably due to this. Python 3.9 here.

Maybe you can update your post, since Python 3.7 support will be removed on R2022a.

Answer 2

[Jim Hokanson]

Here's another approach. This is alluded to by http://www.mathworks.com/help/matlab/matlab_external/handling-data-returned-from-python.html

data = double(py.array.array('d',py.numpy.nditer(x))); %d is for double, see link below on types
data = reshape(data,[4 4])'; %Could incorporate x.shape here ...

py.array.array is apparently the way Matlab suggests getting data from Python into Matlab (see link above) https://docs.python.org/2/library/array.html

This however requires an iterable over which the array can be constructed, hence the call to nditer()

Once the value is an array, then Matlab has written functionality for casting to a Matlab type, in this case via double.

[Peter]

Jim, this is a beautiful solution. And it's quick too. Thank you so much for the code and for your explanation, Peter.

[Daniel Golden]

Minor modifications to make the code more generalizable, including some you suggested in your comments:

    data_row = double(py.array.array('d', py.numpy.nditer(x, pyargs('order', 'F'))));  % Add order='F' to get data in column-major order (as in Fortran 'F' and Matlab
    data_size = cell2mat(cell(x.shape));
    data = reshape(data, data_size);  % No need for transpose, since we're retrieving the data in column major order

[David Laurenson]

I love this solution! I had to modify it slightly (I'm not sure if this is a version issue or not), as cell2mat didn't work properly for me as cell() returned a cell array of py.int, not integers... Then I got to addressing the issue of multidimensional arrays, and realised that things were getting more complicated. As a result, I've written two functions - mat2nparray and nparray2mat - that will convert between the two formats:

function result = mat2nparray( matarray )
%mat2nparray Convert a Matlab array into an nparray
%   Convert an n-dimensional Matlab array into an equivalent nparray  
data_size=size(matarray);
if length(data_size)==1
    % 1-D vectors are trivial
    result=py.numpy.array(matarray);
elseif length(data_size)==2
    % A transpose operation is required either in Matlab, or in Python due
    % to the difference between row major and column major ordering
    transpose=matarray';
    % Pass the array to Python as a vector, and then reshape to the correct
    % size
    result=py.numpy.reshape(transpose(:)', data_size);
else
    % For an n-dimensional array, transpose the first two dimensions to
    % sort the storage ordering issue
    transpose=permute(matarray,[2 1 3:length(data_size)]);
    % Pass it to python, and then reshape to the python style of matrix
    % sizing
    result=py.numpy.reshape(transpose(:)', fliplr(size(transpose)));
end
end

and

function result = nparray2mat( nparray )
%nparray2mat Convert an nparray from numpy to a Matlab array
%   Convert an n-dimensional nparray into an equivalent Matlab array
data_size = cellfun(@int64,cell(nparray.shape));
if length(data_size)==1
    % This is a simple operation
    result=double(py.array.array('d', py.numpy.nditer(nparray)));
elseif length(data_size)==2
    % order='F' is used to get data in column-major order (as in Fortran
    % 'F' and Matlab)
    result=reshape(double(py.array.array('d', ...
        py.numpy.nditer(nparray, pyargs('order', 'F')))), ...
        data_size);
else
    % For multidimensional arrays more manipulation is required
    % First recover in python order (C contiguous order)
    result=double(py.array.array('d', ...
        py.numpy.nditer(nparray, pyargs('order', 'C'))));
    % Switch the order of the dimensions (as Python views this in the
    % opposite order to Matlab) and reshape to the corresponding C-like
    % array
    result=reshape(result,fliplr(data_size));
    % Now transpose rows and columns of the 2D sub-arrays to arrive at the
    % correct Matlab structuring
    result=permute(result,[2 1 3:length(data_size)]);
end
end

[Christoph Wiedemann]

Hey thanks for all these beautiful functions. I had to modify David's solution for multi-dimensional arrays. Here is the new code, embedded in a matlab class using static functions, and some test code:

classdef matpy
    %MATPY Summary of this class goes here
    %   Detailed explanation goes here

      properties
      end

      methods(Static)

          function result = mat2nparray( matarray )
              %mat2nparray Convert a Matlab array into an nparray
              %   Convert an n-dimensional Matlab array into an equivalent nparray  
              data_size=size(matarray);
              if length(data_size)==1
                  % 1-D vectors are trivial
                  result=py.numpy.array(matarray);
              elseif length(data_size)==2
                  % A transpose operation is required either in Matlab, or in Python due
                  % to the difference between row major and column major ordering
                  transpose=matarray';
                  % Pass the array to Python as a vector, and then reshape to the correct
                  % size
                  result=py.numpy.reshape(transpose(:)', int32(data_size));
              else
                  % For an n-dimensional array, transpose the first two dimensions to
                  % sort the storage ordering issue
                  transpose=permute(matarray,[length(data_size):-1:1]);
                  % Pass it to python, and then reshape to the python style of matrix
                  % sizing
                  result=py.numpy.reshape(transpose(:)', int32(fliplr(size(transpose))));
              end
          end

          function result = nparray2mat( nparray )
              %nparray2mat Convert an nparray from numpy to a Matlab array
              %   Convert an n-dimensional nparray into an equivalent Matlab array
              data_size = cellfun(@int64,cell(nparray.shape));
              if length(data_size)==1
                  % This is a simple operation
                  result=double(py.array.array('d', py.numpy.nditer(nparray)));
              elseif length(data_size)==2
                  % order='F' is used to get data in column-major order (as in Fortran
                  % 'F' and Matlab)
                  result=reshape(double(py.array.array('d', ...
                      py.numpy.nditer(nparray, pyargs('order', 'F')))), ...
                      data_size);
              else
                  % For multidimensional arrays more manipulation is required
                  % First recover in python order (C contiguous order)
                  result=double(py.array.array('d', ...
                      py.numpy.nditer(nparray, pyargs('order', 'C'))));
                  % Switch the order of the dimensions (as Python views this in the
                  % opposite order to Matlab) and reshape to the corresponding C-like
                  % array
                  result=reshape(result,fliplr(data_size));
                  % Now transpose rows and columns of the 2D sub-arrays to arrive at the
                  % correct Matlab structuring
                  result=permute(result,[length(data_size):-1:1]);
              end
          end

          function test()
              A = 1:5;
              Anp = matpy.mat2nparray(A);

              sa = size(A);
              sAnp = cellfun( @(x) double(x), cell(Anp.shape));
              assert (all(sAnp == sa));
              for i1=1:size(A,1)
                  for i2=1:size(A,2)
                      assert(A(i1,i2) == Anp.item(int32(i1-1), int32(i2-1)));
                  end
              end
              Anpm = matpy.nparray2mat(Anp);
              assert(all(A == Anpm));

              A = reshape(1:6, [2,3]);
              Anp = matpy.mat2nparray(A);

              sa = size(A);
              sAnp = cellfun( @(x) double(x), cell(Anp.shape));
              assert (all(sAnp == sa));
              for i1=1:size(A,1)
                  for i2=1:size(A,2)
                      assert(A(i1,i2) == Anp.item(int32(i1-1), int32(i2-1)));
                  end
              end
              Anpm = matpy.nparray2mat(Anp);
              assert(all(all(A == Anpm)));

              A = reshape(1:(2*3*4), [2,3,4]);
              Anp = matpy.mat2nparray(A);

              sa = size(A);
              sAnp = cellfun( @(x) double(x), cell(Anp.shape));
              assert (all(sAnp == sa));
              for i1=1:size(A,1)
                  for i2=1:size(A,2)
                      for i3=1:size(A,3)
                          display(sprintf('%d %d %d -> %f %f', i1,i2,i3, A(i1,i2,i3), Anp.item(int32(i1-1), int32(i2-1), int32(i3-1))));
                          assert (A(i1,i2,i3) == Anp.item(int32(i1-1), int32(i2-1), int32(i3-1)))
                      end
                  end
              end
              Anpm = matpy.nparray2mat(Anp);
              assert(all(all(all(A == Anpm))));
          end

      end

end

[Dev-iL]

Hi Christoph,

Thanks for the code! However, it doesn't support the conversion of scalar arrays:

Error using reshape
Size vector must have at least two elements.

Error in matpy.nparray2mat (line 55)
                  result=reshape(result,fliplr(data_size));

This can be solved by modifying the first `if` statement in nparray2mat to:

if any(numel(data_size) == [0,1])

Cheers!

[Eric Cousineau]

To build further upon y'all's work, I've made a really rough prototype, using `matpy`, to get `numpy`-friendly proxy that allows more natural referencing, indexing / slicing, etc., via `subsref` and `subsassgn`:

• NumPyProxy
• Example slicing / indexing into `numpy` array in MATLAB

Example code:

pyA = py.numpy.eye(3);
mlA = NumPyProxy(pyA);

mlA(:)
double(mlA(:))
mlA(3, 2)

sub = mlA([2, 1], 3)
double(sub)  % 2017-05-30T01:09-04:00 - Presently a bug, wrong order
sub2 = mlA([1, 2], 3)
double(sub2)

mlA([2, 1], 3) = 100 * [1, 2]
mlA(:) = 5

Note that PyProxy also permits casting of >1-D arrays, using `subsref` tricks, whereas MATLAB R2016b presently does not permit passing 2-D matrices (at least, as far as I've tried using other people's examples).

This is an ongoing effort to test some Drake functionality: GitHub Issue

This has only been tested / tinkered with in R2016b, and still needs refinement / robustification.

I will try to see if hacks like this aren't needed in future versions of MATLAB.

Thanks a ton for taking the time to post y'all's stuff!

Answer 3

[Shaowu Pan]

Weird discussion...

def npArray2Matlab(x):
    return matlab.double(x.tolist())

[Laszlo Kormoczi]

Then how to use this in MATLAB?

[timo kvamme]

Thank you, this worked

Answer 4

[Jim Hokanson]

Here's a very ugly solution.

temp = cellfun(@cell,cell(x.tolist),'un',0);
data = cell2mat(vertcat(temp {:}));

A slightly cleaner solution:

data = typecast(uint8(char(x.flat.base.data)),'double');
data = reshape(data,[4 4])'; %Could incorporate x.shape here ...

Then of course:

plot(data)

Sorry I don't have time to explain how you would know that you need to do these things. I have to run!

Jim

[Peter]

Thank you Jim for your answer!

The cellfun solution works. But the typecasting does not work all the times - it's very odd.

Peter

[Jim Hokanson]

This assumes the data type is double. If it isn't then the typecasting won't work.

[Peter]

Jim, it's a bit complicated. For example, if I run:

for jj = 1:10
    x = py.numpy.random.random([10,10]);
    try
        data = typecast(uint8(char(x.flat.base.data)),'double');
        disp(numel(data))
    catch me
        warning(me.message)
    end
end

I should get 100 doubles, but I end up with something very different:

Warning: The first input must contain a multiple of 8 elements to convert from uint8 (8 bits) to double (64 bits). 
         29.00
          8.00
          7.00
         10.00
Warning: The first input must contain a multiple of 8 elements to convert from uint8 (8 bits) to double (64 bits). 
         24.00
         20.00
Warning: The first input must contain a multiple of 8 elements to convert from uint8 (8 bits) to double (64 bits). 
         56.00

Do you have any idea why the typecasting is broken? I would guess that it's something to do with the conversion from the flat.base to char.

Peter.

[Jim Hokanson]

Interesting. The problem occurs when there is a zero in the underlying raw buffer (not a zero value but a 0 byte). I'm not sure if this desired or if it is a bug.