applyting function to all elements in a matrix

What is1./(1+exp(-z)), (z can be a matrix ) and how is it different from 1/(1+exp(-z))?

[Stephen23]

https://www.mathworks.com/help/matlab/matlab_prog/array-vs-matrix-operations.html

[Vladimir Sovkov]

The second version is a little bit strange, although without formal errors. The thing is that exp(-z) is per-element, i.e., it computes the matrix, whose every element is the exponent of the corresponding element of the initial matrix, while the operation / computes the matrix inversion. If you want all the operations to be done in the sense of the matrix algebra, you should use expm(-z). The first version is totally per-element, hence, it looks quite consistent.

Answer 1

[KALYAN ACHARJYA]

First one: 1./(1+exp(-z)), have you noticed {dot} before division sign, it represents the dot array operation (element wise), lets say matrix A & B,

>> a=magic(3);
>> b=rand(3,3);
>> a*b      % Matrix operation
ans =
    8.1855    8.5246    8.5199
    7.8620    6.5847   10.2725
   11.6650    9.5398    7.9509
>> a.*b   % Array element wise operation
ans =
    6.5178    0.9134    1.6710
    2.7174    3.1618    3.8282
    0.5079    0.8779    1.9150 

Please do read the above link referred by Stephen Cobeldick

So the expression 1./(1+exp(-z)), there is no issue considering z may have any matrix.

>> z=rand(4,4)  % random z considered
z =
    0.9649    0.4854    0.9157    0.0357
    0.1576    0.8003    0.7922    0.8491
    0.9706    0.1419    0.9595    0.9340
    0.9572    0.4218    0.6557    0.6787
>> result=1./(1+exp(-z))
result =

    0.7241    0.6190    0.7142    0.5089
    0.5393    0.6900    0.6883    0.7004
    0.7252    0.5354    0.7230    0.7179
    0.7226    0.6039    0.6583    0.6635

Second one: 1/(1+exp(-z))

The denominator 1+exp(-z) have no issue, considering z as a arbitary matrix, but when a scalar divided by matrix (vector), I don't think is there any direct approach. Read this thread, about "What's the meaning of a scalar divided by a vector" because 1 as numerator is scalar on the other hand denominator 1+exp(-z) is a vector.

In Matlab it shows the error related to dimention mismatch

>> result=1/(1+exp(-z))
Error using  / 
Matrix dimensions must agree.

Hope this helps to clear your doubt.

[Vladimir Sovkov]

Indeed, 1/... does not work as a matrix operation. To get it in the matrix sense, it must be either (1+expm(-z))^(-1) or inv(1+expm(-z)), and these expressions only make sense for square well-conditioned matrices. For some related purposes, the pseudoinversion pinv or the left division \ can be used.