How is the 2D filter function used in this code?

In one of the cody answers (solution 712713), I found a following code:

function B = hanlelize(A)
    F = flip(eye(111)); 
    B1 = filter2(F,A); 
    B2 = filter2(F,A|1); 
    B = B1./B2; 
end

This function convert an input matrix A to a Hankel matrix B by replacing each skew-diagonal of A with its mean. For example, if the input is

A =

     3     7    10     2
     3     5     1     2
     6     3     2     7

then, the matrix B1, B2, B will be

B1 =

     3    10    21     6
    10    21     6     4
    21     6     4     7
    
B2 =

     1     2     3     3
     2     3     3     2
     3     3     2     1

B =

     3     5     7     2
     5     7     2     2
     7     2     2     7

The code looks elegant but I'm not so familiar with the 2D filter function used in this code.

• How is the 2D filter working in the computation of B1? Why does the 2D filter by flipped identity matrix result in B1?
• How is the 2D filter working in the computation of B2? What does 'A|1' mean? I checked the documentation of filter2 but I couldn't find such an expression.

I'm looking forward to your help.

Answer 1

[Steve Eddins]

For each element in A, filter2 essentially "slides" the filter F so that its center lines up with the element in A, and then it does a point-wise multiplication between the elements in A and the corresponding elements in F, and then sums all those products. With a filter that is just ones along the anti-diagonal, each element of the output matrix is found by summing up the input matrix elements along the corresponding anti-diagonal. Eh, it's a bit hard to describe in words. Let's try it with a small A and small F.

A = magic(5)

A = 5×5

17 24 1 8 15 23 5 7 14 16 4 6 13 20 22 10 12 19 21 3 11 18 25 2 9

F = flip(eye(3))

F = 3×3

0 0 1 0 1 0 1 0 0

B1 = filter2(F,A)

B1 = 5×5

17 47 6 15 29 47 10 21 42 36 9 23 39 55 43 16 36 57 68 5 23 37 46 5 9

Where does B1(4,2) come from?

B1(4,2)

ans = 36

It is the sum of A(4,2), the element to its upper right, A(3,3), and the element to its lower left, A(5,1).

A(4,2) + A(3,3) + A(5,1)

ans = 36

In the expression A|1, the vertical bar is the element-wise OR operator. When you OR any value with 1, you always get 1, so this expression appears to be just a tricky way to compute true(size(A)). In Cody, solutions with a smaller code size (based on some metric) are scored higher, so perhaps A|1 had a slightly smaller code size metric than true(size(A)).

[Steve Eddins]

I made a small correction to my answer. Where I originally wrote false(size(A)), it should have been true(size(A)).

[Daigo]

Thank you for your detailed explanations! Now I can make a good use of filter2 function.