A MUX block just combines its inputs into a single vector output so basic vector algebra will work. But it really depends on how the Masked Parameter is used in the custom block. If each signal needs is simply exposed addition, subtraction, multiplication, and division, you can simply vector input as a parameter with the same size as the MUX signal. Take the following model below as an example:
The masked block contains only a Gain block with the Parameter "Mult" as the gain.
The input parameter to the masked parameter is a vector as shown below:
Notice the Display block shows that each signal is multiplied by the corresponding multiplier in the masked parameter. This is a simplified version of how you manipulate a MUX signal using a gain. As I mentioned before, it might be more complicated with the custom block but you may try using a vector as an input.