Manually modifying weights in Matlab SeriesNetwork without retraining

Dear all, I want to manually change weights in a convolutional layer of a trained network, keeping all weights in all other layers constant and explicitly see how classifying accuracy is changing. As far as I am concerned, to classify using modified weights I have to initialize new network with trainNetwork and train it.

net = trainNetwork(merchImagesTrain,layers,options);

I don't want to train at all, I want keep every weight in every layer as it is. Aborting training at the very beginning or shuffling training labels doesn't work well for this problem because there are tiny changes caused by these operations to the weights anyways.

[Kirill Korotaev]

I am going to answer my own question in case anybody meets the same goal.

The key is to set 'L2Regularization' parameter to 0, set minuscule learning rate and train for 1 epoch