How to choose the optimum values of the training options Transfer learning of the pretrain Deep networks in MATLAB

Dear all I have a question, I am using Transfer Learning of the pretraining Deep learning networks I would like to know how to choose the optimum values of the training options(Solver, initial learning rate, max epoch, miniBatchsize etc) should I keep trying different values or choosing the default values, please