Multiple input for GoogleNet network?

Hi,

I am new to deep learning applications. I am trying to use multiple inputs to feed GoogleNet pretrained network. For instance, I have dataset A images (10000) which has 2 classes. I also have dataset B (10000) from another sensor which has also 2 classes same as dataset A. Is it possible to feed the network with both datasets to improve the performance of classification?

I have tried to concetenate both dataset while giving to the input, i.e both dataset has equal size and 20000 images in total. But it didn't work well for accuracy. Then I thought I should extract features seperately by using two GoogleNet and then classify with one GoogleNet classifier. How can I do that? Or another approach for solution?

Thank you...