Which are the correct images and ROIs to train a haarcascade classifier for sad faces?

Hello, I want to create a haarcascade classifier to detect sad faces. I plan to replace the smile haarcascade classifier that an existing example of opencv uses and perform sadness detection. Same for open mouth detection. The problem is I am not sure I am using the correct positive and negative sample images. I have gathered about 1200 possitive images with sad faces of 4 different persons with some different angles and about 12000 negative images with faces of the same persons and different mouth positions from the sad ones. I pointed only the mouths as ROIs on the possitive pictures using the Training Image Labeler app and exported the ROIs to use them when I call the trainCascadeObjectDetector function. When I ran this function with all these images my system ran out of memory so I tried loading 100 positive samples and 1500 negative but the produced xml does not achieve the sadness detection. I provide you a link with one sample positive photo with ROI (https://www.dropbox.com/s/ou4dssajgiyhcrb/Untitled.png?dl=0 )and one negative photo ( https://www.dropbox.com/s/jwzewh59ebph2ut/5_1.jpg?dl=0 ). Can someone tell me if the samples and all the procedure I followed is correct? Should I load so many negative samples?