click push button and interact with window opened by it

Hi,

I have a Matlab app called PIVlab (version 3.02) that I want to open and operate using a script. When I open the app, I'm prompted with this:

i want the script to push on the 'Import images' push button. I sometimes move the window arround because I use the computer while running the app so I need the script to work even if the window moved or is in the background. Once the button is clicked, this window pops up:

In this new window, I want to be able to change the input in the 'Current folder' text box with another file location and go to that location.

Next I want to be able to select ALL files that show in the panel below (where the files PIVlab_****_A and B are.

Once selected, I want the script to click on 'Add' and then on 'Import'.

If succesful, the window should close and the images should show up in the previous window.

I have a skeleton for the script (from ChatGPT) up to Step 2 but I don't know how to properly activate the buttons

function automatePIVlab(runNumbers)
    % Ensure the PIVlab app is loaded
    PIVlab_GUI;  % Opens the PIVlab App
    pause(4);   % Wait for the app to fully load

    % Load the run log for calibration data
    runLog = table2struct(readtable('D:/WASIRF2024_runLog.xlsx'));
    
    % Base directories for images, sessions, and mat files
    baseImageDir = 'D:/WASIRF2024PIV/';
    sessionDir = 'D:/WASIRF2024PIV_sessions/';
    matFileDir = 'D:/WASIRF2024PIV_matfiles/';
    
    for runIdx = 1:length(runNumbers)
        runNum = runNumbers(runIdx);
        disp(['Processing run number: ', num2str(runNum)]);

        % Step 2: Load Images
        imageFolder = [baseImageDir 'Run' num2str(runNum)];
        pivlab_import_images(imageFolder);  % Load images into PIVlab
        
        % Step 3: Load calibration parameters
        timeStep_ms = runLog(runNum).PulseDistance_us / 1000;
        pivlab_set_calibration(1227.62, 46.5, timeStep_ms);  % Set calibration

        % Step 4: PIV settings
        pivlab_set_piv_settings();  % Set PIV settings

        % Step 5: Analyze all frames
        pivlab_analyze_all_frames();  % Analyze all frames

        % Step 6: Get and save derived parameters
        pivlab_derive_parameters();  % Derive various parameters

        % Step 7: Save session and MAT file
        sessionFile = [sessionDir 'Run' num2str(runNum) '.mat'];
        matFile = [matFileDir 'Run' num2str(runNum) '.mat'];
        pivlab_save_session(sessionFile);  % Save PIVlab session
        pivlab_export_mat_file(matFile);  % Export MAT file
    end
end


% Helper function to interact with the 'Import images' button and load images
function pivlab_import_images(imageFolder)
    % Locate the 'Import images' button on the left panel
    importButton = findall(0, 'Style', 'pushbutton', 'String', 'Import images');
    
    if isempty(importButton)
        error('PIVlab "Import images" button not found.');
    end
    
    cb = get(importButton, 'Callback');
    %now invoke the callback. hObject will be the button's own handle
    cb(importButton, []);    %uicontrol pushbutton have empty event

    disp('Import images button found! Simulating click...');
    
    % Simulate a mouse click to trigger the button
    buttonPosition = getpixelposition(importButton, true);  % Get button position on the screen
    robot = java.awt.Robot();
    robot.mouseMove(buttonPosition(1) + buttonPosition(3)/2, buttonPosition(2) + buttonPosition(4)/2);
    pause(0.5);
    robot.mousePress(java.awt.event.InputEvent.BUTTON1_MASK);  % Simulate mouse click
    robot.mouseRelease(java.awt.event.InputEvent.BUTTON1_MASK);  % Simulate mouse release
    pause(2);  % Wait for the dialog to open
    
    % Now check if the 'Import images' dialog has opened
    checkImportDialog(imageFolder);  % Call the dialog function directly (no parallelism)
end

% Function to check and interact with the import dialog
function checkImportDialog(imageFolder)
    % Check if the 'Import images' dialog has opened
    importImageDlg = findall(0, 'Type', 'figure', 'Name', 'Import images');
    
    if isempty(importImageDlg)
        error('PIVlab import image dialog not found.');
    else
        disp('Import dialog found, proceeding...');
    end
    
    % Simulate entering the image folder path
    currentFolderTextBox = findall(importImageDlg, 'Style', 'edit');
    set(currentFolderTextBox, 'String', imageFolder);  % Set image folder path
    
    % Simulate selecting all images (Ctrl+A)
    robot = java.awt.Robot();
    robot.keyPress(java.awt.event.KeyEvent.VK_CONTROL);
    robot.keyPress(java.awt.event.KeyEvent.VK_A);
    robot.keyRelease(java.awt.event.KeyEvent.VK_A);
    robot.keyRelease(java.awt.event.KeyEvent.VK_CONTROL);
    
    pause(1);  % Short pause to allow image selection
    
    % Click 'Add' button to add images
    addImagesButton = findall(importImageDlg, 'String', 'Add');
    if isempty(addImagesButton)
        error('Add button not found in the import images dialog.');
    end
    addImagesButton.Callback();  % Simulate button press
    
    % Finally, click 'Import' to complete image loading
    importButton = findall(importImageDlg, 'String', 'Import');
    if isempty(importButton)
        error('Import button not found.');
    end
    importButton.Callback();  % Simulate button press
end

Any help would be greatly appretiated.

[Malay Agarwal]

Hi @jcd,

Could you share the app so that I can try to come up with a way to interact with it?

Answer 1

[Kanishk]

Hi jcd,

I understand that you want to open and operate a MATLAB App from another MATLAB script. This can be done by accessing the components and necessary callback functions of the app.

The components of the App are easily accessible in the external script as they are defined in the public methods. To access the privately defined callback function in the script, we must define publicly accessible methods calling to the callback functions.

To define public methods, go to the Code Browser in Code view of App designer.

methods (Access = public)
    function simulateButtonPush(app)
        app.ButtonPushed(); 
    end
end

We can use these public components and callbacks to operate the app from external script.

app = PIVlab; 

% To simulate a Button Press 
app.simulateButtonPush(); 

% To edit Text Field 
app.EditField.Value = 'Value'; 

To learn more about components in App Designer, please look at the following official MATLAB Documentation.

https://www.mathworks.com/help/releases/R2023a/matlab/creating_guis/choose-components-for-your-app-designer-app.html

Hope this helps!

Thanks