Video preview has growing delay when used in uifigure, but not in normal figure

Hey there,

I am building an app which includes axes to display a preview of my camera (gentl adaptor). I created the axes with uiaxes.

My problem is, that the video is really laggy and the delay grows the longer the preview is running. When I stop the preview, it keeps running until it displays the last recorded frame. The memory usage of matlab is also steadily increasing while it is running.

If I try the example of a custom preview GUI everything runs smoothly. But that one does not create a axis object at all.

I also tried it this way:

figure
ax = axes
im = image(ax, zeros(1542, 2064,3 ));
axis(ax, 'image')
vid = videoinput('gentl')
preview(vid, im)

And this also works fine.

I then tried replacing the uiaxes object in my app with an axes object (same parent), but then it does not show.

Is there a way to not use the uiaxes, because I think that thats the issue?

I also noticed, that the uiaxes do not use as much GPU as the axes, if thats useful.

This is my code (was created with app designer and then ported to normal matlab code in an attempt to fix the lag):

classdef Preview_Window < handle

    % Properties that correspond to app components
    properties (Access = public)
        UIFigure                    matlab.ui.Figure
        Menu                        matlab.ui.container.Menu
        Menu2                       matlab.ui.container.Menu
        GridLayout                  matlab.ui.container.GridLayout
        LeftPanel                   matlab.ui.container.Panel
        GridLayout2                 matlab.ui.container.GridLayout
        ResetCameraInterfaceButton  matlab.ui.control.Button
        PreviewPanel                matlab.ui.container.Panel
        GridLayout4                 matlab.ui.container.GridLayout
        StartButton                 matlab.ui.control.Button
        StopButton                  matlab.ui.control.Button
        RightPanel                  matlab.ui.container.Panel
        UIAxes                      matlab.ui.control.UIAxes
        Timestamp                   matlab.ui.control.Label
    end

    % Properties that correspond to apps with auto-reflow
    properties (Access = private)
        onePanelWidth = 576;
    end

    properties (Access = private)
        feed; % camera feed
        src; %getsource(feed)
        canvas; %image handle to show the preview on
    end  
    methods (Access = private)
    
        function preview_fcn(app, ~, event, himage)
            % callback of preview() 
            % display some additional information

            % Get timestamp for frame.
            tstampstr = event.Timestamp;
            framerate = event.FrameRate;
            %status = event.Status;
            resolution = event.Resolution;
            
            % Set the value of the text label.
            app.Timestamp.Text = strcat(tstampstr, " | " , num2str(framerate), " | ", resolution);
            %app.Status.Text = strcat( "| ", status, " | :", resolution);
            % Display image data.
            himage.CData = event.Data;
        end
        
        function reset_camera(app)
            % setup camera and interface
            app.ResetCameraInterfaceButton.Enable = false;
            
            % preset axes with black picture
            app.canvas = image(app.UIAxes, zeros(1542, 2064, 3) );
            colormap(app.UIAxes, gray);
            axis(app.UIAxes, 'image');
            
            %reset camera interface and try to open camera feed
            imaqreset
            try
                app.feed = videoinput("gentl", 1 , 'Mono8');
            catch
                warndlg("Can't open Camera feed. Fix and reset interface!", "Warning!")
                return
            end
            %Settings for camera
            triggerconfig(app.feed, 'manual');
            app.feed.FramesPerTrigger = 1;
            app.src = getselectedsource(app.feed);
            app.src.ReverseY = 'True';
            app.src.ExposureTime = 60000;
            
            % Setup preview container
            frame = getsnapshot(app.feed);
            vidRes = app.feed.VideoResolution;
            nBands = app.feed.NumberOfBands;
            app.canvas = image(app.UIAxes, zeros(vidRes(2), vidRes(1), nBands));
            app.UIAxes.XLim = [0, double(vidRes(1))];
            app.UIAxes.YLim = [0, double(vidRes(2))];
            pbaspect(app.UIAxes, [vidRes(1),vidRes(2),1]);
            app.canvas.CData = frame;
            
            % activate custom preview callback
            setappdata(app.canvas, 'UpdatePreviewWindowFcn', @app.preview_fcn);
            
            % enable buttons
            app.StartButton.Enable = true;
            app.ResetCameraInterfaceButton.Enable = true;
        end   
    end
    % Callbacks that handle component events
    methods (Access = private)

        % Code that executes after component creation
        function startupFcn(app)
            app.UIAxes.Title.String = '';
            app.UIAxes.XLabel.String = '';
            app.UIAxes.YLabel.String = '';
            app.UIAxes.Visible = 'off';
            app.UIAxes.XTick = [];
            app.UIAxes.YTick = [];
            disableDefaultInteractivity(app.UIAxes)
            app.UIAxes.PlotBoxAspectRatioMode = 'manual';
            app.reset_camera;         
        end

        % Changes arrangement of the app based on UIFigure width
        function updateAppLayout(app, event)
            currentFigureWidth = app.UIFigure.Position(3);
            if(currentFigureWidth <= app.onePanelWidth)
                % Change to a 2x1 grid
                app.GridLayout.RowHeight = {480, 480};
                app.GridLayout.ColumnWidth = {'1x'};
                app.RightPanel.Layout.Row = 2;
                app.RightPanel.Layout.Column = 1;
            else
                % Change to a 1x2 grid
                app.GridLayout.RowHeight = {'1x'};
                app.GridLayout.ColumnWidth = {220, '1x'};
                app.RightPanel.Layout.Row = 1;
                app.RightPanel.Layout.Column = 2;
            end
        end

        % Button pushed function: StartButton
        function StartButtonPushed(app, event)
            app.StopButton.Enable = true;
            app.StartButton.Enable = false;
            
            %app.src.ExposureAuto = 'Continuous';
            
            preview(app.feed, app.canvas);
 
        end

        % Close request function: UIFigure
        function UIFigureCloseRequest(app, event)
            try
                stoppreview(app.feed);
            catch
            end
            try
                delete(app.feed);
            catch  
            end
            delete(app)
        end

        % Button pushed function: StopButton
        function StopButtonPushed(app, event)
            app.StopButton.Enable = false;
            app.StartButton.Enable = true;  
            
            stoppreview(app.feed);   
        end

        % Button pushed function: ResetCameraInterfaceButton
        function ResetCameraInterfaceButtonPushed(app, event)
            app.reset_camera;
        end
    end

    % Component initialization
    methods (Access = private)

        % Create UIFigure and components
        function createComponents(app)

            % Create UIFigure and hide until all components are created
            app.UIFigure = uifigure('Visible', 'off');
            app.UIFigure.AutoResizeChildren = 'off';
            app.UIFigure.Position = [100 100 640 480];
            app.UIFigure.Name = 'MATLAB App';
            app.UIFigure.CloseRequestFcn = @(obj, event)app.UIFigureCloseRequest;
            app.UIFigure.SizeChangedFcn = @(obj, event)app.updateAppLayout;

            % Create Menu
            app.Menu = uimenu(app.UIFigure);
            app.Menu.Text = 'Menu';

            % Create Menu2
            app.Menu2 = uimenu(app.UIFigure);
            app.Menu2.Text = 'Menu2';

            % Create GridLayout
            app.GridLayout = uigridlayout(app.UIFigure);
            app.GridLayout.ColumnWidth = {220, '1x'};
            app.GridLayout.RowHeight = {'1x'};
            app.GridLayout.ColumnSpacing = 0;
            app.GridLayout.RowSpacing = 0;
            app.GridLayout.Padding = [0 0 0 0];
            app.GridLayout.Scrollable = 'on';

            % Create LeftPanel
            app.LeftPanel = uipanel(app.GridLayout);
            app.LeftPanel.Layout.Row = 1;
            app.LeftPanel.Layout.Column = 1;

            % Create GridLayout2
            app.GridLayout2 = uigridlayout(app.LeftPanel);
            app.GridLayout2.RowHeight = {'1x', '1x', '1x', '1x', '1x', '1x', '1x', '1x'};
            app.GridLayout2.RowSpacing = 25;

            % Create ResetCameraInterfaceButton
            app.ResetCameraInterfaceButton = uibutton(app.GridLayout2, 'push');
            app.ResetCameraInterfaceButton.ButtonPushedFcn = @(obj, event)app.ResetCameraInterfaceButtonPushed;
            app.ResetCameraInterfaceButton.Enable = 'off';
            app.ResetCameraInterfaceButton.Layout.Row = 6;
            app.ResetCameraInterfaceButton.Layout.Column = 1;
            app.ResetCameraInterfaceButton.Text = {'Reset Camera'; 'Interface'};

            % Create PreviewPanel
            app.PreviewPanel = uipanel(app.GridLayout2);
            app.PreviewPanel.Title = 'Preview';
            app.PreviewPanel.Layout.Row = [1 3];
            app.PreviewPanel.Layout.Column = 1;

            % Create GridLayout4
            app.GridLayout4 = uigridlayout(app.PreviewPanel);
            app.GridLayout4.ColumnWidth = {'1x'};

            % Create StartButton
            app.StartButton = uibutton(app.GridLayout4, 'push');
            app.StartButton.ButtonPushedFcn = @(obj, event)app.StartButtonPushed;
            app.StartButton.Enable = 'off';
            app.StartButton.Layout.Row = 1;
            app.StartButton.Layout.Column = 1;
            app.StartButton.Text = 'Start';

            % Create StopButton
            app.StopButton = uibutton(app.GridLayout4, 'push');
            app.StopButton.ButtonPushedFcn = @(obj, event)app.StopButtonPushed;
            app.StopButton.Enable = 'off';
            app.StopButton.Layout.Row = 2;
            app.StopButton.Layout.Column = 1;
            app.StopButton.Text = 'Stop';

            % Create RightPanel
            app.RightPanel = uipanel(app.GridLayout);
            app.RightPanel.Layout.Row = 1;
            app.RightPanel.Layout.Column = 2;

            % Create UIAxes
            app.UIAxes = uiaxes(app.RightPanel);
            title(app.UIAxes, 'Title')
            xlabel(app.UIAxes, 'X')
            ylabel(app.UIAxes, 'Y')
            app.UIAxes.PlotBoxAspectRatio = [1 1.02770083102493 1];
            app.UIAxes.Box = 'on';
            app.UIAxes.Position = [6 28 408 425];

            % Create Timestamp
            app.Timestamp = uilabel(app.RightPanel);
            app.Timestamp.Position = [6 1 408 22];
            app.Timestamp.Text = '';

            % Show the figure after all components are created
            app.UIFigure.Visible = 'on';
        end
    end

    % App creation and deletion
    methods (Access = public)

        % Construct app
        function app = Preview_Window

            % Create UIFigure and components
            createComponents(app)

            % Execute the startup function
            app.startupFcn

            if nargout == 0
                clear app
            end
        end

        % Code that executes before app deletion
        function delete(app)

            % Delete UIFigure when app is deleted
            delete(app.UIFigure)
        end
    end
end

Thanks in advance for any help!

[Raphael Kriegl]

After further trial and error, I could replace the uiaxes with axes, but that did not fix the issue with the delay.

Answer 1

[Raphael Kriegl]

I got an answer from the MATLAB support team:

After collaborating with my colleagues I figured that this is a known limitation with the "uiaxes" performance in terms of rendering the images. Currently, "uiaxes"/"uifigure" (javascript based) does not support high frame rates like the traditional MATLAB figure window (java based) and the frames passed to the "uiaxes" are queued for rendering, this is causing the memory increase they noticed. For the same reason, it will cause a lag which worsens with larger images and high acquisition frame rates if the acquisition is running for a long duration .

Until the performance bottleneck in the "uiaxes" is addressed, there is little we can do to help you, if you goal is to have the live preview running in the app.

Having said that, for the time being, I would suggest you to open the preview in traditional MATLAB figure window outside of the app and i believe you have already tried this.

Another thing to experiment is to turn the imaqmex's "slowpreview" flag which currently sets a max limit of 10 frames for previewing. However, with the "uiaxes" rendering performance even 10 FPS might be high. Following is the code to enable slow preview.

>> imaqreset

>> imaqmex('feature', '-slowpreview', 'true')

>> vid = videoinput(...)Please note that this is an undocumented feature and it could change and/or be removed in the future.

Additionally, lowering the resolution and/or ROI can help as this decreases the load on the "uifigure".

Answer 2

[Evan Bates]

Raphael, thank you for posting that response from Matworks. That is a huge help. I just ran into this issue as well. I’m glad the problem is not on my end. They should add functionality in the preview function to set the playback framerate.

I found a workaround to this issue. I made a UpdatePreivewWindowFcn and inside the callback function I added a pause(1/60). The preview function will dump frames if the UpdatePreivewWindowFcn has not finished executing, therefore slowing the video to the desired framerate (frame rate is set by the time in the pause).

Hope this helps

[Luisa Cruz]

Hi Evan, can you please give more details about UpdatePreviewWindowsFnc ?