No sound when playing movie in Matlab2020b

Hi :)

I'm trying to play movies in Matlab with sound for an experiment. I've managed to display the movies (once with PsychToolbox and once with ComputerVision Toolbox) - but the sound is missing in both. I have a hunch that this is somehow related to my sound settings, but haven't been able to figure it out. Some facts:

- using Matlab2020b

- operating system: Windows 10

- tried out different movie formats, such as .mp4, .mov etc. - they all run, but without sound

- added PsychPortAudio in the PsychToolbox version, but that didn't help

Ideally, I would like to solve this with PsychToolbox to keep things uniform (the rest of the script is solely PsychToolbox).

Thank you in advance for your help! (See code below)

The ComputerVIsion Toolbox version

%dispMoviesMatlab - ComputerVision Toolbox

path = 'C:\Users\maxab\OneDrive\Desktop\Masters\Experiment\Movies\Chosen';
moviefile = '27Eng.mp4';%possibleMoviesCell{ii};

%CHANGE FILE PATH FOR WHEREVER THE VIDEOS ARE ON EXPERIMENTAL
%COMPUTER
moviename = strcat(path,'\','27Eng.mp4');
videoFReader = vision.VideoFileReader(moviename, 'PlayCount',1,'AudioOutputPort', true);
videoPlayer = vision.VideoPlayer;
while ~isDone(videoFReader)
  videoFrame = step(videoFReader);%videoFReader();
  step(videoPlayer,videoFrame);
end

And the PsychToolbox version

close all
clear 
clc

%Basic settings for Psychtoolbox
PsychDefaultSetup(2);
Screen('Preference', 'VisualDebugLevel',1); %tp avoid showin the psychtoolbox logo at the first screen 
Screen('Preference', 'SkipSyncTests', 2); %skips the synchroniztion test to bypass errors

%Set basic preferences
path = 'C:\Users\maxab\OneDrive\Desktop\Masters\Experiment\Movies\Chosen';
bgClr = 127;
rectColor = 127;
fixClrs = [0 255];
scr = max(Screen('Screens'));

%Display movie-loop

[wpnt,winRect] = PsychImaging('OpenWindow', scr, bgClr, [], [], [], [], 4);
hz=Screen('NominalFrameRate', wpnt);
device = [];
%InitializePsychSound
pahandle = PsychPortAudio('Open', device)

%get the file to play
moviefile = '27Eng.mp4';%possibleMoviesCell{ii};

%CHANGE FILE PATH FOR WHEREVER THE VIDEOS ARE ON EXPERIMENTAL
%COMPUTER
moviename = strcat(path,'\','27Eng.mp4');
movie = Screen('OpenMovie', wpnt, moviename);
Screen('PlayMovie', movie, 1);

% First draw a fixation point
oldTextSize=Screen('TextSize', wpnt ,100);
DrawFormattedText(wpnt, '+', 'center', 'center', 0);
Screen('Flip',wpnt);
pause(2)
startT=GetSecs;

%play the actual movie
while ~KbCheck
    % Wait for next movie frame, retrieve texture handle to it
     tex = Screen('GetMovieImage', wpnt, movie);
    % Valid texture returned? A negative value means end of movie reached:
    if tex<=0
        % We're done, break out of loop:
        break;
    end
    % Draw the new texture immediately to screen:
    Screen('DrawTexture', wpnt, tex);
    % Update display:
    Screen('Flip', wpnt);
    % Release texture:
    Screen('Close', tex);
end
% Stop playback:
Screen('PlayMovie', movie, 0);
endT=GetSecs;
% Close movie:
Screen('CloseMovie', movie);
Screen('FillRect', wpnt, rectColor);
Screen('Flip', wpnt);
%EThndl.sendMessage(sprintf('end - %s',moviefile),endT);
PsychPortAudio('Stop', device)
        % stop recording

Answer 1

[Samay Sagar]

The Computer Vision Toolbox in MATLAB primarily focuses on computer vision tasks and does not directly support synchronized audio playback with video. However, you can try to synchronize audio and video using MATLAB's built-in functions in combination with the Computer Vision Toolbox for video display.

Here's how you can play video with sound using MATLAB's built-in functions and the Computer Vision Toolbox:

1. Read the audio data from the video file using “audioread”
2. Play the audio using MATLAB's “audioplayer”.
3. Use the “vision.VideoFileReader“ to read video frames.
4. Display the video frames using the ”vision.VideoPlayer”.

videoFile = 'path_to_your_video_file.mp4'; % Replace with your video file path

% Read audio from the video file
[audio, Fs] = audioread(videoFile);

% Create a player for audio
player = audioplayer(audio, Fs);
% Create a System object for reading video
videoFReader = vision.VideoFileReader(videoFile, 'PlayCount', 1);

% Create a video player object for displaying video frames
videoPlayer = vision.VideoPlayer;

% Start audio playback
play(player);

% Playback loop
while ~isDone(videoFReader)
    [videoFrame, audioFrame] = step(videoFReader);
    step(videoPlayer, videoFrame);
    if ~isOpen(videoPlayer)
        break;
    end
end

% Release the video player and file reader
release(videoPlayer);
release(videoFReader);

% Stop audio playback
stop(player);

You can checkout the following question to find out more about using PsychToolbox for video playback with sound.

https://www.mathworks.com/matlabcentral/answers/251504

Read more about “audioplayer” and “VideoPlayer” here:

https://www.mathworks.com/help/releases/R2020b/matlab/ref/audioplayer.htmlhttps://www.mathworks.com/help/releases/R2020b/vision/ref/vision.videofilereader-system-object.html

https://www.mathworks.com/help/releases/R2020b/vision/ref/vision.videoplayer-system-object.html

Answer 2

[Walter Roberson]

%dispMoviesMatlab - ComputerVision Toolbox

audiopath = 'C:\Users\maxab\OneDrive\Desktop\Masters\Experiment\Movies\Chosen';
moviefile = '27Eng.mp4';%possibleMoviesCell{ii};

moviename = fullfile(audiopath, moviefile);
[~, Fs] = audioread(moviename, [1 1]);  %probe the sampling frequency

videoFReader = vision.VideoFileReader(moviename, 'PlayCount', 1, 'AudioOutputPort', true);
videoPlayer = vision.VideoPlayer;
audioWriter = audioDeviceWriter(Fs);

while ~isDone(videoFReader)
  [videoFrame, audioframe] = step(videoFReader);
  step(videoPlayer, videoFrame);
  step(audioWriter, audioframe);
end

%add in appropriate code to close the devices