Write Assessments for Function-Type Learner Solutions

Assessments for testing learner solutions to a function-type problem are more complex than those you write for script-type problems. To aid in assessment writing, MATLAB^® Grader™ provides built-in functions for assessing variable equality and the presence or absence of specific keywords and functions.

The following concepts and procedures can help in writing assessments:

MATLAB Grader Built-In Functions—Provided functions to help assess variable equality and the presence or absence of specific functions and keywords.
The Execution Model—Explains how assessments are run and assessment results are marked.
Call Learner Solution in Assessments—Describes how you call the learner solution so that you can assess their code.
Use Reference Solution to Assess Learner Solution—Explains how you write your own code that tests the learner solution.

You can examine some examples of assessments in Example: Assessments for the "Write a function to calculate the normalized sinc" Coding Problem.

Read about factors that affect the assessment of learner solutions in Test Learner Solutions.

MATLAB Grader Built-In Functions

For function-type solutions, you can use the built-in functions that check for variable equality and keyword or function presence:

assessVariableEqual — Check whether a variable in the learner solution equals a specified value within a tolerance. Tolerance is for numeric data types only.
assessFunctionPresence — Check for the presence of specific functions or keywords in the learner solution.
This option is also available as a single test: from the Test Type menu, select Function or Keyword is Present and provide the specified function names or keywords and any additional feedback that you want to show for incorrect solutions.
assessFunctionAbsence — Check that certain functions or keywords are not present in the learner solution.
This option is also available as a single test: from the Test Type menu, select Function or Keyword is Absent and provide the specified function name or keyword and any additional feedback that you want to show for incorrect solutions.

If you prefer to write your own assessment code, you can also use the built-in functions in a MATLAB Code test type to evaluate submissions and provide any additional feedback that you want to show for incorrect solutions.

Execution Model

The execution model explains how assessments are run and assessment results are marked in MATLAB Grader.

Each assessment that you write for a function-type solution typically includes a call to the learner solution. You can provide inputs to the function and evaluate any returned values. You can also call the reference solution to compare its output with the learner solution output.
Each assessment runs sequentially and independently of the other assessments. If one assessment fails, the subsequent assessments still run.
Variables created in one assessment are not available in the next one. Define all the required variables in each assessment.
An assessment is marked correct if it does not return an error. If there are errors, the assessment is marked incorrect.
If the assessment is a pretest, the learner can view the assessment code by clicking the arrow to the left of the test name, regardless of whether the solution passed or failed the assessment.

Note

The default error message generated by assessVariableEqual includes the name of the variable tested. For function-type problems, this variable is created by the instructor in the assessment script and not by the learner. Use meaningful variable names the learner can recognize, such as an output variable defined in the function declaration.

Each assessment for a given coding problem impacts the execution time for running the solution and the assessments. For more details, see Time Limits for Submissions.

Call Learner Solution in Assessments

Function-type problems are evaluated by calling the learner solution with specific inputs and testing the output. This section uses code examples from My first FUNCTION problem, which you can find on your MATLAB Grader home page in the MathWorks^® Collections under Getting Started with MATLAB Grader.

In the example My first FUNCTION problem, the learner writes a function that converts temperature in Fahrenheit to Celsius. The assessment first defines a positive temperature, calls the learner function with that input, and captures the output in a new variable.

The following assessment uses MATLAB code to test if the value the learner supplied is correct for positive temperatures.

% Create test input
temp = 78;

% Run student solution.
tempC = tempF2C(temp);

You can then add additional code to this assessment that tests if the value of the output variable is correct.

Use Reference Solution to Assess Learner Solution

To compare the learner solution to the reference solution, call the reference function with the same inputs.

To call the reference solution, use the syntax reference.myFunction. Replace myFunction with the name of the function you created in the reference solution.

In the example from My first FUNCTION problem, the reference solution is called with the same input, and the output is captured in a new variable. Finally, assessVAriableEqual is used to compare the output of the learner solution to the output of the reference solution.

Reference Solution

function tempC = tempF2C(tempF)
    tempC = (tempF-32)/1.8;
end

Assessment

% Run reference solution.
refT = reference.tempF2C(temp); 

% Compare.
assessVariableEqual('tempC', refT);

Example: Assessments for the "Write a function to calculate the normalized sinc" Coding Problem

In the example Write a function to calculate the normalized sinc from the MathWorks Collections, learners are instructed to write a function that returns the sinc of the input values without using the built-in sinc function from MATLAB.

Assessment 1. Test If the Equation for `sinc` Has Been Implemented Correctly

The first assessment checks if the equation for sinc has been implemented correctly. If the learner used the built-in sinc function in their solution, the instructor adds the following feedback to the default error message: "Implement the sinc function yourself. Do not use the built-in function."

% Create a random input. 
% Make sure not to have pi, 2pi, 3pi, etc where sin(pi*x) = 0
x = 0.25*randi([1 3]);

% Run learner solution.
y = normsinc(x);

% Run reference solution
yReference = reference.normsinc(x); 

% Compare.
assessVariableEqual('y', yReference);
assessFunctionAbsence('sinc', 'FileName', 'normsinc.m', 'Feedback',...
    'Implement the sinc function yourself. Do not use the built-in function.')

This assessment then displays the following additional feedback if the learner solution fails the assessment:

Double check the equation for sinc. In particular, pay attention to the order of operations and need for parenthesis.

Test If 0 Is Handled Correctly

The second assessment checks if the learner function returns the correct output when the input is 0.

x = 0
% Run learner solution.
y = normsinc(x); 

% Run reference solution.
yReference = reference.normsinc(x); 

% Compare.
assessVariableEqual('y', yReference);

If the learner solution fails this assessment, the instructor has provided this additional feedback:

Test your function using 0 as input, the result should be 1. Try using an if-else statement or logical indexing to achieve this.

Test If Solution Works with Matrices

The third assessment checks if the learner function works with matrices.

% In this test we'll create a random non-square matrix
% Use randi to set the dimensions
dim1 = randi([2 5]);
dim2 = randi([6 9]);
% Create a random matrix with values between [-3 3]
x = 6*rand(dim1,dim2) - 3;
% Add a 0 to the x to see that 0 students still meet requirement with a matrix
x(3) = 0;
 
% Run learner solution.
y = normsinc(x);
 
% Run reference solution
yReference = reference.normsinc(x); 
 
% Compare.
assessVariableEqual('y', yReference);

If the learner solution is incorrect, the instructor provides the following additional feedback:

What does your error message say:
If you receive an error about the wrong size, make sure to use the element-wise operator ./ in your calculation. 
If you receive a wrong value error, make sure any element of an input matrix that is 0 returns 1 for the corresponding element in your output.

Try the Example Function Problem

The first example used in this topic is from My first FUNCTION problem, which you can find in the MathWorks Collections under Getting Started with MATLAB Grader.

The second example used in this topic is from Write a function to calculate the normalized sinc, which you can find in the MathWorks Collections under Getting Started with MATLAB Grader.

To try solving the problem as a learner would, open the problem and then click Learner Preview. Enter your code in the solution box and then click Submit to run the assessments (no submission is actually recorded). Try both correct and incorrect code to see the feedback you get.