Grader says answer Verified in Reference Solution is wrong when submitted by a Learner

The Grader is saying an answer that is Verified by the Reference Solution is wrong when submitted in a Learner Code.

Reference Solution:

eqns = [Mass_bullet*V_i == (Mass_bullet + Mass_block)*v, (Mass_bullet + Mass_block)*g*d*mu == 0.5*(Mass_bullet + Mass_block)*v^2];

Solving for v and d.

Only way to get Grader to say the Learner's value for d is correct is to move mu to front of second expression:

eqns = [Mass_bullet*V_i == (Mass_bullet + Mass_block)*v, mu*(Mass_bullet + Mass_block)*g*d == 0.5*(Mass_bullet + Mass_block)*v^2];

If the Learner enters the code as in the Reference Solution, it produces exactly the same value for d (to all shown sig figs), but Grader says it's incorrect.

How can I get Grader to confirm the correct d from the Learner without having the Learner type that line of code in a specific order?

[Cris LaPierre]

Can you share your complete reference solution? Or at least something similar if you don't want students to come across the solution here.

Answer 1

[Cris LaPierre]

From what I can see, the issue is that you are comparing two symbolic variables. Symbolic variables do not have a concept of tolerance, so the values have to be exactly equal in order to pass the assessment. Changing the order of operations produces a slightly different (Δ= 8.8818e-16) value for Distance.

In order to incorporate tolderance, the final answer must be numeric. My suggestion would be to either have the students convert the symbolic values to doubles

Distance = double(Distance)
Velocity = double(Velocity)

or you can perform the conversion in the assessment test (select MATLAB Code as the type). Adding the code below as a new MATLAB Code assessment test to the problem in its current state works (converts the vpa solution to double), or you could have the students stop at the solve step. I've tested the assessment code for a variety of permutations of mu, g, and d). It worked for all of them.

% Get reference solution
refD = double(referenceVariables.Distance);
Distance = double(Distance);

% Compare with learner solution. 
assessVariableEqual('Distance', refD);

Note that I use the same variable name (Distance) because, in the scenario where there is an issue, the variable name is included in the default error message. Using a different variable name the students don't see in their code could be confusing.

Look to this doc page if you want to change the default tolerance value.

I wanted to add a final comment on Validate Reference Solution. This test just makes sure the reference solution can pass the assessment tests. Validating, then, does not mean the reference solution is correct, the assessment tests are robust, etc. Please do still exercise the normal caution you use when creating any problem for your students.

[William Stuckey]

That worked! I was thrown because the Distance outcome run either way agreed to 13 sig figs and the default tolerance says "+/- 0.0001 absolute". I didn't realize the default tolerance is different for different types of variables. There is no mention of that at the bottom of the Test block.

[Cris LaPierre]

I'll pass that on. I can see how this could be misleading in this scenario.

Answer 2

[William Stuckey]

syms h v 
Mass_bullet = 0.006;
Mass_block = 1;
V_i = 400;
g = 9.81;
eqns = [Mass_bullet*V_i == (Mass_bullet + Mass_block)*v, (Mass_bullet + Mass_block)*g*h == 0.5*(Mass_bullet + Mass_block)*v^2];
vars = [h v];
[Height, Velocity] = solve(eqns, vars);
Height = vpa(Height);
Velocity = vpa(Velocity);
syms d v 
Mass_bullet = 0.012;
Mass_block = 1.5;
V_i = 615;
g = 9.81;
mu = 0.220;
eqns = [Mass_bullet*V_i == (Mass_bullet + Mass_block)*v, mu*(Mass_bullet + Mass_block)*g*d == 0.5*(Mass_bullet + Mass_block)*v^2];
vars = [d v];
[Distance, Velocity] = solve(eqns, vars);
Distance = vpa(Distance)
Velocity = vpa(Velocity)

[William Stuckey]

Oops, changing this line

eqns = [Mass_bullet*V_i == (Mass_bullet + Mass_block)*v, mu*(Mass_bullet + Mass_block)*g*d == 0.5*(Mass_bullet + Mass_block)*v^2];

to:

eqns = [Mass_bullet*V_i == (Mass_bullet + Mass_block)*v, (Mass_bullet + Mass_block)*g*d*mu == 0.5*(Mass_bullet + Mass_block)*v^2];

still doesn't get the Learner Grade for d to be correct (actually the check is for Distance).

[Cris LaPierre]

Which variables are you assessing? Are you using a MATLAB code assessment or just the default variable comparison?

[William Stuckey]

I'm having MATLAB Grader check Distance and Velocity. Regardless of which way I type the Reference Solution, the Grader says Distance is wrong if mu is typed at the end of the line by the Learner.

[William Stuckey]

I first tried Variable Equals Reference Solution checking Distance, then I changed it to

% Get reference solution for x.

xReference = referenceVariables.Distance;

% Compare with learner solution.

assessVariableEqual('Distance', xReference,'AbsoluteTolerance',.001);

Neither way gets Grader to confirm Learner's Distance value when mu is typed this way:

eqns = [Mass_bullet*V_i == (Mass_bullet + Mass_block)*v, (Mass_bullet + Mass_block)*g*d*mu == 0.5*(Mass_bullet + Mass_block)*v^2];

Instead of this way:

qns = [Mass_bullet*V_i == (Mass_bullet + Mass_block)*v, mu*(Mass_bullet + Mass_block)*g*d == 0.5*(Mass_bullet + Mass_block)*v^2];

even if the Reference Solution is typed this way:

eqns = [Mass_bullet*V_i == (Mass_bullet + Mass_block)*v, (Mass_bullet + Mass_block)*g*d*mu == 0.5*(Mass_bullet + Mass_block)*v^2];