Matlab Grader & LMS Viewing Student Solutions

This is a different issue than a recent post of mine regarding downloading student solutions.

When I view Learner Solutions in Canvas+Grader there's no identification of which student submitted which solution. In other words it's not at all clear to me how (if?) I can look at the submission for a particular student of my choosing. The "Selected Learner" tab within the page lets me select a student but apparently not by student name.

So - how can I, for a given student, look at (for example) the most recent code that the student has submitted? If a student mails me and asks me to look at the code they submitted, how can I do this basic thing?

Answer 1

[Cris LaPierre]

The LMS sends us anonymous ids that cannot be matched back to a student. Since the id is meaningless, it is not displayed.

The compromise solution we could implement for LMS users was a mechanism for students to voluntarily identify themselves by sharing a solution id with their instructor. The instructor can look for this specific solution, and therefore see what the student has done. This is explained here.

[Justin Wyss-Gallifent]

This I understand - except clearly when the grade is calculated by Matlab that grade is sent back to the LMS attached to the anonymous ID. The question is then - why isn't the code sent back with it? This way the LMS would have access to the code as well as the grade and the LMS itself could show the code corresponding to the grade.

[Cris LaPierre]

The LMS is the one that generates and sends us the code. However, as best I have been able to tell, the LMS does not provide a look up table to manually match these anonymous ids up with actual students. In the end, it was a design decision made that took into consideration these things.

[Justin Wyss-Gallifent]

No, I'm not suggesting that you (Matlab) should have such a lookup table but rather that given that Matlab Grader definitely sends the grade back to the LMS (no doubt attached to the anonymous ID) I'm rather suprised that the code doesn't get back with it. Matlab wouldn't need a lookup table for this.

That is, right now no doubt Matlab Grader currently sends something back like this:

{'id':'jslfsdlfjsdkjsldjfs45454','grade':9.5}

I'm curious why it doesn't send something back like this:

{'id':'jslfsdlfjsdkjsldjfs45454','grade':9.5,'codewhichgeneratedthisgrade':XXXXXXX}

But I guess this would be dependent on the LMS being able to handle the result. Just seems odd from an educational perspective.

[Brandon Armstrong]

MATLAB Grader follows the IMS Global 1.1 LTI standards. If code was sent back in the response, it would not be captured or handled by the LMS since that is not part of the standard.

[Justin Wyss-Gallifent]

I sincerely appreciate knowing this but you have to admit, it's pretty messed up from an educational standpoint.

ps. I read the IMS Global 1.1 LTI standards. It was clearly written by people who have no idea how teaching works. I say that as a software consultant as well as an educator. <Shrug/>

[Jeff Alderson]

We do have an LTI-supported method of passing the learner solution in a custom field/extension. Since LTI was written to be agnostic of the type of tool being connected, the result of a programming-type assessment is something that is left for the tool provider to implement. In this case, the LMS would need to be modified to inspect the LTI response to the basic outcoems service and look for a custom payload. This is possible for you if your LMS is Moodle, or otherwise open source, and you look into extending the Moodle plugin for the LTI basic outcomes service. If you use Moodle as your LMS, and this is something that you would like to pursue, please reach out to our account representative.

The long term solution here is using Caliper analytics and using the assessment event message framework. We are planning to support the IMS Caliper analytics message framework to handle passing learner events, including code submissions, back to the LMS. Every submission would generate a grade event that a listener on the consumer side could listen for. Since most of the LMS platforms now support Caliper and have their own event stores, we have a standard location where we would publish our grading events (and metadata/student solutions) to. We are exploring support for this, participating in the Caliper working groups developing the standards, and will pursue certification for our solution when it is ready for customers. If you would like to be included in testing of this feature, please reach out to your MathWorks account representative.

[Justin Wyss-Gallifent]

I'm assuming something off-the-shelf like Canvas is not going to inspect the LTI response in such a customizable way? I can nudge our people if that might help.

[Jeff Alderson]

Canvas may have custom API access available to you that we are not familiar with. I would initiate a technical support request with MathWorks and coordinate it with your local LMS administrator.

[Justin Wyss-Gallifent]

Jeff - our institution (our LMS administrator) contacted Matlab who then told us that there is no such customizable field.

[Justin Wyss-Gallifent]

The issue here that that I don't want to reveal to the students that I cannot actually see their solutions, since once they know that they're much more likely to cheat. They need to know (or at least think) that I can see everything they submit.

Fact of the matter is that the LMS/Grader implementation is crippled to the point of being effectively useless. I'm not blaming Matlab or Canvas, just speaking the truth from a teaching perspective.

I met via zoom with two Matlab reps yesterday and as per their suggestion I'm going to move away from the LMS integration and to the online version.

[Marwan Bikdash]

It is very important to be able to see an individual student's solutions for many reasons including the prevention of cheating.

Can Mathworks include a signature field in the learner solution. For instance, the student has to provide their email or name used to validate their log in

[Cris LaPierre]

The best place to submit suggestions and enhancement requests is this page:

https://servicerequest.mathworks.com/mysr/cp_case_new1?cc=us

[Rik]

Regarding the suggestion of Cris to have the students voluntarilly identify: you could have them use their student ID to set the random seed. That way you even force them to write general code, instead of code that only works on a single example.

[Cris LaPierre]

This approach won't work in MATLAB Grader, as the reference solution does not have access to the student number. However, for script type problems, the reference workspace and learner workspace share the same seed, so you can already use functions that generate random numbers to create variables for learners to use in their solution without having to set the seed manually.