How to determine if segmentation violation is caused by MATLAB function or user-defined function?

I understand its meaningless to post the stack trace of a segmentation violation here to find out the cause. However, I hope to find out how to determine that specific function name that causes the crash.

If the MEX-function belong to MATLAB, i can send the stack trace to MATLAB technical support. If the function is user-defined function, i can focus on the specific function code. Thanks!

Stack Trace: (Partial)
  [  0] 0000000001AC831D        m_interpreter.dll+164637 (inNarginout+003981)
  [  1] 0000000007E01128 distcompdeserialize.mexw64+004392 (mexFunction+000296)
  [  4] 00000000020B7918        m_interpreter.dll+6387992 (inPathNotification::`vftable'+000728)
  [ 15] 0000000002384E83               libmex.dll+020099 (mexRunMexFile+000131)
  [ 16] 0000000002382DBC               libmex.dll+011708 (inSwapMexfileReader+000220)
  [ 17] 0000000002382F84               libmex.dll+012164 (inSwapMexfileReader+000676)
  [ 18] 00000000023F3544         m_dispatcher.dll+341316 (Mfh_file::dispatch_fh+000340)
  [ 19] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)

??? Error using ==> parallel_function at 587 Deserialization threw an exception

Answer 1

[Mark]

The general way to read a stack trace is that each row indicates a function call, and the top one is the deepest/most recent function call before the crash as indicated by the "[0]". From there, it lists the exe/dll/mex file with the problematic function call, in the case of frame "0", it is in "m_interpreter.dll". After that is the function within it being called. Depending on how the exe/dll/mex file was compiled, it may list the function name, or just the function's address (In this case it is just the address of the function). Lastly, it lists the arguments to the function and/or their addresses.

Given your particular stack trace, it looks like the break occurred somewhere in "m_interpreter.dll", but that does not necessarily mean that the problem is in that file. It could very well be a working function that was given invalid inputs, so it is worthwhile to check the other functions/dlls/mex files earlier in the stack trace as well.

[Michael Teo]

I have checked the stack trace but found no dll/mex/function name that belong to user-custom library.

Answer 2

[Walter Roberson]

I am not very familiar with those tracebacks, but it appears that a Mex file called back to Matlab in a parallel function, and that the parallel function had trouble interpreting the stream of data that it got that it needs to turn in to local objects and variables in order to work on the variables on the other core / thread. Does that situation sound familiar in terms of what you are doing with your mex files?

[Michael Teo]

Yes, you are right that error is caused by function within a parfor loop.

As for the deserialization part, i think its auto-handled by MATLAB parallel & distributed processing. The user-custom MEX-functions that i have are all logic computation of native data-type.

In our MATLAB m-file codes, we make use of containers.Map for data store that will be processed by the parfor loop. Do you think containers.Map is the culprit? (just guessing)

[Kaustubha Govind]

If you are able to whittle down your code to just the use of containers.Map (and eliminate all user-written MEX funtions) to reproduce this, it might be worth reporting it to Tech Support.

[Michael Teo]

Kaustubha, i don't think i am able to do because i don't know the specific function that cause the error.

The most difficult part is that it took 40hrs to generate the error in the last run. I do not know how long its going to take to generate the same error in the 2nd run. It may or may not generate any error at all.

Life is tough, programming without debug-gable log is tougher lol.

Answer 3

[Michael Teo]

Just to give additional info, the error is generated after running the parfor loops (same codes) for 40hrs++.

Answer 4

[Michael Teo]

Stack Trace:
  [  0] 0000000001AC831D        m_interpreter.dll+164637 (inNarginout+003981)
  [  1] 0000000007E01128 distcompdeserialize.mexw64+004392 (mexFunction+000296)
  [  4] 00000000020B7918        m_interpreter.dll+6387992 (inPathNotification::`vftable'+000728)
  [ 15] 0000000002384E83               libmex.dll+020099 (mexRunMexFile+000131)
  [ 16] 0000000002382DBC               libmex.dll+011708 (inSwapMexfileReader+000220)
  [ 17] 0000000002382F84               libmex.dll+012164 (inSwapMexfileReader+000676)
  [ 18] 00000000023F3544         m_dispatcher.dll+341316 (Mfh_file::dispatch_fh+000340)
  [ 19] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)
  [ 20] 0000000001B35F4B        m_interpreter.dll+614219 (QueryMLFcnTable_m_interpreter+393131)
  [ 21] 0000000001AF1A5C        m_interpreter.dll+334428 (QueryMLFcnTable_m_interpreter+113340)
  [ 22] 0000000001AF2AB5        m_interpreter.dll+338613 (QueryMLFcnTable_m_interpreter+117525)
  [ 23] 0000000001AF45AD        m_interpreter.dll+345517 (QueryMLFcnTable_m_interpreter+124429)
  [ 24] 0000000001AF4742        m_interpreter.dll+345922 (QueryMLFcnTable_m_interpreter+124834)
  [ 25] 0000000001AF5B79        m_interpreter.dll+351097 (QueryMLFcnTable_m_interpreter+130009)
  [ 26] 0000000001B4ADCD        m_interpreter.dll+699853 (QueryMLFcnTable_m_interpreter+478765)
  [ 27] 0000000001B21EF9        m_interpreter.dll+532217 (QueryMLFcnTable_m_interpreter+311129)
  [ 28] 0000000001B21F25        m_interpreter.dll+532261 (QueryMLFcnTable_m_interpreter+311173)
  [ 29] 00000000023F3544         m_dispatcher.dll+341316 (Mfh_file::dispatch_fh+000340)
  [ 30] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)
  [ 31] 00000000043EDB84               udd_mi.dll+187268 (MErrorException::cleanupMatlabException+040068)
  [ 32] 00000000044034B4               udd_mi.dll+275636 (judiSetDynamicPropertyValue+013124)
  [ 33] 0000000004403A29               udd_mi.dll+277033 (judiSetDynamicPropertyValue+014521)
  [ 34] 0000000002D8D705                 mcos.dll+1431301 (mdMethodSignatureError+000789)
  [ 35] 00000000023A9657         m_dispatcher.dll+038487 (Mfh_MATLAB_fn::dispatch_fh+000279)
  [ 36] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)
  [ 37] 0000000001B35A16        m_interpreter.dll+612886 (QueryMLFcnTable_m_interpreter+391798)
  [ 38] 0000000001B43323        m_interpreter.dll+668451 (QueryMLFcnTable_m_interpreter+447363)
  [ 39] 0000000001B44FC0        m_interpreter.dll+675776 (QueryMLFcnTable_m_interpreter+454688)
  [ 40] 0000000001B476E4        m_interpreter.dll+685796 (QueryMLFcnTable_m_interpreter+464708)
  [ 41] 0000000001ADD16D        m_interpreter.dll+250221 (QueryMLFcnTable_m_interpreter+029133)
  [ 42] 0000000001ADA090        m_interpreter.dll+237712 (QueryMLFcnTable_m_interpreter+016624)
  [ 43] 0000000001AF3777        m_interpreter.dll+341879 (QueryMLFcnTable_m_interpreter+120791)
  [ 44] 0000000001AF45AD        m_interpreter.dll+345517 (QueryMLFcnTable_m_interpreter+124429)
  [ 45] 0000000001AF4742        m_interpreter.dll+345922 (QueryMLFcnTable_m_interpreter+124834)
  [ 46] 0000000001AF5B79        m_interpreter.dll+351097 (QueryMLFcnTable_m_interpreter+130009)
  [ 47] 0000000001B4ADCD        m_interpreter.dll+699853 (QueryMLFcnTable_m_interpreter+478765)
  [ 48] 0000000001B21EF9        m_interpreter.dll+532217 (QueryMLFcnTable_m_interpreter+311129)
  [ 49] 0000000001B21F25        m_interpreter.dll+532261 (QueryMLFcnTable_m_interpreter+311173)
  [ 50] 00000000023F3544         m_dispatcher.dll+341316 (Mfh_file::dispatch_fh+000340)
  [ 51] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)
  [ 52] 0000000001B35F4B        m_interpreter.dll+614219 (QueryMLFcnTable_m_interpreter+393131)
  [ 53] 0000000001AF1A5C        m_interpreter.dll+334428 (QueryMLFcnTable_m_interpreter+113340)
  [ 54] 0000000001AF2AB5        m_interpreter.dll+338613 (QueryMLFcnTable_m_interpreter+117525)
  [ 55] 0000000001AF45AD        m_interpreter.dll+345517 (QueryMLFcnTable_m_interpreter+124429)
  [ 56] 0000000001AF4742        m_interpreter.dll+345922 (QueryMLFcnTable_m_interpreter+124834)
  [ 57] 0000000001AF5B79        m_interpreter.dll+351097 (QueryMLFcnTable_m_interpreter+130009)
  [ 58] 0000000001B4ADCD        m_interpreter.dll+699853 (QueryMLFcnTable_m_interpreter+478765)
  [ 59] 0000000001B21EF9        m_interpreter.dll+532217 (QueryMLFcnTable_m_interpreter+311129)
  [ 60] 0000000001B21F25        m_interpreter.dll+532261 (QueryMLFcnTable_m_interpreter+311173)
  [ 61] 00000000023F3544         m_dispatcher.dll+341316 (Mfh_file::dispatch_fh+000340)
  [ 62] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)
  [ 63] 0000000001B35F4B        m_interpreter.dll+614219 (QueryMLFcnTable_m_interpreter+393131)
  [ 64] 0000000001AF1A5C        m_interpreter.dll+334428 (QueryMLFcnTable_m_interpreter+113340)
  [ 65] 0000000001AF2AB5        m_interpreter.dll+338613 (QueryMLFcnTable_m_interpreter+117525)
  [ 66] 0000000001AF45AD        m_interpreter.dll+345517 (QueryMLFcnTable_m_interpreter+124429)
  [ 67] 0000000001AF4742        m_interpreter.dll+345922 (QueryMLFcnTable_m_interpreter+124834)
  [ 68] 0000000001AF5B79        m_interpreter.dll+351097 (QueryMLFcnTable_m_interpreter+130009)
  [ 69] 0000000001B4ADCD        m_interpreter.dll+699853 (QueryMLFcnTable_m_interpreter+478765)
  [ 70] 0000000001B21EF9        m_interpreter.dll+532217 (QueryMLFcnTable_m_interpreter+311129)
  [ 71] 0000000001B21F25        m_interpreter.dll+532261 (QueryMLFcnTable_m_interpreter+311173)
  [ 72] 00000000023F3544         m_dispatcher.dll+341316 (Mfh_file::dispatch_fh+000340)
  [ 73] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)
  [ 74] 0000000001F6BD16        m_interpreter.dll+5029142 (QueryMLFcnTable_m_interpreter+4808054)
  [ 75] 0000000001F701E8        m_interpreter.dll+5046760 (QueryMLFcnTable_m_interpreter+4825672)
  [ 76] 0000000001F71155        m_interpreter.dll+5050709 (QueryMLFcnTable_m_interpreter+4829621)
  [ 77] 0000000001C5754D        m_interpreter.dll+1799501 (QueryMLFcnTable_m_interpreter+1578413)
  [ 78] 0000000001CC3757        m_interpreter.dll+2242391 (QueryMLFcnTable_m_interpreter+2021303)
  [ 79] 0000000001CC3875        m_interpreter.dll+2242677 (QueryMLFcnTable_m_interpreter+2021589)
  [ 80] 0000000001EBF642        m_interpreter.dll+4322882 (QueryMLFcnTable_m_interpreter+4101794)
  [ 81] 0000000001AF1E37        m_interpreter.dll+335415 (QueryMLFcnTable_m_interpreter+114327)
  [ 82] 0000000001AF45AD        m_interpreter.dll+345517 (QueryMLFcnTable_m_interpreter+124429)
  [ 83] 0000000001AF4742        m_interpreter.dll+345922 (QueryMLFcnTable_m_interpreter+124834)
  [ 84] 0000000001AF5B79        m_interpreter.dll+351097 (QueryMLFcnTable_m_interpreter+130009)
  [ 85] 0000000001B4ADCD        m_interpreter.dll+699853 (QueryMLFcnTable_m_interpreter+478765)
  [ 86] 0000000001B21EF9        m_interpreter.dll+532217 (QueryMLFcnTable_m_interpreter+311129)
  [ 87] 0000000001B21F25        m_interpreter.dll+532261 (QueryMLFcnTable_m_interpreter+311173)
  [ 88] 00000000023F3544         m_dispatcher.dll+341316 (Mfh_file::dispatch_fh+000340)
  [ 89] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)
  [ 90] 0000000001B35F4B        m_interpreter.dll+614219 (QueryMLFcnTable_m_interpreter+393131)
  [ 91] 0000000001AF1A5C        m_interpreter.dll+334428 (QueryMLFcnTable_m_interpreter+113340)
  [ 92] 0000000001AF2AB5        m_interpreter.dll+338613 (QueryMLFcnTable_m_interpreter+117525)
  [ 93] 0000000001AF45AD        m_interpreter.dll+345517 (QueryMLFcnTable_m_interpreter+124429)
  [ 94] 0000000001AF4742        m_interpreter.dll+345922 (QueryMLFcnTable_m_interpreter+124834)
  [ 95] 0000000001AF5B79        m_interpreter.dll+351097 (QueryMLFcnTable_m_interpreter+130009)
  [ 96] 0000000001B4ADCD        m_interpreter.dll+699853 (QueryMLFcnTable_m_interpreter+478765)
  [ 97] 0000000001B21EF9        m_interpreter.dll+532217 (QueryMLFcnTable_m_interpreter+311129)
  [ 98] 0000000001B21F25        m_interpreter.dll+532261 (QueryMLFcnTable_m_interpreter+311173)
  [ 99] 00000000023F3544         m_dispatcher.dll+341316 (Mfh_file::dispatch_fh+000340)
  [100] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)
  [101] 0000000001B35F4B        m_interpreter.dll+614219 (QueryMLFcnTable_m_interpreter+393131)
  [102] 0000000001AF1A5C        m_interpreter.dll+334428 (QueryMLFcnTable_m_interpreter+113340)
  [103] 0000000001AF2AB5        m_interpreter.dll+338613 (QueryMLFcnTable_m_interpreter+117525)
  [104] 0000000001AF45AD        m_interpreter.dll+345517 (QueryMLFcnTable_m_interpreter+124429)
  [105] 0000000001AF4742        m_interpreter.dll+345922 (QueryMLFcnTable_m_interpreter+124834)
  [106] 0000000001AF5B79        m_interpreter.dll+351097 (QueryMLFcnTable_m_interpreter+130009)
  [107] 0000000001B4ADCD        m_interpreter.dll+699853 (QueryMLFcnTable_m_interpreter+478765)
  [108] 0000000001B21EF9        m_interpreter.dll+532217 (QueryMLFcnTable_m_interpreter+311129)
  [109] 0000000001B21F25        m_interpreter.dll+532261 (QueryMLFcnTable_m_interpreter+311173)
  [110] 00000000023F3544         m_dispatcher.dll+341316 (Mfh_file::dispatch_fh+000340)
  [111] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)
  [112] 0000000002D04107                 mcos.dll+868615 (mcos::COSJITInterface::getPropertyInfo+083335)
  [113] 0000000002CEE791                 mcos.dll+780177 (mcos::Enumeration<mcos::COSString>::nextElement+030209)
  [114] 0000000002D0572A                 mcos.dll+874282 (omCallMethod+001514)
  [115] 0000000002D061E5                 mcos.dll+877029 (omCallMethod+004261)
  [116] 0000000002D8DCC9                 mcos.dll+1432777 (mdMethodSignatureError+002265)
  [117] 00000000023A9657         m_dispatcher.dll+038487 (Mfh_MATLAB_fn::dispatch_fh+000279)
  [118] 00000000023A1B65         m_dispatcher.dll+007013 (Mfunction_handle::dispatch+000021)

This error was detected while a MEX-file was running. If the MEX-file
is not an official MathWorks function, please examine its source code
for errors. Please consult the External Interfaces Guide for information
on debugging MEX-files.

If it is an official MathWorks function, please
follow these steps to report this problem to The MathWorks so we
have the best chance of correcting it:

The next time MATLAB is launched under typical usage, a dialog box will
open to help you send the error log to The MathWorks. Alternatively, you
can send an e-mail to [e-mail] with the following file attached:
    C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\matlab_crash_dump.3876

If the problem is reproducible, please submit a Service Request via:
    http://www.mathworks.com/support/contact_us/ts/help_request_1.html

A technical support engineer might contact you with further information.

Thank you for your help. MATLAB may attempt to recover, but even if recovery appears successful,
we recommend that you save your workspace and restart MATLAB as soon as possible.

??? Error using ==> parallel_function at
587
Deserialization threw an exception

Error in ==> backtestFolio at 53
parfor i = 1:aParamRowCount

Error in ==> walkForwardAnalysis at 71
    aFolioRatios = backtestFolio(...

Error in ==> walkForwardOptimization at 29
[aMktSignals, aFolioSignals] =
walkForwardAnalysis(...

Error in ==>
StrategyRunner>StrategyRunner.run at 70
            [aReturns, aRatios,
            aMonthlyDates] = ...

[Kaustubha Govind]

Michael: I think you should report this to Tech Support.