mdf file replay error

[arohami]

i'm trying to load and replay a mdf file (xcp) with tsmaster. i can load it without any problem (in Application>calibration manager> devices > measurements) but when i try to replay it, i get this error:

14:13:35.303 [12296] 2023.6.3.898 : Error: ECU_LIST not found in mat file
14:13:35.303 [12296] 2023.6.3.898 Error: missing or invalid: ECU_LIST
14:13:37.510 [11072] 2023.6.3.898 : Error: MeasurementStartTimeStr not found in mat file
14:13:37.518 [11072] 2023.6.3.898 : Error: MeasurementStopTimeStr not found in mat file
14:13:37.535 [11072] 2023.6.3.898 : Error: MeasurementStopTime not found in mat file

I also added a2l file under ECU > database, and added all the variables to "measurements list" to make sure i'm not missing anything (although i highly doubt this step is even relevant!). and the errors still persist. can you perhaps guide me through this? is there a doc or video that could help me with this?

note: I have already seen this video and it didn't help to overcome the issue. specially because i have another project with different mdf file and a2l that works without any problem.

any help?
TNX

[freshhope]

the only restriction is that Mdf file should be created by TSMaster calibration module, then it can be replayed in TSMaster

[arohami]

Thanks for the fast response as always :)
may i ask why is there such a restriction? does TSMaster modifies the headers or data structure in a way that it can replay it later?
and isn't there a way to read/play mdf or mf4 files that haven't been recorded by TSMaster calibration module?

[freshhope]

You are welcome. The restriction is because this mdf file is exported by mat file, so it should follow mat file structure. Two years ago I planned to create a converter to make real mdf file but due to lack of time, this converter is not built. So I recommend two solutions:
[1] if you want to manipulate TSMaster's mdf file, you can discover this file structure by referencing source code of converter_mat_mdf.py and converter_mdf_mat.py in directory "TSMaster\bin\Data\Components\Calibration", and write your own code for converter
[2] Direct use asammdfgui.exe which can be built in "https://github.com/danielhrisca/asammdf", this is a great tool for you to view mdf files !

[arohami]

ok, thanks. that was enlightening.
I'm not trying to modify the mdf file itself, but to actually load and use mdf files in TSMaster to be able to use other great features of TSMaster as well.

I planned to create a converter to make real mdf file

i'm sorry. my knowledge is really limited. what do you mean by real mdf file? does that mean mdf files are actually a very customized data format in nature? and there are not standards to dictates its exact expected data structure?

asammdfgui.exe

actually i was using this tool to visualize data. but i thought it would be much greater if i could find a way to do the same thing in TSMaster so i can use one tool to work with data as much as possible :)

[freshhope]

Hi, real mdf file means TSMaster can export and import standard mdf files generated by other tools such as CANape and INCA. This feature is to be developed, it is not ready yet.

In order to replay successfully in current version of TSMaster, you need to load a correct A2L database in calibration module, so as to generate all system variables as signals defined in your A2L file. Then you need to write a converter to convert your mdf file to mat file, I think it is easier, you can reference to source code "converter_mdf_mat.py", this code reads a mdf file and then creates a mat file. You can also reference to application note "TSMaster\bin\Help\Application Notes\AN0003\AN0003.pdf", this doc describes mat file format, but it is in Chinese, sorry for that.

[arohami]

i will try that as soon as find some time and report the results here
thanks

[arohami]

@freshhope
i have played a little bit more with different files and it turns out that the problem i have with MDF files is perhaps not related to the TSMaster's converter (or i have 2 separate problems!)
I tried to load some mf4 files like i had done before. these files were newly recorded data. when i tried to load them in TSMaster i had the same error as when i tried to load MDF files:

11:13:58.500 [4952] 2023.6.3.898 : Error: ECU_LIST not found in mat file
11:13:58.500 [4952] 2023.6.3.898 Error: missing or invalid: ECU_LIST
11:14:48.118 [4560] 2023.6.3.898 : Error: MeasurementStartTimeStr not found in mat file
11:14:48.130 [4560] 2023.6.3.898 : Error: MeasurementStopTimeStr not found in mat file
11:14:48.155 [4560] 2023.6.3.898 : Error: MeasurementStopTime not found in mat file

and as i recall, you said that the data should be recorded using TSMaster! But i know for sure now that this mf4 and those previous mdf files that i said throw me an error, are recorded using CANAPE. although all of them can be played without any problem in asammdf gui.
so the questions is:

1. does failing to load both newly recorded mf4 and mdf files mean that TSMaster can't load neither of them if they haven't been recorded using TSMaster?

2. also (it's still unclear to me) why exactly TSMaster needs A2L files (as you mentioned) to properly load mdf/mf4 files? i checked multiple times and i could actually load/play the older mf4 files (that maybe have been recorded using TSMaster!) without loading any other file (including a2l) or doing any configuration in a completely blank project.

i will try other ways to convert mdf/mf4 files directly to .mat files outside of TSMaster. I'll report the results when i have done it.

[freshhope]

1. yes
2. because currently TSMaster needs A2L for data visualization

[arohami]

and one more thing that i've done to resolve the issue:
i used asammdfgui to generate .mat file from mdf/mf4 to load in TSMaster. i have two group of files (mdf/mf4), group1 that are currently loading properly in TSMaster and group 2 that give me errors. i generated .mat files for both groups (with the help of asammdfgui) and tried to load them in TSMaster after.
i noticed that the size of generated .mat files are different from those TSMaster generates for group1 (?). beside that, both groups are throwing errors in TSMaster when i try to load them!

i guess that means TSMaster uses different method or specific configuration to generate .mat files (because of difference in size) and it could be the main issue here?

[arohami]

i couldn't install alpha or beta version!
there's an error for both, which i don't understand and unfortunately can't copy and past the text either to get help from translator apps!

[freshhope]

This means TSMaster is running in background, please use task manager to kill process "TSMaster.exe" and then start upgrade.

[arohami]

oh! so that's what it says
make sense :)
unfortunately i'm having the same problem with the current alpha (V2023_7.18.929) and beta (V2023_7.11.923) versions :(

[freshhope]

Please wait for some time, I will reply here when this new feature is added

[arohami]

yeah. sure :)

[arohami]

apart from the issues i have about not being able to play back mdf/mf4 files of other softwares, i'm facing a new problem.
i have an old mf4 file and its' corresponding A2L file with actually works fine in TSMaster: the .mat file is created without any problem and i can play it back and plot signals. but somehow, i can't access it in mini-program code. i'm not sure if actually i'm having a correct idea about what i'm doing, but i see mf4 files as a log and i try to access the signal like:

// in step function
s64 v;
app.get_system_var_int64("AccDisplayObj.ACCEL_REQUEST_FUZZY", &v);
if (v != 0.00){
   printf("%f", v);
}

i run the code and replayed mf4 file. but, the output was always 0. i copied this piece of code right from TSMaster > simulation > system variables > right-click on variable > copy read code.
to make sure the type doesn't mismatch, i tried %d, %f, %ld for that print statement but the output is always zero, although plot obviously shows that the variable has actually changed during play back.

• does that mean that mdf/mf4 data is not actually passed to the program? and does that mean that i can't use for example XCP data in my code for simulation and log playback purposes?

again, to make sure i'm understanding things correctly, i tried to actually write some data on the same variable and then read it:

app.set_system_var_int64("AccDisplayObj.ACCEL_REQUEST_FUZZY", 2);
s64 v;
auto ret = app.get_system_var_int64("AccDisplayObj.ACCEL_REQUEST_FUZZY", &v);
printf("########### %d, %ld", ret, v);

surprisingly, the output is still 0! and checking system variables also confirms that the variable's value is not changed!

• so my question is, what am i understanding incorrectly? shouldn't the value of the variable in the code above change?

[freshhope]

please download alpha update tomorrow, I will release it a day later

[freshhope]

• does that mean that mdf/mf4 data is not actually passed to the program? and does that mean that i can't use for example XCP data in my code for simulation and log playback purposes?
  I correct the code, now you can get sys var value in var change events
  

• so my question is, what am i understanding incorrectly? shouldn't the value of the variable in the code above change?
  Writing value into system variable created calibration module results in modifying this signal inside ECU, this requires online operation and also hardware support.
  I you just want to set value into this system variable, you can use direct memory address access, then you are able to read out value without any problem:
  

[arohami]

I confirm that using "on var change" actually works and i can use it to read out mf4 data. although it still bugs a bit that it doesn't work the same if i put the code in step function and the fact that when i write values to the variable, its value doesn't change!

[arohami]

by the way, is "dark theme" a new feature? i was looking for a way to to make it dark from day 1! but i still can't find where the option is! working with that white theme really hurts ! where/how can i enable dark mode?

[freshhope]

during replay, step function is useless

there is no dark theme, i always use negativescreen software to make all things dark, so the software interface is completely white, that means after using this software, everthing is completely dark

[arohami]

ok! that's nice. then i think part of the issue with mdf/mf4 files has been resolved.
Thanks mate :)
then the only problem i still have is with loading mdf/mf4 files from other software. I hope i can see that feature soon.

[freshhope]

After this feature is released, i will notice you

[arohami]

now that i can read mf4 file data, i have another question. is it possible to play both blf (can) and mf4 together at the same time? i think i mentioned before, my goal in general is to have a integrated simulation environment. i'm looking for a way to play back recorded data and pass all the values to my code as the first step to have some sort of simulation.

[freshhope]

replay together is possible, what you need to do is to set recording bus frames option here:

[arohami]

sorry, i don't quite get it! from what i understood you're saying that: "i should check that box when i'm going to record my data? then i can replay blf and mf4 together"?

my situation is somewhat complicated i guess. i have an already recorded CAN (blf) file and a XCP (mdf) file and i want to replay them synchronously, read their data and pass them to my code just as it happened in real time.
by the way, just to make sure, i checked that box and replayed the mdf file, but (as i guessed), only the mdf file was replayed.

[freshhope]

replay them synchronously is not possible without combining them together into one mf4 file.

This feature is tested, it will replay mf4 and CAN frames together.

[arohami]

ok. i'm sure it's a tested feature and works. i'm just asking how i can combine the two files (blf and mf4) together. by "set recording bus frames option", you mean that i should check the box while i'm recording the data in the first place? and there's no way to combine the two later? am i right? i want to make sure i'm not missing anything here.

[freshhope]

yes, currently, no way to combine them

[arohami]

one more thing about mdf/mf4 files. when i'm using on_var_change function i can correctly read xcp variables. but the problem in this way, i have to have a function for each variable that i want to read. and my mf4 file actually has hundreds or maybe some thousands of variable.

1. is there any other way to resolve this issue?
2. also all data in mf4/mdf files are played back very fast and almost instantly (without considering their timestamp!). but i want to play back those data according to their time stamp. for example if a x variable has changed after 10 seconds of starting the recording, then i want to have that change in play back at 10s after playback (e.g to pass that data to a controller or another function).
3. other issue i'm having right now is that all on_var_change callback on variable x are called (e.g 1000 times) and then all the variable changes on y are called (another 1000 times) but plotting the graph shows that those variables were changing together!

[freshhope]

1. currently not possible, I plan to add a generic sys var on change event callback, you can then deal with each var in one callback
2. that is because this is offline replay, not online replay, what you need is online replay, but this online replay module does not support mf4 file, maybe in the future, we can add this support
3. same as 2., in offline replay, signal is replayed one by one