Populate Exam List

[lucaboesch]

Hi SEB Server developers

Although in a commment it reads that it needs some time to populate the exam list, upon a successful connection to a LMS I do not see any recurring webservices requests there (as I would expect it it was populating a list), just when I click the "Test LMS Connection" link. Nothing else.

Am I getting something wrong?

[anhefti]

Hi Luca

Using the "Test LMS Connection" in the LMS Setup section just tests if SEB Server has access to the LMS with the given URL and credentials. It do not fetch any exam data at this time.

The exam data fetching is done in the "LMS Exam Lookup" section. As soon as your LMS Setup is active and you are going to the "LMS Exam Lookup" page, SEB Server starts to fetch exam data from all active LMS Setups. This can sometimes take a while for LMS with a lot of exams or courses. SEB Server stores them in a short-time cache. This is done in a background job for some LMS (Moodle for exmale) and it might happen that the list appears to the user before the whole fetching is done in the background. So the list might not be complete on the first time you show it after new LMS Setup (but will be soon).

This will be improved in the future, for Moodle in particular but also in the GUI, showing when a fetching process is still running not note this to the user.

[lucaboesch]

Dear Andreas

Thank you for getting back to me so fast.
I'm well aware of that.
It could be though that there's an issue. Allow me to attach some debugging code I found whilst

docker logs seb-server

It's those

19:19:37.246 ERROR [asyncService-1]:[ch.ethz.seb.sebserver.webservice.servicelayer.lms.impl.moodle.legacy.MoodleCourseDataAsyncLoader] LMS Setup: 3 Unexpected exception while trying to get course data: 
java.lang.RuntimeException: Set CircuitBeaker to half-open state. Cause: java.util.concurrent.TimeoutException
	at ch.ethz.seb.sebserver.gbl.async.CircuitBreaker.handleClosed(CircuitBreaker.java:182)
	at ch.ethz.seb.sebserver.gbl.async.CircuitBreaker.protectedRun(CircuitBreaker.java:140)
	at ch.ethz.seb.sebserver.webservice.servicelayer.lms.impl.moodle.legacy.MoodleCourseDataAsyncLoader.callMoodleRestAPI(MoodleCourseDataAsyncLoader.java:427)
	at ch.ethz.seb.sebserver.webservice.servicelayer.lms.impl.moodle.legacy.MoodleCourseDataAsyncLoader.getQuizzesBatch(MoodleCourseDataAsyncLoader.java:228)
	at ch.ethz.seb.sebserver.webservice.servicelayer.lms.impl.moodle.legacy.MoodleCourseDataAsyncLoader.loadAllQuizzes(MoodleCourseDataAsyncLoader.java:194)
	at ch.ethz.seb.sebserver.webservice.servicelayer.lms.impl.moodle.legacy.MoodleCourseDataAsyncLoader.lambda$loadAndCache$0(MoodleCourseDataAsyncLoader.java:184)
	at ch.ethz.seb.sebserver.gbl.async.AsyncRunner.runAsync(AsyncRunner.java:36)
	at ch.ethz.seb.sebserver.gbl.async.AsyncRunner$$FastClassBySpringCGLIB$$f147aec0.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
	at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.TimeoutException: null
	at java.base/java.util.concurrent.FutureTask.get(Unknown Source)
	at ch.ethz.seb.sebserver.gbl.async.CircuitBreaker.attempt(CircuitBreaker.java:253)
	at ch.ethz.seb.sebserver.gbl.async.CircuitBreaker.handleClosed(CircuitBreaker.java:163)
	... 16 common frames omitted
19:19:37.676 WARN  [asyncService-7]:[ch.ethz.seb.sebserver.gbl.async.CircuitBreaker] Max blocking timeout exceeded: 20000, CLOSED

Could this be of any help?

Best,
Luca

[anhefti]

Hmm, it seems that SEB Server, while trying to request to the LMS/Moodle did not get any response from it within the configured max-blocking-time of 20seconds.

So Moodle seems not to respond to the SEB Servers course data requests.

This might happen because the LMS Setup is not set up properly or because SEB Server do not have any access on the course data endpoint on Moodle side. But in the second case I would expect a 403 FORBIDDEN response from Moodle but Moodle seems just not responding at all.

You can check the LMS Setup if it is setup correctly by checking the URL and credentials and by deactivate it and try to activate it again.
Make also sure that the Moodel Account given for the LMS Setup to connect to Moodle is valid and has all permissions to access the webservice-endpoints needed for requesting the exam data. Please have a look at the user documentation --> "Create Moodle API Access Account" for that (webservice must be enabled).

Hope this helps.
Best
Andreas

[lucaboesch]

Hi Andreas
Another team has independently set up another Moodle instance to be accessed by this machine and in that instance created a webserver enabled user. The result is unfortunately the same: connection to the system can be established, but the fetching of the exam list fails.
With the same error message.

Best,
Luca

[anhefti]

Hi Luca

Since I only have the above error stacktrace that indicates that Moodle is not answering on time for a particular request call, I can only guess that there either is nothing to reach on the respective request URL or Moodle is not answering for some reason. But maybe there is more valuable information on the log that can helps us. For that it would be helpful to set debug logging on for the respective module by setting:

logging.level.ch.ethz.seb.sebserver.webservice.servicelayer.lms=DEBUG

in your application-prod.properties file of the SEB Server installation and restart the server. Then regenerate the error again and get the log +- some seconds more. So we should be able to find the exact URL that SEB Server is using to call Moodel.

What you also can try is to set the blocking time for the Moodle call to a minute (for example) instead of 20s and see if there will be a response from Moodle in that extended time period.

sebserver.webservice.circuitbreaker.moodleRestCall.blockingTime=60000

For your information

At ETHZ we use a more restrictive Moodle configuration and have exported our role and account settings to support SEB Server here (File is zipped):

webservice_seb-server(1).zip

This file is also linked in the user guide. You can try to apply this settings or at least look at it to see the settings we made.

SEB Server uses the following Moodle webservice functions from the Moodle core:

"core_course_get_courses"
"mod_quiz_get_quizzes_by_courses"
"core_course_search_courses"
"core_course_get_courses_by_field"
"core_user_get_users_by_field"

You can also try to call them with a common curl-tool and the account credentials you have to see closer what the problem might be here.

If this all is not solving the problem, we can also make a Zoom-Call and have a look together?

Best regards
Andreas

[lucaboesch]

Thank you @anhefti

Your instructions helped debug.

At next we encountered

Read timed out; nested exception is java.net.SocketTimeoutException: Read timed out, CLOSED
14:07:41.562 ERROR [asyncService-2]:[ch.ethz.seb.sebserver.webservice.servicelayer.lms.impl.moodle.legacy.MoodleCourseDataAsyncLoader] LMS Setup: 3 Unexpected exception while trying to get course data: 
java.lang.RuntimeException: Set CircuitBeaker to half-open state. Cause: java.util.concurrent.ExecutionException: org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://moodle.bfh.ch/webservice/rest/server.php": Read timed out; nested exception is java.net.SocketTimeoutException: Read timed out

We raised the read timeout as well by raising the value on

sebserver.http.client.read-timeout=X

and then it worked out beautifully.

Best,
Luca

[anhefti]

Hi Luca

Perfect, glad to hear that.

The most connection timeouts are set to 20s for default because usually this is enough. It seems that your Moodle is unusually slow by processing the particular requests described above. Maybe you can also find the cause and improve that on Moodle site.

Best
Andreas

[lucaboesch]

Oh, Andreas, I think it is the sheer dimension of our install we connected it with.
Speaking of which: I think Safe Exam Browser Server is stopping after fetching the first 500 exams, is it not?
Should I file an issue?

Best,
Luca

[anhefti]

Hi Luca

Thank you for the response information, this is very valuable for us.

The whole problem with the current Moodle integration is that Moodle core (webservices) does not allow to search/filter courses in a convenient way e.g. to fetch all courses that are not older then one year. But one has to fetch all courses and quizzes page by page and apply the filter criteria on SEB Server side for caching. This cause a lot of load and traffic for Moodle with a hight number of existing and actual courses.

We are currently working on a Moodle plugin for better SEB Server integration that also addresses this Issue among others. With the new plugin SEB Server will be able to also send the filter criteria to Moodle and fetch only the courses/quizzes data of interest.

For now there is a background service that does the fetching and caching of Moodle courses and it has some default limitations:

this.maxSize = environment.getProperty("sebserver.webservice.cache.moodle.course.maxSize", Integer.class, 10000);
this.pageSize = environment.getProperty("sebserver.webservice.cache.moodle.course.pageSize", Integer.class, 500);

The "sebserver.webservice.cache.moodle.course.maxSize" limitation is the maximum size of course-cache on SEB Server site and is set to 10000 for default. So you should be able to fetch at least 100000 courses/quizzes.

The "sebserver.webservice.cache.moodle.course.pageSize" is the size of a page (or batch) of courses that SEB Server use to fetch for one request. So if you have a Moodle with a lot of courses SEB Server fetches the courses page by page in the background, get all quizzes for the courses in the fetched page and cache them. Then next course page if available...

Now I may have found an Issue with this since it would be possible that for one page (500) courses there is no course with a quiz.
No quiz for a page, then the SEB Server stops fetching because it assumes that there are no more (what may be wrong).

I will fix this for the next version of SEB Server (1.5) for the Moodle core integration. But if we are lucky, there will also be the new plugin available for the next version where this Issue doesn't matter at all. Next version can be expected in late Q1, early Q2 2023.

For a workaround, if you need one, you can try to increase the page size limit to 1000 for example. This would increase also the possibility to have one quiz within the page of 1000 courses. (Note: this might also increase again your Moodle response time)

Beside of that I still wounder that the fetch of one page (500) takes more then 20s from Moodle site since we are using the same limitations and it should be independent from the whole number of courses living on Moodle.

Best
Andreas