ARE EXAM EXIT SURVEY
After listening in on the NJ Board/NCARB discussion on the new testing scenario, Jared Mentioned: “You may hear false information about the exam crashing all the time but the reality is that less than 5% ever report a technical issue”
I cant help but point out the fact that maybe if NCARB had a quick question about the exam experience for the candidate to fill out after completing the test, NCARB would have all the information at their finger tips about exam issues. It is my belief that there is a large group of candidates that experience issues with the exam but still receive a passing score, minimizing the probability that they would report said issues.
So just to summarize, if NCARB wants to get data from their test takers on issues with crashes, one of the easiest ways to go about it would be to have a quick survey question at the end of the exam addressing this.
FYI: This is coming from someone who just passed PA on Monday, but had my test error and completely shut off resulting in the proctor having to reset the exam. I now have to go out of my way to report these issues to NCARB when it could have been handled there right there at the source.
-
Hi James and Shannon,
Thanks for the suggestion of adding it to the post-exam survey. This was considered at one time but it was decided that a post administration survey response wouldn’t allow for the best reporting as it is limited to the end of the appointment and removed from the incident.
It is important for all candidates to report technical issues to the test center administrator and allow the administrator to verify the issue so the administrator can accurately document the condition. Not all exam technical issues have the same root cause. Proper reporting of the issue allows Prometric to verify and document conditions for their use as well as NCARB’s.
-
Nick,
I think it would be greatly beneficial for NCARB to explore this again. Just because it was explored in the past and shot down does not mean the decision would still apply now in the situation we are in with the new testing scenario.
Secondly, I don't understand what you are saying in regards to "that a post administration survey response wouldn’t allow for the best reporting as it is limited to the end of the appointment and removed from the incident.".... so are you telling me that by reporting issues to Prometric after the exam and then reporting issues to NCARB's website after I get home is better then doing it immediately after the exam? That argument holds no water.
NCARB literally has a site to submit these issues at https://www.ncarb.org/contact#expand-jump-examination ... This can easily be incorporated into the test as a survey question. Please instead of dismissing the idea immediately maybe discuss it again. It would at least have the appearance of consideration, instead of appearing as immediate disregard.
I am literally just trying to help NCARB.
-
James,
There are currently two points where exam issues are documented. The first point is when you inform Prometric staff that something irregular occurred during your exam. Prometric documents the issue in detail through a Center Problem Report (CPR). These details include what happened and any supporting documentation, like what an error message said. As stated in the previous post, we need this information to be documented at the point the issue occurs so no important details are lost. Prometric proctors file CPRs for all exam irregularities, not just for technical issues.
The second point of documentation occurs when a testers reaches out to NCARB, per the ARE Guidelines, to report the issue. This provides NCARB a second documented account of what occurred in the test center. Together, we use both pieces of documentation when evaluating why and how an issue occurred.
-
Nick,
Make it more convenient for the second point of documentation to occur... at the testing facility... at the end of the exam.
If NCARB has any interest in getting better data, this is a good way to do it. this is such a basic way to make your data related to technical issues better. Just sit down and discuss it instead of immediately dismissing the idea. We want NCARB to succeed, to do better, and improve poor relations with the test candidates.
-
I think that everyone knows that the questions about readiness and comprehensiveness of the exam compared to health safety and welfare, are not necessarily answered as people actually feel.
For example, several times in my testing I may have felt that the exam questions I received did not compare to the material I studied for that particular exam.
My point lends itself to NCARB Nicks original post - its probably not an accurate account / the best time to ask those questions.
Mark, Archizam
-
Is there a NCARB page where candidates can report the exams that they experienced technical glitches on? I’ve experienced glitches on the majority of exams I took in both ARE 4.0 and 5.0. The only times I reported them were when I couldn’t complete the exam due to a total crash or when the computer had to be restarted. A candidate’s focus during an exam division is on the content. Calling the proctor over every time an error box pops up would drastically reduce the time a candidate can spend answering questions. Candidates should be confident that NCARB corrects their test software on their own. We are being tested on content, not the technical operability of the program. It sounds like there is a technical problem in how NCARB receives or parses data from exam audits since the findings are not consistent with candidate experience.
-
Arielle,
you make some valid points.
The candidates taking the tests shouldn't be burdened with calling over the proctor every time the errors appear. NCARB should have their employees go to Prometric, take the exams, and log all the errors they encounter so we don't have to do the job of reporting errors. This process should have been completed before anyone even saw ARE 5.0 and it should have been tested and modified until there were no such errors appearing. We, as the test taker should never see errors or have crashing and lagging issues, we should only have to focus on the content of the exam and doing our best.
You can report your errors at https://www.ncarb.org/contact#expand-jump-examination I did the same last week after my PA exam crashed.
-
Thanks James,
The link is nested under The Examination Icon > Test Center / Exam Issues. I’m going to report the other incidences I had. Have you come across any statement in the website that says submitting a report will not result in reprisal?
A lot of people fill out the survey at the end of the exam in the way they think NCARB wants to hear bc they are worried it will impact their score.
I don’t think the fear is founded, but it would help if NCARB made a statement that reporting errors will not result in a black mark on your NCARB record.
Please sign in to leave a comment.
Comments
10 comments