Exam computer error and fail

Comments

16 comments

  • Avatar
    Lisa Pilorz

    To get any help from NCARB you need to contact them 202-879-0520 I think within 15 days of your exam.

    good luck

    0
    Comment actions Permalink
  • Avatar
    NCARB

    Hi Geraldine,

    As Lisa mentioned, you need to reach out to NCARB Customer Service as soon as possible if you experience a technical issue with your exam. They will be able to assist you. 

    0
    Comment actions Permalink
  • Avatar
    Cassandra Cook (Edited )

    Geraldine,

    I had a very similar experience to you when I took my last exam in July.  I was completely thrown off and lost about a minute of testing time when I only had 20 left.  (I had burned the amount of time I had left in my mind while I waited for the reboot)  I contacted NCARB immediately and they essentially said they "looked at my log" and it didn't show that I had lost any time (even though I DID lose some of that precious time) so there was nothing they could or would do.

    NCARB doesn't consider a technical issue a problem because their software is supposedly designed to stop the counter as soon as an error occurs...but news flash: things don't always work the way they are supposed to and glitches occur...but NCARB will not take responsibility for them unless it is egregious and it shows in their "log."

    0
    Comment actions Permalink
  • Avatar
    Kittijuk Tongthaworn

    I had the same problem. Computer crash mid-way, took a while to reboot but the clock didn't reset. My exam was on 11/30 in San Bruno, Ca.

    0
    Comment actions Permalink
  • Avatar
    Michael Riscica

    NCARB doesn't care and has done absolutely nothing to address these issues over the past 4 years.. They seem to think these computer problems are rare and only happen to 5% of ARE Candidates. 

    In October 2020 I recently surveyed 1,114 ARE Candidates and NCARB Stakeholders and 73% of people reported experiencing a software problem. 72% of candidates reported feeling dissatisfied with NCARB. https://youngarchitect.com/ncarbsurvey2020

     

    2
    Comment actions Permalink
  • Avatar
    NCARB

    Hi Everyone,

    I want to address some of the comments in this thread. NCARB has worked with our vendors to release multiple updates over the previous years that continue to improve software performance. Improvements continue as can be seen with the new PDF reader going live on Monday, December 14, 2020. In addition, improvements were made to improve delivery to candidates with certain visual impairments.

    Unlike the information reported by Young Architect, NCARB has access to all exam data which provides a more accurate reflection of ARE deliveries as a whole. Our comprehensive data shows less than 5% of candidates report any technical issue at the test center and less than 1% of candidates experience a technical issue significant enough not allowing that candidate to complete their division and be appropriately assessed.

    It is unfortunate that any candidate experiences a technical issue. When it does happen, it is best that every candidate reports this to their test center administrator so it can be verified and documented. Through properly reported data, NCARB can continue to work with all our vendors to improve the systems being used.

    0
    Comment actions Permalink
  • Avatar
    Caitlin Torrance

    This sounds like we should be reporting everything to Prometric officially, pass or fail, which I will admit I have not done because it seemed pointless to report on a passed exam. I've taken 4 exams, and experienced lag, calculator issues, PDF loading issues, and have never used the highlighter. None of my exams had a crash, but 100% of my exams have had technical issues. The worst was twice during one exam, going from one question to the next was about 10 seconds of blank screen. It was enough to twice cause me to think "now it's my turn". It's that's common of a worry. Should that have been reported, even though I passed?

    What do you need from the community to get more accurate data? Is there a place we can report technical issues that happened on a passed exam? Maybe a twice-per-year survey of all testing candidates asking about what works and what doesn't about the process? Maybe add a question to the exit exam survey that covers this, something like "Did you experience any technical issues that resulted in lost time?" Or a check-all-that-apply for technical issues: PDF lag, calculator, slow loading of questions, glitched drag-and-drop, temporary blank screen, computer crash, etc.

    No one expects this to be perfect. Software crashes, hardware breaks. But what should be a rare occurrence is a constant topic within the ARE testing community, both on these forums and off. People keep talking about this because it is still a problem.

    2
    Comment actions Permalink
  • Avatar
    Cassandra Cook (Edited )

    Nick from NCARB-

    First, I will tell you why your exam data does not accurately reflect what's going on...for example you say only 5% of the candidates report any technical issues.  Here are four reasons why people don't or won't report an issue:

    1. they know NCARB won't do anything to change it in a timely manner that will affect their path. 

    2. the process of challenging your score is completely futile so what's the point of saying anything?  Doing nothing and saying nothing will get them through this as quickly as possible.

    3. they don't want to make a stink before they get their license from NCARB for fear of retribution or retaliation.

    4. they have been made to feel like all of these issues are somehow a reflection of their competence to perform up to the "standards" of a practicing architect.

     

    You say that less than 1% of candidates experience a technical issue significant enough?  I will challenge that statement by asking: Who determines what is considered "significant enough?"

    1. There have been numerous studies about the cognitive ramifications of work flow being interrupted.  I have read that on average it is between 10 to 23 minutes for someone to get back into the same mindset they were in before the interruption.

    2. When you are waiting for 8+ seconds for a pdf to load or for your next question to appear those seconds add up to an already very tightly timed test.  That information is not going to be found on the "logs" or in the data, per se.

     

    Don't get me wrong, I love architecture as a profession but this process has been completely crippling to my career and the lack of timely action from NCARB to make things right is more than disheartening, it's infuriating.

    So from my point of view, this lack of urgency to correct these technical issues is a symptom of a much greater problem at NCARB.

    3
    Comment actions Permalink
  • Avatar
    James Martzahl

    It seems like making it harder to report technical issues may be in NCARB's interest to make the exams appear less faulty. There are ways for NCARB to go about getting better hard data on all the technical issues. If an independent survey shows there is vastly more technical issues than ones being reported that probably indicates the way to report technical issues is inconvenient. If NCARB wants better data all they need to do is add a survey question at the end of the exam asking if the candidate experienced technical issues and a short description. 

    Having Jared Say today at the NJ Board/NCARB discussion on the new testing scenario "You may hear false information about the exam crashing all the time but the reality is that less than 5% ever report a technical issue” makes me so angry. Instead of addressing the issue head on its easier for NCARB to just dismiss these situations as false. Having someone from NCARB label its test candidates liars instead of addressing the issue is just down right demeaning. 

    2
    Comment actions Permalink
  • Avatar
    Sarah Rinehart

    All very great points, Cassandra. I have also made the suggestion here in another thread to include another question at the end of the exam survey so folks can let them know if they experience a technical issue. It seems like very low hanging fruit. But that suggestion went ignored as will yours because it doesn't suit NCARB.

    Nick NCARB - I would love to hear someone from NCARB, perhaps yourself, address Cassandra's points above with specificity.

    2
    Comment actions Permalink
  • Avatar
    Michael Riscica (Edited )

    Nick,

    Why would me or the 1,100 NCARB Stakeholders LIE and share false information about the exam crashing?!??? What's in it for me?   

    Please explain.

    2
    Comment actions Permalink
  • Avatar
    NCARB

    Michael,

    Why would you interpret an assessment of the full data (over 100,000 exam administrations), compared to your limited survey, as evidence that people lied on your survey? That is not a leap you should make. What NCARB is sharing is data from a much a larger sample of confirmed test administrations.

    As asked by others above, NCARB does want every candidate to appropriately report technical issues through the Prometric TCA. This provides an auditable trail that is built during the testing event which would be the most valuable information to collect in an effort to continue to refine exam delivery.

    0
    Comment actions Permalink
  • Avatar
    James Martzahl

    Nick, 

    i am sorry to say but your response is tone deaf. We are telling you that we are noticing a striking difference between the data from an independent study and the data from a company that doesn't make it easy to report issues. Also, your response is tone deaf when NCARB's relationship with testing candidates is at an all time low... Why cant we all come to the table and work together to make sure we get a good pool of data that everyone can agree is fair and transparent?

    We put the independent survey together as a result of NCARB not listening and its evident today as it ever was that NCARB has no intention of listening. We literally did NCARB's job in trying to assess the issues candidates are experiencing. If our data is so full of lies then why doesn't NCARB make their own official survey addressing these issues?

    If you want eventually stop hearing these accusations (which NCARB deems false) then you need to allow those against you a seat at the table to fix this as a whole. I am sick and tired of being told that the candidates are wrong and all our concerns are null and void. If there wasn't so many issues with the software your job would literally be easier, and you wouldn't continue to see posts like this show up on the NCARB community. 

    Lastly, May I suggest you settle down on your public relations. Denying and placing blame on people that pay NCARB in test fees and record fees don't deserve to be talked down to like children. 

    Please, all we are trying to do is help NCARB. We need to feel like we are being listened to and taken seriously.  

     

     

    1
    Comment actions Permalink
  • Avatar
    Kathryn O'Regan

    I took 8 exams: 2 in 4.0 and all 6 in 5.0.  I experienced a power outage at the test site on site planning 4.0. While the test did reboot, I lost some of my saved work and some questions that I had completed were showing up as incomplete.  I failed and tried to contact NCARB.  All emails were form emails saying "the center confirms your report of power outage but your data was saved."  Ok, some of that data was saved, but not all, and it didn't save the work I had done on the vignette, so I had to redraw everything.  I tried calling the NCARB office in DC, and once I was able to speak to someone and explain the problem, they hung up on me, and every other time after that, I was put on hold or disconnected.

    In 5.0, I had one crash and reboot, and one test where EVERYTHING took at least 10 seconds to load (beyond the normal lag of a few seconds). I didn't report because 1) I passed and 2) my last experience proved it wasn't worth the headache.

    So 3/8 of my exams were glitchy.  37.5% error rate.  Statistical anomaly if you're so certain that only 5% of test takers experience errors.

    1
    Comment actions Permalink
  • Avatar
    Michael Riscica (Edited )

    Nick, 

    I just don't understand. Why is 73% of 1,113 Candidates reporting Technical problems and how come your data is showing less then 5%???? Where is the disconnect??? Who's data is inaccurate and why?!??? 

    Please explain why sooo many Candidates are reporting the same problems after all these years. 

    This is would be a great time for NCARB to be transparent and show us your data.  I have a hard time believing your words after hearing from 1,113 ARE Candidates who are telling a much different story then NCARB is regarding technical Issues inside the testing center.  

    0
    Comment actions Permalink
  • Avatar
    Cassandra Cook

    Where did Nick NCARB go?

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk