Major Exam Software Issues

Comments

13 comments

  • Avatar
    Brandon Estes

    Aw man!  This sucks!  I had to deal with this for two exams and I did a bunch of research and combed nearly 3 years of this ARE 5.0 Community... NCARB staff is well-aware of these problems, so is the NCARB board.  I wrote a letter to my state elected rep, to the CA Architect Board, and sent copies to the NCARB Board member for CA and the Board chair...

    Of course, NCARB staff replied with a letter basically saying I was crazy and NCARB is perfect.  Again, this has been an on-going with problem with ARE 5.0, NCARB is aware of it, but they won't address publicly nor do they fix it, and it's been about 4 years now.  If you have the energy (which I hope you do), figure out which board member serves your area and send him a letter and copy the Board Chair.  

    The US is broken up in Regions and there is a board member for each Region.  You can google them and get a work addy.  I would suggest that you not send a letter to NCARB because they will not give it to the Board and they will not provide any info on Board members for you to contact them - I tried.  But you should write to the Board members because NCARB staff will not let the board know about anything like this.  And who knows - maybe the board doesn't give a ****.  But at least you can do your effort to shed light on this.  I have only the two big exams left (was a 4.0 to 5.0 transfer).  If I don't get through the PPD/PDD by the time other exams expire, I am putting this behind me.  NCARB is a circus and I want my life back.  Had I seen behind the curtain when I was younger, I would have studied engineering.  School doesn't teach you what you need to know, NCARB doesn't test you on what you need to know, and if you actually become licensed, it doesn't pay as well as you expect.  So I went to work for the state government.

    Board members: https://www.ncarb.org/about/board-directors. 

    3
    Comment actions Permalink
  • Avatar
    Michael Riscica

    Brandon, 

    I've also contacted many different State Boards about the software issues on NCARB's ARE 5.0 software. I'd like to chat with you further.

    Will you email me. Mikeriscica@gmail.com 

    0
    Comment actions Permalink
  • Avatar
    Victoria Shingleton

    Something similar happened with me today with PPD. Everything was going smoothly - I had already answered all questions and was just reviewing my answers, and then suddenly a notification popped up that read, "Program Failure." YIKES! Not something you want to see while taking an exam! I called in the Prometric staff member who was very calm and nice. I asked her if this was bad, and she said no, nothing to be worried about (how can you not be worried after taking a 120 question test and then reading Program Failure and having someone hard close the program and reboot the computer before you were able to submit?!?!?!). She rebooted the computer three or four times before deciding it would be best to switch me to a different station. The only positive in this is that she let me go to the bathroom while she was on hold with Prometric trying to move my test to a different machine. After around 25 minutes, I was set up on a new computer. The good news is that I guess your answers are saved to the server while you take the test so they can move you to another station when this happens. It picked up where I left off, but I didn't have enough time to check my answers to make sure they were the same and frankly I just didn't care anymore at that point.

    Luckily for me, this happened at the end of my test. And the 25 minutes I spent waiting for my station to be reloaded was AGONIZING. My stomach just dropped. Had it been halfway through my test, it would've totally messed with my mindset. Heck, I was almost done and I went in and just checked a few things to make sure my answers looked like they were and then clicked, "End Exam," I was so drained.


    The Prometric staff was very nice. She told me that she recorded the error and took that station out of service. She said the error is recorded with Prometric and NCARB, and I'm welcome to pursue it with them further if I want. I'm not going to pursue it just because I don't think it's worth my effort, especially seeing how when other people do pursue it, NCARB takes forever to get back and then says, "Yeah, nah, it's good."

    2
    Comment actions Permalink
  • Avatar
    Jesse Damrow (Edited )

    Thanks for the feedback, it is more than I have received from NCARB that's for sure. I am at peace with the exam software crashing mid exam....I understand this happens from time to time and is very unfortunate. What I cannot accept is questions where the options are not selectable, essentially rendering the question unanswerable. Not holding my breath for a response.....I hope they get that sorted out however. 

    1
    Comment actions Permalink
  • Avatar
    Andrew Kim

    I'm not sure how common the software issues are, but if my experience today is any indication, the software is not very good. Took my first test today (PcM) and the system logged out 4 or 5 times in the first 30 minutes. It really threw off my rhythm. The Prometric staff was trying to be helpful and moved me to another terminal while they were on the phone with NCARB. But not much more they could do. It'd be good to know how much the clock is still running while the issues are taking place because I had to rush through the last 5 or so questions to finish, basically making educated guesses. Hopefully this is just a one-off and others aren't experiencing something similar.  

    1
    Comment actions Permalink
  • Avatar
    Brandon Estes

    HAHA!  I am laughing because this is not a "one-off" - this is inherent in the 5.0 exam software.  It has been from the start, NCARB knows all about it, but they won't acknowledge nor fix it.  I took a PDD exam on Monday and some of the drag-drop questions wouldn't allow me to drag drop - shocker... I didn't bother to report it because I think the exams are intentionally designed with these flaws.  I take PPD again on Friday and I expect the same issues.

    I failed PDD on Monday and if I fail PPD this Friday (which I hope I do, otherwise I have to try to pass PDD again), I am done.  I have already boxed up all my study materials and I am going to donate most of my books.  I already feel a weight lifted from my shoulders.  

    I have taken PDD three times now and this Friday will be three times for PPD.  Pass rates are dismal and the software is laughable.  I refuse to contribute another $230 towards NCARB's executive staff's Benz payments.  The system is a boondoggle, NCARB is a joke, the profession is imploding, and the small handful of architects that control the discipline (board members/exec's at AIA, NCARB, NAAB) don't want to make changes because that means competition.  Competition means less $$ for them.  Monopolies are great for the people at the top.   The previous Chair of the NCARB board, David Hoffman, left his post to make room for a female board member (no people of color on the board, nobody that's not a senior citizen, test-takers still not represented).  Where did David go?  Oh, he's down the street on the Board of NAAB now.

    2
    Comment actions Permalink
  • Avatar
    Heidi Han

    I also have had similar situations!  I have taken the 5.0 versions of the exam (transitioning from 4.0) two times, and on BOTH occasions, had major software + computer issues.

     

    The first time:

    I had nearly 10+ mins left on the exam clock, but the prompt said that my exam was finished and it proceeded to ask me to start taking the survey.  Who knows what I could have achieved in 10 more minutes?  10 more minutes to check questions.  10 more minutes to plow through 5 or more questions (I did not finish all the exam questions this day!).

     

    The second time:

    I started my exam, and about 25 questions in, the computer program just stopped and booted me out.  Given my first experience, I was very very nervous because it was almost like reliving the first attempt - added stress and worry.  A member of Prometric came to move me to another station, but as you can imagine, getting used to a new station/location/environment and being nervous for any other computer related issues did not help my test taking worries at all.

     

    There are many other worries and concerns that test-takers have (passing, recalling information!) on test taking day - and repeated computer software malfunctions should not be one of them.  We have also given NCARB a lot of money to develop these exams and software.  This is really unacceptable.

    I will be going in for a 3rd exam on the 5.0 system and I am already nervous of the situation I may run into this time.

    In addition to my study strategies, I have now learned to factor in 'what-if' situations (in addition COVID-19 worries).

    I don't believe that a candidate should be worried about software malfunction issues when testing.  This is really unprofessional of NCARB.

    1
    Comment actions Permalink
  • Avatar
    Peter Mall

    After experiencing issues did you pass or fail the exam? I have experienced software problems on almost every divisions of the ARE I have taken. It causes stress and anxiety, but it didn't cause me to fail. I wouldn't classify them as "major problems", just a minor hiccup. Most of the stories above, mine included, are resolved when the proctor resets the computer.

    This may be an unpopular opinion, but I believe a candidate should be prepared for anything and everything. In practice we encounter issues everyday which requires us to think on our feet - be it an RFI, critical path item, consultant coordination, construction error, or even software problems. Have you ever considered this is by design? Its not that there is a software problem, its how you handle it.

    -4
    Comment actions Permalink
  • Avatar
    Victoria Shingleton

    LOL @Peter such an architect answer.

    Public: This building is ugly! And its reflective panels are blinding everyone on the corner!

    Architect: Have you ever considered this is by design?

     

    In all seriousness.... yes, the exam seems to recover just fine. It doesn't seem like any answers are lost. And sure, candidates should be prepared for "anything." But I don't think we should be making excuses for NCARB and Prometric regarding software failure. The fact that software failure is so common and has been for years.... even decades, I'm hearing, is problematic. I do not believe that this exam is designed to shut down to "program failure" by design. I'm not sure if the program failure is due to the software or the hardware it is being run on, but technology is so advanced these days and there is ample availability of skilled professionals who could be hired to solve this problem. It's simply embarrassing for NCARB that this continues to happen.

    5
    Comment actions Permalink
  • Avatar
    Peter Mall

    @Victoria, I think you misunderstand. I am not defending the software. I am just saying that we know issues CAN occur and a candidate should be prepared to NOT let it throw them off. There is so much preparation needed for the exams themselves, I just don't advocate for chasing software issues. Its a classic "bigger fish to fry" scenario. If anyone is overly concerned they can wait until NCARB rolls out online-proctored exams.

    0
    Comment actions Permalink
  • Avatar
    Victoria Shingleton

    Peter Mall, I don't think anyone here is chasing software issues. Maybe that in itself is an issue because I'm certainly not holding my breath that this will ever be fixed. And I have no faith in the online proctored exam and will continue to go to the testing center. At least at the testing center, when your program shuts down, you at least have the Prometric staff to work to fix the issue. If the program fails during at home proctored exams, who do you call.... Ghost Busters?

    There's a big difference between "chasing issues," and bringing to the company/organization's attention that there is an issue.

    Let's say you hire someone from a cleaning service to clean your home. That person shows up, kicks some dust around, and leaves. House is not cleaned. Do you call the company and let them know that you're unhappy with their service? Or do you not bother because you still need your house cleaned, so that's the "bigger fish to fry."

    Or how about you buy a computer. Computer arrives and doesn't work. Do you call the company and let them know the product is broken? Or do you go and buy another computer because you need one that works and that is the "bigger fish to fry."

    In either scenario, you probably do both. Yeah, hire another housekeeper or start cleaning yourself, or go ahead and buy another computer because you need it asap, but that doesn't mean you wouldn't pursue the issue with the company. Really it's in the company's best interest - they should want to know when their product or service is failing the consumer.

    So I don't think anyone should stop testing (assuming their goal is licensure), but I also think it's appropriate to bring this common program failure issue to NCARB's attention, because we all share the common goal of licensing architects.

    1
    Comment actions Permalink
  • Avatar
    Jesse Damrow

    Call me naive but I do not believe software issues should be "expected". There is enough to prepare for given the wide swath of study and reference materials, adding "potential software issue strategies" is not in my future plans. If the exam crashes, and the proctor resets, so be it. You really don't lose time (so they say) except......what if you are mid calculation on a long question? Having to stop and start in that regard definitely puts you at a disadvantage. The more troubling one for me however are the drag and drop issues. There should in NO instance be a question that is "unanswerable" due to a software issue. I am sure there could be a scenario wherein you could miss the cut by one question, and that question wasn't available to answer. If the software is directly responsible for accounting for a "wrong" answer that could have otherwise been "correct"......then that test should be nullified, NO question about it. 

    Not crying over spilled milk here however.......we must continue and do our best with what we've been presented. I just wish they would at least acknowledge these issues instead of treating the community that supports their business as if it is just tough luck and without response.  

    6
    Comment actions Permalink
  • Avatar
    sepeedeh Rastegar-Panah

    I had huge amounts of technical issues and disturbances on my in person exam with the software the other day too. And just received my score with only one Level 3 which is a huge bummer with respect to all the time loss and due stress I had to go through because of those glitches. 

    Multiple issues that happened during my exam include
    1- The exam ended abruptly and without the regular procedures. I was watching my timer and I still had over one minute when this happened ; proceeded to the end exam page.
    2- the exam loaded super slow on my monitor , I could not move between questions without it taking at least 7 seconds (yes I counted it) to load for text questions and at least 15 for almost all of the questions with an image. 
    3- The time it took the whiteboard to appear and load (sometimes it just didnt!) was ridiculous. Putting the whole taking a way scratch paper & pencil from an architect "to -be " story to the side , Replacing it with a glitchy incapable software is just not acceptable, it makes me so furious... if all the architects in the past 70 years or so used a scratch paper why all of a sudden can't I?
    4- I could not load case study drawings without a huge delay and same story about the code excerpts section I dont think I even was able to load the code at all, just trying to cross check a documents between tabs was impossible due to the lag and at that point I was afraid of clicking the whiteboard and getting interrupted again.... like just memorize everything in the code and forget about trying to go through it on the exam and by the way no lighlighting there either ... 
    5- My exam was interrupted by software difficulties two times , that I had to walk out of the testing room and wait for it to be resent to the proctor again, no explanation whatsoever during this whole time, They just waved at me like hurry up it has started sit and go!!!! I had to recalc my time to see where I stand  -   just trying to get reoriented ate up a lot of time and also added a huge load of stress. I was not explained how much time I lost or what was going to happen not before when I was waiting and not after the exam.
    I would like NCARB to require the test center to provide a report on the speed and quality of the internet that was received by the machine I was testing on during that exam time.
    I studied and tried really hard so I wouldn't have to retake this exam and these issues are just unacceptable. I would perform an in person paper based exam anytime NCARB requires me to, I'm okay w taking the test someone took 40 years ago but just this quality of computer based testing was just unacceptable. 

    1
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk