Major Exam Software Issues

Comments

12 comments

  • Avatar
    Brandon Estes

    Aw man!  This sucks!  I had to deal with this for two exams and I did a bunch of research and combed nearly 3 years of this ARE 5.0 Community... NCARB staff is well-aware of these problems, so is the NCARB board.  I wrote a letter to my state elected rep, to the CA Architect Board, and sent copies to the NCARB Board member for CA and the Board chair...

    Of course, NCARB staff replied with a letter basically saying I was crazy and NCARB is perfect.  Again, this has been an on-going with problem with ARE 5.0, NCARB is aware of it, but they won't address publicly nor do they fix it, and it's been about 4 years now.  If you have the energy (which I hope you do), figure out which board member serves your area and send him a letter and copy the Board Chair.  

    The US is broken up in Regions and there is a board member for each Region.  You can google them and get a work addy.  I would suggest that you not send a letter to NCARB because they will not give it to the Board and they will not provide any info on Board members for you to contact them - I tried.  But you should write to the Board members because NCARB staff will not let the board know about anything like this.  And who knows - maybe the board doesn't give a ****.  But at least you can do your effort to shed light on this.  I have only the two big exams left (was a 4.0 to 5.0 transfer).  If I don't get through the PPD/PDD by the time other exams expire, I am putting this behind me.  NCARB is a circus and I want my life back.  Had I seen behind the curtain when I was younger, I would have studied engineering.  School doesn't teach you what you need to know, NCARB doesn't test you on what you need to know, and if you actually become licensed, it doesn't pay as well as you expect.  So I went to work for the state government.

    Board members: https://www.ncarb.org/about/board-directors. 

    2
    Comment actions Permalink
  • Avatar
    Michael Riscica

    Brandon, 

    I've also contacted many different State Boards about the software issues on NCARB's ARE 5.0 software. I'd like to chat with you further.

    Will you email me. Mikeriscica@gmail.com 

    0
    Comment actions Permalink
  • Avatar
    Victoria Shingleton

    Something similar happened with me today with PPD. Everything was going smoothly - I had already answered all questions and was just reviewing my answers, and then suddenly a notification popped up that read, "Program Failure." YIKES! Not something you want to see while taking an exam! I called in the Prometric staff member who was very calm and nice. I asked her if this was bad, and she said no, nothing to be worried about (how can you not be worried after taking a 120 question test and then reading Program Failure and having someone hard close the program and reboot the computer before you were able to submit?!?!?!). She rebooted the computer three or four times before deciding it would be best to switch me to a different station. The only positive in this is that she let me go to the bathroom while she was on hold with Prometric trying to move my test to a different machine. After around 25 minutes, I was set up on a new computer. The good news is that I guess your answers are saved to the server while you take the test so they can move you to another station when this happens. It picked up where I left off, but I didn't have enough time to check my answers to make sure they were the same and frankly I just didn't care anymore at that point.

    Luckily for me, this happened at the end of my test. And the 25 minutes I spent waiting for my station to be reloaded was AGONIZING. My stomach just dropped. Had it been halfway through my test, it would've totally messed with my mindset. Heck, I was almost done and I went in and just checked a few things to make sure my answers looked like they were and then clicked, "End Exam," I was so drained.


    The Prometric staff was very nice. She told me that she recorded the error and took that station out of service. She said the error is recorded with Prometric and NCARB, and I'm welcome to pursue it with them further if I want. I'm not going to pursue it just because I don't think it's worth my effort, especially seeing how when other people do pursue it, NCARB takes forever to get back and then says, "Yeah, nah, it's good."

    1
    Comment actions Permalink
  • Avatar
    Jesse Damrow (Edited )

    Thanks for the feedback, it is more than I have received from NCARB that's for sure. I am at peace with the exam software crashing mid exam....I understand this happens from time to time and is very unfortunate. What I cannot accept is questions where the options are not selectable, essentially rendering the question unanswerable. Not holding my breath for a response.....I hope they get that sorted out however. 

    0
    Comment actions Permalink
  • Avatar
    Andrew Kim

    I'm not sure how common the software issues are, but if my experience today is any indication, the software is not very good. Took my first test today (PcM) and the system logged out 4 or 5 times in the first 30 minutes. It really threw off my rhythm. The Prometric staff was trying to be helpful and moved me to another terminal while they were on the phone with NCARB. But not much more they could do. It'd be good to know how much the clock is still running while the issues are taking place because I had to rush through the last 5 or so questions to finish, basically making educated guesses. Hopefully this is just a one-off and others aren't experiencing something similar.  

    0
    Comment actions Permalink
  • Avatar
    Brandon Estes

    HAHA!  I am laughing because this is not a "one-off" - this is inherent in the 5.0 exam software.  It has been from the start, NCARB knows all about it, but they won't acknowledge nor fix it.  I took a PDD exam on Monday and some of the drag-drop questions wouldn't allow me to drag drop - shocker... I didn't bother to report it because I think the exams are intentionally designed with these flaws.  I take PPD again on Friday and I expect the same issues.

    I failed PDD on Monday and if I fail PPD this Friday (which I hope I do, otherwise I have to try to pass PDD again), I am done.  I have already boxed up all my study materials and I am going to donate most of my books.  I already feel a weight lifted from my shoulders.  

    I have taken PDD three times now and this Friday will be three times for PPD.  Pass rates are dismal and the software is laughable.  I refuse to contribute another $230 towards NCARB's executive staff's Benz payments.  The system is a boondoggle, NCARB is a joke, the profession is imploding, and the small handful of architects that control the discipline (board members/exec's at AIA, NCARB, NAAB) don't want to make changes because that means competition.  Competition means less $$ for them.  Monopolies are great for the people at the top.   The previous Chair of the NCARB board, David Hoffman, left his post to make room for a female board member (no people of color on the board, nobody that's not a senior citizen, test-takers still not represented).  Where did David go?  Oh, he's down the street on the Board of NAAB now.

    1
    Comment actions Permalink
  • Avatar
    Heidi Han

    I also have had similar situations!  I have taken the 5.0 versions of the exam (transitioning from 4.0) two times, and on BOTH occasions, had major software + computer issues.

     

    The first time:

    I had nearly 10+ mins left on the exam clock, but the prompt said that my exam was finished and it proceeded to ask me to start taking the survey.  Who knows what I could have achieved in 10 more minutes?  10 more minutes to check questions.  10 more minutes to plow through 5 or more questions (I did not finish all the exam questions this day!).

     

    The second time:

    I started my exam, and about 25 questions in, the computer program just stopped and booted me out.  Given my first experience, I was very very nervous because it was almost like reliving the first attempt - added stress and worry.  A member of Prometric came to move me to another station, but as you can imagine, getting used to a new station/location/environment and being nervous for any other computer related issues did not help my test taking worries at all.

     

    There are many other worries and concerns that test-takers have (passing, recalling information!) on test taking day - and repeated computer software malfunctions should not be one of them.  We have also given NCARB a lot of money to develop these exams and software.  This is really unacceptable.

    I will be going in for a 3rd exam on the 5.0 system and I am already nervous of the situation I may run into this time.

    In addition to my study strategies, I have now learned to factor in 'what-if' situations (in addition COVID-19 worries).

    I don't believe that a candidate should be worried about software malfunction issues when testing.  This is really unprofessional of NCARB.

    0
    Comment actions Permalink
  • Avatar
    Peter Mall

    After experiencing issues did you pass or fail the exam? I have experienced software problems on almost every divisions of the ARE I have taken. It causes stress and anxiety, but it didn't cause me to fail. I wouldn't classify them as "major problems", just a minor hiccup. Most of the stories above, mine included, are resolved when the proctor resets the computer.

    This may be an unpopular opinion, but I believe a candidate should be prepared for anything and everything. In practice we encounter issues everyday which requires us to think on our feet - be it an RFI, critical path item, consultant coordination, construction error, or even software problems. Have you ever considered this is by design? Its not that there is a software problem, its how you handle it.

    -3
    Comment actions Permalink
  • Avatar
    Victoria Shingleton

    LOL @Peter such an architect answer.

    Public: This building is ugly! And its reflective panels are blinding everyone on the corner!

    Architect: Have you ever considered this is by design?

     

    In all seriousness.... yes, the exam seems to recover just fine. It doesn't seem like any answers are lost. And sure, candidates should be prepared for "anything." But I don't think we should be making excuses for NCARB and Prometric regarding software failure. The fact that software failure is so common and has been for years.... even decades, I'm hearing, is problematic. I do not believe that this exam is designed to shut down to "program failure" by design. I'm not sure if the program failure is due to the software or the hardware it is being run on, but technology is so advanced these days and there is ample availability of skilled professionals who could be hired to solve this problem. It's simply embarrassing for NCARB that this continues to happen.

    4
    Comment actions Permalink
  • Avatar
    Peter Mall

    @Victoria, I think you misunderstand. I am not defending the software. I am just saying that we know issues CAN occur and a candidate should be prepared to NOT let it throw them off. There is so much preparation needed for the exams themselves, I just don't advocate for chasing software issues. Its a classic "bigger fish to fry" scenario. If anyone is overly concerned they can wait until NCARB rolls out online-proctored exams.

    0
    Comment actions Permalink
  • Avatar
    Victoria Shingleton

    Peter Mall, I don't think anyone here is chasing software issues. Maybe that in itself is an issue because I'm certainly not holding my breath that this will ever be fixed. And I have no faith in the online proctored exam and will continue to go to the testing center. At least at the testing center, when your program shuts down, you at least have the Prometric staff to work to fix the issue. If the program fails during at home proctored exams, who do you call.... Ghost Busters?

    There's a big difference between "chasing issues," and bringing to the company/organization's attention that there is an issue.

    Let's say you hire someone from a cleaning service to clean your home. That person shows up, kicks some dust around, and leaves. House is not cleaned. Do you call the company and let them know that you're unhappy with their service? Or do you not bother because you still need your house cleaned, so that's the "bigger fish to fry."

    Or how about you buy a computer. Computer arrives and doesn't work. Do you call the company and let them know the product is broken? Or do you go and buy another computer because you need one that works and that is the "bigger fish to fry."

    In either scenario, you probably do both. Yeah, hire another housekeeper or start cleaning yourself, or go ahead and buy another computer because you need it asap, but that doesn't mean you wouldn't pursue the issue with the company. Really it's in the company's best interest - they should want to know when their product or service is failing the consumer.

    So I don't think anyone should stop testing (assuming their goal is licensure), but I also think it's appropriate to bring this common program failure issue to NCARB's attention, because we all share the common goal of licensing architects.

    0
    Comment actions Permalink
  • Avatar
    Jesse Damrow

    Call me naive but I do not believe software issues should be "expected". There is enough to prepare for given the wide swath of study and reference materials, adding "potential software issue strategies" is not in my future plans. If the exam crashes, and the proctor resets, so be it. You really don't lose time (so they say) except......what if you are mid calculation on a long question? Having to stop and start in that regard definitely puts you at a disadvantage. The more troubling one for me however are the drag and drop issues. There should in NO instance be a question that is "unanswerable" due to a software issue. I am sure there could be a scenario wherein you could miss the cut by one question, and that question wasn't available to answer. If the software is directly responsible for accounting for a "wrong" answer that could have otherwise been "correct"......then that test should be nullified, NO question about it. 

    Not crying over spilled milk here however.......we must continue and do our best with what we've been presented. I just wish they would at least acknowledge these issues instead of treating the community that supports their business as if it is just tough luck and without response.  

    4
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk