The Interface

Comments

18 comments

  • Avatar
    Antonio Ribeiro Cardoso

    Hi Barry,

    I have to agree with you. The interface is terrible and the lag while loading the pdf's makes no sense in this day and age. I avoided using as much as possible because I knew they would just take too long to load and scroll through. Even while moving between one question and the next it took too long.

    These delays add up, and the test is extensive, meaning we end up losing precious time.

    Regarding the study cases, the horizontal split of the screen makes reading the reference materials and the questions. In my first case study question, I didn't even realize there was a question below the reference materials. A vertical split would be much better.

    Also, I had at least one question where I had to pick the correct detail out of 4 and the images were barely readable. There is no zoom function in those questions. One of the details was very different but the remain ones were really similar and hard to read on the screen.

    On a separate note, and at least on my exam center, access to water was only given during our 15 minutes break. Not to mention the military facility like search we are subject to access the room. That includes when we come in and out of break. I think at the very least water should be present in the room or accessible, otherwise, this becomes not just a mental exercise but also a physical endurance one.

     

  • Avatar
    Ryan NCARB

    Hey Barry and Antonio,

    It is unfortunate you both are so disappointed with the new exam interface.  I could spend time telling you about all of the user and functionality testing with real candidates long before launch, but I don't think you are interested in hearing it.  I will just remind you of two things.  First, the ARE 5.0 Demonstration Exam is available to you through your NCARB Record to be able to practice all of the tools and become familiar with the interface.  The demo exam is the exact same version as is used in the test center.  There are also several videos that have been posted with tips on navigation.  Second, ARE 5.0 is delivered over the internet, so each time you advance to a new question it is being loaded.  Those delays between questions have been accounted for in the overall testing time window for each division.

    As for your comments about the ARE 5.0 Handbook not addressing anything that is on the exam, that is simply false.  The sections, objectives, cognitive complexity, and content percentages is exactly what is use to compile each form of the exam in each division.  A lot of folks on here have found it very helpful, so you may want to ask others how they best used it.  And yes, there are a range of easy and difficult questions - that is part of the assessment.

    Hope this helps.

  • Avatar
    barry ballinger

    I studied the handbook, watched the videos, and took the demonstration exam. I could describe specifically how the actual exam differed significantly and how none of those were adequate preparation for the exam, but the forces of NCARB would punish me. However, I will stick to the interface. I'm not sure what methodology you used with real candidates. Having to refer to several documents, while looking through a small viewer, while the document zooms inconsistently, and while the clock was rolling...anyone should understand this would be a significant problem. I'm not talking about the lag. It's trying to pan and zoom through multiple documents.

    Participants will often tell researches what they want to hear in focus groups or behavior studies within controlled environments. That's why many researchers choose to do research in natural settings. Yet, here are candidates in an actual settings, telling you what you don't want to here, and you seem dismissive of our experiences. 

  • Avatar
    Antonio Ribeiro Cardoso

    Hi Ryan,
    My intent was to leave honest feedback of my experience and you seem to have been somewhat dismissive of it. The interface has issues that should be addressed, and although some might not have had the same experience, I believe there are others, like myself that believe there is room for improvement.
    The lags seem to either be a broadband, coding or equipment issue and all of those can be addressed. Also, I can assume that not all the test centers have the same size monitors and if some of the details are perfectly readable (and I'm referring to the ones in the questions, not the reference materials) in other cases, like mine it can be really hard to read, especially when they barely differ from each other. A zoom function for these would be extremely helpful in mitigating any issues related to image size or resolution.

  • Avatar
    Ryan NCARB

    Hey Guys,

    I didn't mean to seem dismissive of your feedback at all!  We are definitely not.  We have noted your comments and concerns and look at all of them.  I was just hoping to convey that the ARE 5.0 interface wasn't created in a vacuum and that some things are just limitations.

    One other thing I'll add Antonio, ALL Prometric test centers do have the same size monitors (23" HD), internet requirements, and equipment standards.  That consistency and fairness is one of the main reasons we continue to deliver through Prometric.

    And Barry, "the forces of NCARB" never punish anyone.  We simply assess and score all candidates exactly the same way.  Fairness is a really big deal.

  • Avatar
    Stephen Parra

    Hey Ryan,

    I will say I greatly appreciate your diligent effort to lend transparency to the transition process with your administration updates table and responses to member comments. Admittedly the ARE 5.0 user interface (UI) is a tremendous improvement over the 4.0 and those particularly heinous vignette programs we had to suffer through – good riddance.

    I would suggest the UI issues including some of those mentioned in other comments above are not limited to the ARE 5.0's UI but are also probably due to prometrics hardware capabilities.

    For instance computers at my PDD testing site had the click sensitivity set so high that it was virtually impossible to answer a multiple choice question in just one click or to use the calculator at all without duplicating or triplicating functions. The disabled number pad on ARE 5.0 was the insult to this injury that made the calculator unusable. I was relocated to another station which  also had the same issue.

    Additionally, some of the drawings in the hot spot questions were almost impossible to read absent a zoom feature comparable to the one available in the case studies. My computer froze during an interactive question necessitating a restart.

    I trust you can see how issues like these can eat away at candidates psychologically over the course of a 4.5 hour exam. In the aggregate they undoubtedly hinder a candidate's performance. I raised these concerns with customer service and was dismissed simply because I had managed to "complete" the exam. 

    If fairness is important, it seems unfair that out of the 600 exams which will set the cut score, at least one of those was "completed" without the benefit of the full functionality of the tools NCARB leads us to believe we can rely on in the exam room.

    I raise these issues here not to simply complain further, but to lend credence to the general argument that the ARE 5.0 and Prometric's machines need further tweaking!

    Thanks!

    Stephen

     

  • Avatar
    James Glenn

    I am fresh from taking the PDD exam, and have to agree with some of the items noted above. Specifically the inability to zoom in on "hotspot" questions; which meant I couldn't read what the plan, site plan, or elevation was showing and basically guessed to the best of my knowledge. It seems these could have been composed better at least for legibility as opposed to being a simple scanned image.

    My testing station logged out of the test partway thru and had to be restarted; which is disturbing to say the least.  find it hard to believe Prometrics cannot avoid simple issues such as that.

  • Avatar
    Richard Wilson

    Completed exam #2 under ARE 5.0 - and the user interface problems persist with absolutely no changes to the failed delivery system. Here's a link to my first critique of the UI - which I won't repeat here.

    Come on NCARB- the UI is awful!!! I agree with what others have said above: I am discouraged to use the resource materials, the calculator, the case study scenario information, or mark questions to revisit due to extreme time constraints. I am also discouraged from taking any more exams until I receive confirmation that NCARB has made major UI fixes, and has posted a new sample exam for us to test.

    The kinds of problems we have been presenting to you, on these community boards, should not be difficult to correct in today's age of prevalent computer programmers.

    NCARB has recently released Free Re-takes for early testers. However, how can the NCARB organization expect to produce valuable cut scores if those scores were generated off of a testing program that knee-caps the user? And reduces tester study time and effort?

  • Avatar
    Jared NCARB

    NCARB has been following up with our vendors regarding the UI issues you and others have noted. I would love to be able to promise the ability to have everything tweaked and re-tweaked immediately but that is simply not the reality of implementing changes across a multi-vendor system. We will keep listening and working to make ARE 5.0 better and I am happy to hear those of you with 4.0 experience feel 5.0 is an improvement from the legacy technology.

    In response to some of the concerns about the interface and its impact on setting the cut score, please note that the interface and time constraints you have all faced will be experienced by all going through the cut score process. One key element of setting a cut score is fairness. Every cut score panelist that is involved in determining the passing threshold will experience the exam using the exact same interface and under the same time constraints as each of you. They will be taking this into consideration throughout the cut score process.

    A note regarding providing NCARB feedback on item specific content. Thank you all for respecting the confidentiality agreement and not posting sensitive item data on the community. But, if you want provide item feedback, you can do that through email at are@ncarb.org  Specific item feedback allows NCARB staff to better educate our item writers on what is and is not working well from a candidate perspective.

  • Avatar
    Joseph Miller

    Jared, or anyone else. I just came back from testing where I lost connection to the exam. So Promtetic restarted the computer, no connection. Moved me to a different computer, no connection, restarted the second computer, no connection. They told me to go home and that NCARB would be in contact. I contacted NCARB and they said they knew of the issue and are working to resolve it. It sucks, but I understand this has been happening. 

    My question is to people who have had a similar experience where they were testing and due to technical issued were unable to continue testing. How does NCARB solve this?

    I had 2 hours and 20 questions left, but I can see NCARB saying that I am unable to pickup at question 100, because I had "access" to the whole exam. Can they see what questions were opened? I am avoiding study material until they get back to me, in the hopes that I can continue the exam where it dropped me. 

  • Avatar
    Jesse McConnell

    I had the same issue on the same test today. Prometric staff told me that I would have to start over from the beginning. And prometric will give me a call within 24 hrs to reschedule my exam. This info was from prometric not NCARB.

  • Avatar
    Jared NCARB

    Joseph & Jessica, sorry about the technical problem you both experienced. There was a system-wide issue that impacted all ARE candidates at the same time.

    You will be contacted by Prometric to reschedule your appointment as soon as possible and at no charge. You will be given a different form of the exam on your retest so you will have the full exam again. We don't allow people to pick up from where they left off as candidates could certainly go back and change answers to previously answered questions.

    You don't need to avoid study material before your retest. Your free retest will be a completely new test event.

  • Avatar
    Joseph Miller (Edited )

     

    Jared, So to be clear, are we still are considered part of the 600 if we have to reschedule and don't make the cut?

  • Avatar
    James Glenn

    Jared, has the issue been corrected at this time? I have a PPD exam tomorrow morning and want to be sure this is not an ongoing issue.

  • Avatar
    Autumn Hillis (Edited )

    Hi Jared, I also encountered the same problem today, and unfortunately I don't have the same feelings as Joseph. I do not see this as the ability to "retest" for free, because I was never actually able to complete my exam that I paid for in the first place. I understand difficulties can arise in this early tester phase, so I was patient with the technical difficulties and even minor delay, but then became frustrated that I had to reschedule.

    I am sure most candidates would agree with me that carving at least 5hrs out of a workday for each test is no easy feat, and the financial burden of these missed hours can really add up. I now have to take at least 10hrs total off from work without pay (or wait a very long time for Saturday availability) to "retake" my exam due to no fault of my own while candidates who fail the exam are also being allowed this privileged?

    It may sound petty to some, but this means my paycheck will be a few hundred dollars short, all the while I am being told to reschedule a test before being told what the problem was or being assured that it has been fixed and wont happen again. Hoping for some resolution on this issue!

  • Avatar
    Jared NCARB

    James and Autumn, the issue experienced at test centers early today has been resolved. Candidates are currently testing in Prometric test centers across the country.

    Autumn, NCARB does understand the commitment of taking time away from work and/or family to test. We also understand your frustration of having been at the test center and then having to reschedule. The problem experienced was not due to anything related to being an early tester but was a system issue that unfortunately impacted all ARE candidates.

    When contacted by Prometric to reschedule, please discuss your desire to get a weekend appointment to avoid having to miss any additional paid work. Please note, by testing today you are guaranteed the early tester incentive regardless of when your rescheduled appointment takes place. Please do not feel that you have to rush back into test if that does not work for your schedule. 

  • Avatar
    Caroline Fernandes

    @NCARB,  please reply to this on the thread that is about this problem.  

    https://are5community.ncarb.org/hc/en-us/community/posts/115000646647-TESTING-CENTER-Connection-Error-Not-Able-to-Test

    There are a lot of people (including myself) who were affected by this, upset, and without information that makes any sense.  When I just spoke to NCARB on the phone, I was told that it was NCARB who will be contacting me to reschedule.  Prometric has already told me that NCARB needs to update my eligibility for me to be able to reschedule.  And here you say that Prometric will be contacting us.  Please clarify who and WHEN this will be happening -- "soon" is not a good enough answer...

  • Avatar
    Roberta Militello (Edited )

     

    I recently transitioned to ARE 5.0 with the hope that the recent UI updates would significantly improve my testing experience (2 exams left.) In past 4.0 exams, I could barely read the information on the vignettes, and the program froze multiple times while zooming and panning. I appreciate the enormous effort that NCARB has made to improve the exam – the practice questions seem better aligned with the profession and many of the new question types are an improvement. But, the 5.0 demonstration exam has me worried that critical issues have not been addressed. 

    The lack of a zoom function in the main body of the question (illegible drawings etc.), low fidelity monitors in the exam facility, and delays while waiting for resources to load should be a top priority for NCARB. In modern work places, high fidelity monitors combined with scrolling zoom-wheel mouses are commonly provided for those in need of extra magnification and improved clarity. Being unable to properly access (and see) materials is nerve wracking, time consuming, and ultimately amounts to bias. Please make this a top priority. 

Post is closed for comments.

Powered by Zendesk