Live polls needed during ncarb webinars talking about testing



  • Avatar
    James Martzahl

    This would be an excellent way to make NCARB have to listen to us and actually respond to the data in real time.

    Comment actions Permalink
  • Avatar
    Richard Balkins

    I rather we keep remote proctoring but also keep scratch paper at test centers and maybe also during remote proctoring (until we have digital whiteboard working right and decent enough that we can actually use it but we need a transition time on that that is reasonable). 

    I ALSO agree that we need to have bugs fixed / adequately addressed. NCARB and Zoomorphix Systems should be able to implement any necessary software work needed for remote proctoring with Prometric and PSI's proctoring network/etc. and still also make bug fixes because after all, their software system is already built for web deployment (remote and test center deployment since before ARE 5.0 even existed since they created Exam Studio. In other words, it is already built for web from day 1. I assume NCARB had since ARE 5.0 was conceived for a future plan for remote proctoring which is partly why they CHOSE Zoomorphix's SaaS software system in the first place. If I was hired to do analysis of software bugs, I would have figured out the issue pretty quickly as to what is conflicting like what kind of firewall is running on Prometric test centers because I would reason that their firewall will be a factor in slowing the rate of PDFs loading as well as server bandwidth capacity issues. We are talking about some number of examinees. How many examinees are testing concurrently each day? WoW game server(s) manages a million CCUs. Can the servers handle even 1/10th or even 1 Percent of the throughput at any one time. We are talking about moving data like multi-megabytes size PDF content to the examinee. You'll also now have to consider the webcam traffic in each direction with remote proctoring but it might not impact the exam content server but it will impact the data load between the examinee's computer and TCP/IP or UDP packets' destination(s). Since our network services are going to vary but if we have 100,000 examinees testing each day, it is necessary to handle in excess of the 100,000 CCUs (Concurrent Users = Examinees and remote proctors). Otherwise you get lags and other possible instability issues. In an exam environment like the ARE, you don't want anything that would be perceivable by the examinees that would slow them down like long loading time for PDFs. I see the issues with the practice demo and I have 100 Mbps internet. It is understandable that some people won't have even that speed of internet service and proctoring centers might not have the best performing LAN network and internet service.

    1 Gbps internet service might seem like a lot but when it is bogged down by 250 computers, that internet is going to be diminished like 4 Mbps. Lets not forget that if Prometric or PSI are using cable internet ISP service, it's is likely assymmetrical so upload rate is still ~35 Mbps (regardless of ISP package from 100 Mbps to a "Gig" service package) compared to ~960 Mbps download rate. Then you have computers on the network with a plethora of security software (firewalls, antivirus/antimalware with real-time scanning, etc.), all of which WILL impact the rate in which PDF will load and render in the exam software and overall sluggishness. This will contribute to the less than stellar experiences people have had. We know darn well that Prometric or PSI isn't going to remove their "security" software to make the exams run faster and smoother on their computers. If they aren't all 18 core i9 processor based computers with 128 GB of RAM and have RTX2080 video cards or the newer RTX 3090 video cards with at least 1D/1U Gbps (1000 Base-T Full-Duplex) fiber-optic connection per computer with a network capacity of 1 Gbps (down and up)   or basically 2 Gbps Throughput (symmetrical full duplex)  x number of computers at test center x 150%.... we are likely to have not so stellar exam experiences at the test center especially with their security software running. That's what I would call up to date and up to snuff computers not those i5 quad-cores from 10 years ago with only 16Gb to 32Gb RAM. 

    We can't assume we'll see that at all test centers. If there are bugs or sluggish performance in the ARE 5.(1) then we are going to need commitment to fix the bugs or do things to improve the loading or even image scan quality of some of the content on the exams. I seen less than stellar scans in the practice exam where small prints are pixelated and it is still sluggish in loading. Can NCARB actually explain what's going on there when I can run javascript PDF / Documents viewers that can load better quality resolution documents and do so faster? I am not asking anyone to disclose confidential exam content in particular because the answer to the question does NOT need to disclose those question nor anyone should disclose actual exam items but they can reference anything on the practice demo. 

    The remote proctoring matter is an independent issue to that of software bugs/glitches/etc. The bugs/glitches may still be present regardless of deployment methodology if it rests in the "Exam Studio" platform by Zoomorphix. I have issues with switching from physical scratch paper to "digital whiteboard" without a real transition period. If the digital whiteboard is buggy/glitchy or otherwise not really working well for candidates and they fail because of it, then we have a real problem.

    It is one thing to launch remote proctoring in November 2020 but then switching to digital whiteboard in mid to late 2021 would have been more wiser step and allow time for bugs to be found and tested in the demo exam without putting examinees at unnecessary risk of failing because of bugs because NCARB won't compensate us test takers. At least we would know the change in advance long enough to get proficient with any sort of janky "digital whiteboard". Right now, scratch paper is important and necessary. I recommended some check-in and check-out procedures and use of a paper shredder to shred scratch paper at check out when you are remote proctoring. If you are taking the test at a test center, you won't need to worry about a paper shredder. This suggestion would help address some aspect of test security in a remote proctoring setting. Of course the cross cutting paper shredder should be inspected to be empty before the exam (just like inspecting that the scratch paper is blank with no writing on it) and paper shredder not used DURING the exam and that the scratch paper is shredded after the exam during check out and it all be viewed by remote proctor/check-out through webcam and recorded as needed. It's a basic procedures. 

    It's not 100% cheat proof and no method is. 

    The digital whiteboard issue is something that people need time to get familiar with just like they did with the old vignettes from NCARB 3.x and 4.x era. I know NCARB got criticism for not learning from the old 3.x and 4.x exam. I think I still have the practice exam software from that era. 

    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk