Cut Score timeline

Comments

2 comments

  • Avatar
    Richard Balkins (Edited )

    Ariana, when psychometric consultants talked about cutscores and duration of "standard setting", with regards to certification I was involved with involving exam development (and ultimately any proctored exam on the planet), there is no absolute duration. These are estimates and are subject to longer duration if it was necessary. The estimates are based on statistical data of number of people taking the particular exam (ARE). The key is they need to reach a number of exams taken and therefore scored. They need enough exams taken in order to establish a cutscore for each division. If for some odd reason that only 2-3 people took ARE exams during the time period from November 15th to January 15th, the standard setting would take longer and their scores may take longer than 4 weeks or so to be graded and given either a pass or fail. It is extremely unlikely that there will only be so few test takers to take the ARE exam divisions that they can't establish a cut score and require a longer time frame. 

    It is very likely that they'll be able to establish a cut-score and determine a pass/fail for tests taken in a 4 to maybe 6 week time frame. Within 8 (10 at most from a realistic perspective) weeks of those exams, you SHOULD know if you passed or failed. 

    They have to gather cut scores for each division not just one because it is 6 proctored exams that makes up the ARE. From prior instances with NCARB and other proctored exams as a gauge, you may see 4-10 weeks (delay) for ARE exams taken between November and March before you get your pass/fail results. Some divisions may take a longer because there maybe a significant drop of people taking certain exam divisions for a period of time until they get their head around the digital whiteboard. Just be prepared for possibility that it takes longer because anyone rushing right away to take some of the exam divisions, they are highly likely to perform horribly because of trying to work with something that is likely to be very janky and frustrating to work with.... with a mouse. 

    NCARB has the reserved right to not issue pass or fail and keep the standard setting going per division as long as it takes. So a significant drop in exam taking of certain divisions during November/December can cause scores on those divisions to take longer. 

    Cut scores are derived through statistical sample. If sample size is too small, they wait until enough exams are taken in the respective divisions to meet sufficient sample size. It's a mathematical thing. 

    Be prepared and I do hope you don't have to wait longer than the 4 weeks. Be prepared to expect the next 3 or so months from November 15 to February/March 16th to be slower than usual. Each division needs a sufficient sample size of tests taken in that division for a cut score to be established for that division. 

    NCARB may not have stated this as something they are expecting but I'm confident behind closed doors that they are especially with the misgauging the response from us regarding changes proposed like the scratch paper & pen/pencil controversy. This isn't the first time NCARB dropped the ball and underestimated in their public estimates. I've known these folks for well over a decade so I don't want you being surprised if they miss the 4 week marks and you haven't got your "pass" or "fail" results. 

     

    0
    Comment actions Permalink
  • Avatar
    Nick NCARB

    Hey Ariana,

    I can confirm that 4 weeks is a rough estimate, but ultimately the timeframe depends on the number of testers per division. To appropriately set a new cut score, NCARB needs at minimum 400 exam administrations per division. What this means is divisional cut scores will be set as they meet that 400 threshold. As an example, if more testers are taking PA and CE than PCM, those divisions will have a new cut score before PCM. 

    After the launch of the ARE with online proctoring, we will keep everyone on the community updated on the current administration numbers. 

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk