Situational Questions

Comments

7 comments

  • Avatar
    Scott Barber

    Hey Nathaniel,

    I think this applies to more than just the CE exam (haven't taken that one yet so I'm not sure if there's something significantly different than other exams for these questions), but here's my initial thoughts. I haven't thought about it too much so I'm sure my logic can use refinement...

    The exams aren't meant to be teaching tools, but are meant to evaluate that we are qualified to be a license architect (most importantly, that we can protect the health, safety, and welfare of the public). In order to determine if we can do this, we have to be able to apply our knowledge on a daily basis to various situations and problems. Of course every situation is different, but if I know the definition of a vapor barrier without being able to know where it needs to be placed in a wall section (based on simple criteria), it can cause significant damage to a wall assembly and lead to mold and other issues for the client. 

    I think to your point, the frustration is that these situational questions appear to be so specific and narrowly-focused...unfortunately, I don't know of a way they can adequately determine our ability to apply this knowledge without a specific scenario. It may be a common situation or broad enough to be applied to an entire region (i.e., what building form and fenestration is ideal for a hot and humid climate), or it may be much more specific that we may not ever face that scenario again in the "real world."  Either way, I would think NCARB has to ask more specific questions to fully understand our ability to apply the knowledge, even if it's not a common scenario (though I don't remember any questions in PA, PPD, or PDD that were way out of left field and not relevant to the "real world")

    In my experience, most analysis tools (homework, exams, quizzes) were also used as teaching tools (receiving feedback that outlined mistakes/errors). I think that's why it may be difficult to differentiate between the ARE and previous exams, which were used as teaching tools. Throughout our education, exams were returned with feedback so we can learn and move forward. But most licensure exams that I know of are meant to analyze your knowledge, not teach you new information.
    However, with the ARE there's only a certain number of questions, so if they told you exactly what question you got wrong, you would eventually be able to study those questions rather than the content on a general level. I had a couple instances in school where a professor would give a "practice exam" before the actual exam. The questions were the exact same and he gave us feedback on the practice exam so that we could learn from our mistakes. However, many took advantage of that approach and just studied the practice exam questions and didn't learn anything new about the content. True, people who choose shortcuts like that will likely not be successful long-term, but it would decrease the effectiveness of the ARE if people were given a list of questions after failing the exam.

    Does that make sense? I'm not sure the full context behind your question but my comment is in response to the frustration I've seen others have recently. 

    0
    Comment actions Permalink
  • Avatar
    nathaniel hocker (Edited )

    Scott,

     Thanks for the feedback. I don't know if I agree entirely, though appreciate the response. I would never ask it to be easy, as that would defeat the purpose of it being a test. I do stand, however, with my initial response and thought(s), the scoring is terrible. And, I do agree with you, the specificity of some of these questions are quite narrow.

    I also agree that, none of the exams I have taken thus far, have been solely the subject matter being tested on. I guess this keeps us "on our toes", though, does make it hard to re-study for a potential FAIL.

    0
    Comment actions Permalink
  • Avatar
    Scott Barber

    I'm glad you don't agree entirely! :) I'd be surprised if my initial rambling of thoughts was fully coherent and accurate, haha. I didn't get the impression that you wanted it to be easy (and I'm sorry if that seemed implied, not my intent), I was just explaining the abuse of this approach that I've seen before.

    What type of scoring do you think would be more helpful? Under 4.0 it looks like there were only three levels (pass, fail, or extra fail), so at least the 5.0 score report has 4 levels of success...but I do think a percentage could be more beneficial - is that what you had in mind? I'm curious what concerns NCARB has to prevent them from doing that. I would imagine someone failing by 1% would be extremely frustrated or discouraged (or maybe overly confident that they shouldn't study much for the next exam?), and someone who only got 1% right could give up on the ARE completely. Who knows?

    I'm torn about the subject matter that 'leaks' into other exams...in one way, it lets you study for multiple exams at once (which I like, personally), but it also makes it difficult if you only study for one exam (which a lot of people do). I understand it to an extent, because in practice we'll have to consider issues that aren't directly related to the phase we're in, but it can make these exams difficult to be fully prepared for. I haven't found many questions that I thought were outside of the scope of the specific test I was taking (and when I did it was some overlap between PDD and PPD, which already is a blurry distinction)...hopefully nobody fails an exam because of one of these questions. 

    0
    Comment actions Permalink
  • Avatar
    Derek Wendt (Edited )

    Yea, I think somewhere on NCARB mentioned how the questions are scored. If I can interpret this correctly, it sounds like they weighted the questions by how difficult it is to answer for the general test-takers? and if they ask you questions that throw you off, there might be a low score for this because it shouldn't affect your entire exam score greatly.

    I've taken few 4.0 in the past and always hated the questions NCARB threw in the exams that were not content-related. Like an architectural history question came up at least once or twice - how does that make me a better architect than a historian? I was baffled by these types of questions because quite frankly, I don't care and perhaps most of our clients may not even care.

    So then I'm going fresh with the 5.0 and thankfully, I didn't get any questions that were out of principles, like those history questions. But there have been questions that are like over-arching in a way. There's always a little bit of something from other exams but I don't believe I've had more than 2-3 questions. It's okay if you didn't know the answer because it's really HARD to study all the exams at once before taking them all. Do your best at answering these situational or "unrelated to exam content" questions because I believe they are low weighted points.

    I think it's really easy to fall into the anger trap thinking you have been scammed because of these questions. Or thinking "why the heck they ask me these questions from the exams I HAVEN'T study for yet." It would make sense to have questions that focus solely on its topic, but, again, there'll be out-of-topic questions that will have a low weighted score (if I'm right). It's frustrating, but don't fret or give up. Look at the areas you didn't do well and understand why. Read those study areas more deeply and slowly.

     

    0
    Comment actions Permalink
  • Avatar
    Scott Barber

    Good thoughts, Derek. I know that all questions are worth one point, but the percentage required to pass varies for each exam (and I think each version of the exam), so it could be that the "weight" of each one is factored in based on the percentage. I'm glad 5.0 has been better for you - I didn't take any under 4.0 but from what I've heard the 5.0 exams are a lot better overall. 

    0
    Comment actions Permalink
  • Avatar
    nathaniel hocker (Edited )

    Derek,

    All questions are worth one point.

    I agree with you on the historian point of view; in fact, I chuckled at that. I am not angry at all, personally, but that is good advice for anyone reading it. My first fail, most assuredly, made me angry.

    Good feedback!

    0
    Comment actions Permalink
  • Avatar
    Michelle NCARB

    Hi all -

    So a few things here.  First things first, Nathaniel and Scott are correct: every scored question on the ARE is worth one point.  The questions are not weighted for difficulty or any other factor.  

    Scott's also correct that number required to pass - or the cut score - varies for each form (version) of each division.  That's why the score report doesn't include a percentage score: it wouldn't be useful information.  My percentage score might be higher than Ryan's percentage, but what if Ryan had a more difficult version of the test?  His required percentage to pass would be lower.  All forms are balanced for level of difficulty and timing to ensure fairness for all testers.

    Derek, you also mentioned "if they ask you questions that throw you off, there might be a low score for this".  You might be thinking of pretest questions, which are included in every test as a way of collecting statistics on newly written questions.  They’re distributed randomly throughout the exam but are unscored and have no impact on your overall results. Since you won't know which questions are pretest, it's always best to answer every question as if it's being scored.  This is standard practice within the testing industry and helps ensure the questions are fair and reasonable.

    And as far as Derek's original question: take a look at pages 7-8 in the ARE 5.0 Handbook.  Understanding cognitive complexity and how that's used on the ARE might help as you approach the various questions on your test.

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk