WRITING CERTIFICATION STYLE QUESTIONS:
A PRIMER FOR NP FACULTY
GARY LAUSTSEN
Nurse practitioner (NP) faculties prepare advanced practice nurses (APRN) for careers in clinical practice, interprofessional collaboration within healthcare systems, and other professional roles. In many states, before obtaining an APRN license, students must successfully complete an accredited educational program and take a national certification examination.
The predominantly computerized NP certification examinations are developed by various certification organizations such as the American Nurses Credentialing Center (ANCC) or the American Academy of Nurse Practitioners (AANP). These organizations recruit NPs active in clinical and/or academic practice to create psychometrically sound questions (also known as items) that are further peer-reviewed and then incorporated into the national certification examinations. To promote student success in passing the certification examination, NP faculty should seek to develop and incorporate their own certification-like items into the academic NP curriculum. The purpose of this article is to present a primer on certification examination item writing to facilitate NP faculty seeking to develop their own certification-like test questions. The content of this article is partly derived from item writing training sponsored by ANCC.
Why Test?
When considering the writing and inclusion of certification-type test items in a NP course or program it is helpful to “begin with the end in mind”. The faculty should ask: What result do I want from my testing of the student? Tests can be developed for a number of unique or multiple purposes. Tests may be independently or concurrently utilized for evaluating students, course content, delivery methods, or even the competencies of the entire NP program.
In considering the types of student evaluation methods available, faculty should consider: Is a written test the best way to evaluate the desired student outcomes and why is the test being given? In most cases the primary goals of administering tests are to evaluate student knowledge/content (memory) or the student’s reasoning or analytical abilities. Secondary goals of the test could be to evaluate and compare the NP students’ test results to national standards and/or to evaluate the students’ meeting of program competencies and benchmarks (Downing, 2006).
What is your test measuring?
Tests can have a hierarchical range of items that measure different levels of cognitive ability and these can be succinctly divided into testing either the student’s memory or their reasoning ability. Test items that ask the student to recall, interpret, or memorize content, facts, concepts, or principles are characteristic of evaluating memory ability. Test items that require the student to demonstrate application, analysis or organization of knowledge, or complex decision making are characteristic of evaluating student reasoning ability (McDonald, 2002).
Test Item Writing Basics
Breaking item writing down into discrete steps helps the NP educator to work through the process of developing certification style questions or item. In this next section, the elements of an item will be reviewed and then a discussion of the five steps for writing test items will be explicated (Osterlind, 1997).
Computerized certification examinations may contain a variety of question formats however; the, majority of items are presented in a multiple choice format with one correct answer. Development of this multiple choice-type item format will be the focus of this article. However, students should be aware that examinations may include alternative format question that may be:
- Multiple-response items (more than one correct response)
- Fill-in-the-blank items (typically calculation-type items)
- Hot spot items (test-taker selects an area on a picture or graphic)
- Chart/exhibit format (interpretation of information in the chart or exhibit format)
- Ordered Response items (options are rearranged into a specific order)
It is first helpful to review some key concepts in question or item writing. The basic unit of the multiple choice test is typically called an “item” rather than a question. A multiple choice test item contains three unique elements:
1) Stem: The stem asks a question or states a problem that needs to be solved.
2) Key: The correct answer or solution to the stem is called the key.
3) Distracters: The other possible, but incorrect responses to the stem, are called distracters. The key and the distracters are also referred to as options.
Steps for Writing a Test Item
The writing of a test item can be broken down into five steps and each step will be discussed and supported with an example.
1. Selecting the topic/content to be evaluated
2. Determining the desired item’s skill level (Content or Reasoning)
3. Writing the stem
4. Writing the key (correct response)
5. Writing the distracters (incorrect options)
The first step in writing a test item is to select the topic or content the faculty wishes to be evaluated in the student. This can be derived from course readings, discussions, lectures, or presentations, clinical or laboratory experiences, or any other educational interventions. An example of this might be derived from a textbook in an advanced pharmacology course.
Hypertension affects all ethnic groups, but African-Americans suffer disproportionately from hypertension and its effects…Hypertension in African-Americans occurs at an earlier age, is more severe, and results in organ damage such as coronary heart disease, stroke, and end-stage renal disease more often than it does in whites. (Arcangelo & Peterson, 2006, p. 197)
In the second step, the faculty determines whether they wish to have the student demonstrate memorization of content/knowledge or the demonstration of reasoning through analysis or application. When constructing the item, the use of specific words can help delineate whether an item is evaluating the student’s content recall or reasoning ability. Table 1 lists words often used in constructing memory and reasoning items.
Table 1. Examples of words for constructing memory and reasoning items
|
Memory |
Reasoning |
|
Define Describe Identify List Recall Recognize Reproduce
|
Apply Analyze Anticipate Assess Compare Contrast Decide Demonstrate Extrapolate Infer Prioritize Synthesize Translate |
In the third step, the faculty considers the content to be evaluated and the level of the item (content or reasoning) and writes the stem. The stem is probably the most important part of the item as it provides the content, scope, and format for the item. In general, it is best to include information in the stem that is relevant to the item. However, in actual clinical practice, during patient encounters, the NP acquires non-relevant data and must learn to identify what assessment data is relevant to the diagnosis and treatment of the patient. Therefore, the faculty item writer may decide for clinically-based items, to include non-relevant data as a part of the goal to evaluate a student’s ability to “sift” important data from extraneous data.
When writing the stem, strive to create clear, grammatically correct, and unambiguous stems. The stem should be succinct and limited in scope. The stem can either be written as an incomplete sentence or as a question. Using the pharmacology content previously identified, an example of a content (memory)-type stem might be:
“When caring for patients from different cultures, the NP is aware that:” Using the
same content an example of a stem using higher level reasoning might be:
“A 50 year-old African-American male with a 10-year history of moderately
controlled hypertension presents to the clinic for routine care.
Assessment data for this visit include:
BMI=30, BP= 142/88 mmHg, P=74, R=16,T=98.2⁰F,
Complete Blood Count (CBC) within normal limits
Basic Metabolic Panel (BMP) within normal limits except
for a BUN of 60mg/dL and a serum creatinine of 2.0 mg/dL.
What is important for the NP to consider in creating a treatment plan for this patient?”
The fourth step item in item writing is to construct the key, otherwise known as the answer. The key or answer needs to be defensible, standard of practice, evidence based content, or content that is agreed upon by experts. The key should not be based on anecdotal or personal opinion. For the first stem the key could be: “African-Americans are at increased risk of hypertension due to salt sensitivity.” For the second stem example the key might be written as: “In African Americans with hypertension, renal disease occurs at an earlier age and is more severe than whites.”
The fifth and final step in writing certification examination-like items is to create the distracters. Distracters are the incorrect responses and should “distract”. They are designed to help the faculty identify knowledgeable or good reasoning students from students guessing or unsure of the correct content. When constructing distracters, they could be a “correct” response, but not necessarily the “best” response. All distracters should be written similarly to the key in length, style, and wording. They can come from class comments, clinical misconceptions, or common misinformation. Well-developed distracters should be believable to help identify the student guessers. It is inappropriate to write “cutesy” distracters, that clearly are incorrect and do not help determine student competency. The distracters and key should be listed in alphabetical order.
Again using our previously developed stems and keys, for the first item, examples of distracters might include:
· Asian-Americans are at increased risk of death from heart disease.
· Hispanics are at increased risk for suicide due to cultural stress.
· Native Americans are at increased risk for Type 1 diabetes due to diet.
For the second item example, possible distracters could be:
· In African Americans, hypertension is commonly due to a genetic susceptibility for developing renal disease.
· In African Americans Type 2 diabetes are common sequelae of chronic hypertension.
· In African Americans with hypertension, they respond to antihypertensive medications similarly to whites.
In Table 2, the stem, key and distracters are put together to demonstrate the completed certification examination-style item.
Table 2. Complete item examples
|
1. When caring for patients from different cultures, the NP is aware that: a. African-Americans are at increased risk of hypertension due to salt sensitivity. b. Asian-Americans are at increased risk of death from heart disease. c. Hispanics are at increased risk for suicide due to cultural stress. d. Native Americans are at increased risk for Type 1 diabetes due to diet |
|
2. A 50 year-old African-American male with a 10-year history of moderately controlled hypertension presents to the clinic for routine care. Assessment data for this visit include: BMI=30, BP= 142/88 mm Hg, P=74, R=16, T=98.2⁰F, Complete Blood Count (CBC) within normal limits and Basic Metabolic Panel (BMP) within normal limits except for a BUN of 60mg/dL and a serum creatinine of 2.0 mg/dL. What is important for the NP to consider in creating a treatment plan for this patient? a. In African Americans, hypertension is commonly due to a genetic susceptibility for developing renal disease. b. In African Americans Type 2 diabetes is a common sequela of chronic hypertension. c. In African Americans with hypertension, renal disease occurs at an earlier age and is more severe than whites. d. In African Americans with hypertension, they respond to antihypertensive medications similarly to whites. |
The second item is an example of using a clinical vignette or case. These types
of items are useful for evaluating a student’s ability to analyze, reason, or
apply their knowledge to real patient scenarios. When writing scenario-type
items, the case should not be too lengthy and should also be a situation similar
to what is encountered in actual practice. On the certification examination,
each item stands alone. However, faculty in their course testing may create one
case presentation and use it for developing more than one item. Resources that
may be useful and found online are the Items Writer Guide from ANCC as well as
sample test questions that may be found online at the following web address http://www.nursecredentialing.org/SampleQuestions.aspx
as well as at the National State Council Boards of Nursing (NSCBN) (2009)
online program to assist with test item writing at http://www.learningext.com/products/assessmentstrategies/introcourse/assessment.asp.
Conclusion
Like any worthwhile pedagogical endeavor, good test or item writing is a skill that requires time, practice, reflection, and refinement. This article has outlined, discussed, and presented some examples of the steps faculty can take in creating test items that are similar to the style of questions students may encounter on certification examinations. NP faculty can advance in their own awareness and understanding in test item writing to promote student success in passing national certification examinations.
About the Author Gary Laustsen, PhD, APRN-CNP(Fam), is an Associate Professor at Oregon Health & Science University, School of Nursing. He is currently the interim Director of the FNP Program and the DNP Rural Health Track. Contact Dr. Laustsen at laustsen@ohsu.edu
References
American Nurses Credentialing Center (ANCC). (n.d.). Item Writer’s Guide.
Washington,
DC: Author. Retrieved from
http://www.nursecredentialing.org/SampleQuestions.aspx
Arcangelo, V.P., & Peterson, A.M. (2006). Pharmacotherapeutics for advanced practice:
A practical approach (2nd ed). Philadelphia: Lippincott Williams & Wilkins.
Downing, S. M. & Haladyna, T. M. (2006). Handbook of Test Development. Routledge.
McDonald, M. E. (2002) Systematic assessment of learning outcomes: Developing multiple-choice exams. Boston: Jones and Bartlett.
National State Council Boards of Nursing (NCSBN) e-learning Test Development and Item Writing V4.0. (2009). An Online Self-Paced CE Course for Nursing Faculty. Retrieved from http://www.learningext.com/products/assessmentstrategies/introcourse/assessment.asp
Osterlind, S. (1997). Constructing Test Items: Multiple-Choice, Constructed-Response, Performance and Other Formats (2nd ed.). Springer.