The CATS Hackathon: Creating and Refining Test Items for Cybersecurity Concept Inventories

Simple item page
dc.contributor.authorSherman, Alan T.
dc.contributor.authorOliva, Linda
dc.contributor.authorGolaszewski, Enis
dc.contributor.authorPhatak, Dhananjay
dc.contributor.authorScheponik, Travis
dc.contributor.authorHerman, Geoffrey L.
dc.contributor.authorChoi, Dong San
dc.contributor.authorOffenberger, Spencer E.
dc.contributor.authorPeterson, Peter
dc.contributor.authorDykstra, Josiah
dc.contributor.authorBard, Gregory V.
dc.contributor.authorChattopadhyay, Ankur
dc.contributor.authorSharevski, Filipo
dc.contributor.authorVerma, Rakesh
dc.contributor.authorVrecenar, Ryan
dc.date.accessioned2019-02-21T19:53:45Z
dc.date.available2019-02-21T19:53:45Z
dc.date.issued2019-01-29
dc.description.abstractFor two days in February 2018, 17 cybersecurity educators and professionals from government and industry met in a "hackathon" to refine existing draft multiple-choice test items, and to create new ones, for a Cybersecurity Concept Inventory (CCI) and Cybersecurity Curriculum Assessment (CCA) being developed as part of the Cybersecurity Assessment Tools (CATS) Project. We report on the results of the CATS Hackathon, discussing the methods we used to develop test items, highlighting the evolution of a sample test item through this process, and offering suggestions to others who may wish to organize similar hackathons. Each test item embodies a scenario, question stem, and five answer choices. During the Hackathon, participants organized into teams to (1) Generate new scenarios and question stems, (2) Extend CCI items into CCA items, and generate new answer choices for new scenarios and stems, and (3) Review and refine draft CCA test items. The CATS Project provides rigorous evidence-based instruments for assessing and evaluating educational practices; these instruments can help identify pedagogies and content that are effective in teaching cybersecurity. The CCI measures how well students understand basic concepts in cybersecurity---especially adversarial thinking---after a first course in the field. The CCA measures how well students understand core concepts after completing a full cybersecurity curriculumen_US
dc.description.sponsorshipWe thank all of the Hackathon participants. This work was supported in part by the U.S. Department of Defense under CAE-R grants H98230-15-1-0294, H98230-15-1-0273, H98230-17-1-0349, and H98230- 17-1-0347; and by the National Science Foundation under SFS grant 1241576 and DGE grant 1820531.en_US
dc.description.urihttps://arxiv.org/pdf/1901.09286.pdfen_US
dc.format.extent8 pagesen_US
dc.genrejournal articles preprintsen_US
dc.identifierdoi:10.13016/m294ps-kpjq
dc.identifier.urihttp://hdl.handle.net/11603/12846
dc.language.isoen_USen_US
dc.relation.isAvailableAtThe University of Maryland, Baltimore County (UMBC)
dc.relation.ispartofUMBC Center for Research and Exploration in Space Sciences & Technology II (CRSST II)
dc.relation.ispartofUMBC Faculty Collection
dc.relation.ispartofUMBC Student Collection
dc.relation.ispartofUMBC Education Department
dc.rightsThis item is likely protected under Title 17 of the U.S. Copyright Law. Unless on a Creative Commons license, for uses protected by Copyright Law, contact the copyright holder or the author.
dc.subjectcybersecurity assessment tools (CATS) projecten_US
dc.subjectcybersecurity educationen_US
dc.subjectcybersecurity concept inventory (CCI)en_US
dc.subjectcybersecurity curriculum assessment (CCA)en_US
dc.subjectstandardized test designen_US
dc.titleThe CATS Hackathon: Creating and Refining Test Items for Cybersecurity Concept Inventoriesen_US
dc.typeTexten_US

Files

Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
1901.09286.pdf
Size:
144.69 KB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
2.56 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

UMBC Computer Science and Electrical Engineering Department
UMBC Education Department
UMBC Faculty Collection
UMBC Student Collection