Evaluating Automatic Malware Classifiers in the Absence of Reference Labels

Simple item page
dc.contributor.advisorNicholas, Charles
dc.contributor.authorJoyce, Robert j
dc.contributor.departmentComputer Science and Electrical Engineering
dc.contributor.programComputer Science
dc.date.accessioned2021-09-01T13:55:38Z
dc.date.available2021-09-01T13:55:38Z
dc.date.issued2020-01-01
dc.description.abstractThe malware analysis community is completely devoid of a diverse, up to date reference dataset with ground truth labels. Consequentially, it is typical for automatic malware classifiers to be evaluated using custom datasets with near ground truth labels. However, classifier evaluation using near ground truth labels can yield erroneous or biased results. We propose an alternative classifier evaluation framework that does not require reference labels. We introduce the concept of a ground truth refinement and propose potential methods for constructing an approximation of one from a malware dataset. We prove that using a ground truth refinement it is possible to compute lower bounds on precision and error rate as well as upper bounds on recall and accuracy without requiring ground truth reference labels. We perform a case study on the popular AVClass malware labeler using our proposed evaluation framework.
dc.formatapplication:pdf
dc.genretheses
dc.identifierdoi:10.13016/m2xtjd-7oqz
dc.identifier.other12166
dc.identifier.urihttp://hdl.handle.net/11603/22879
dc.languageen
dc.relation.isAvailableAtThe University of Maryland, Baltimore County (UMBC)
dc.relation.ispartofUMBC Computer Science and Electrical Engineering Department Collection
dc.relation.ispartofUMBC Theses and Dissertations Collection
dc.relation.ispartofUMBC Graduate School Collection
dc.relation.ispartofUMBC Student Collection
dc.sourceOriginal File Name: Joyce_umbc_0434M_12166.pdf
dc.subjectClassifier Evaluation
dc.subjectData Science
dc.subjectMalware Analysis
dc.subjectMalware Classification
dc.titleEvaluating Automatic Malware Classifiers in the Absence of Reference Labels
dc.typeText
dcterms.accessRightsAccess limited to the UMBC community. Item may possibly be obtained via Interlibrary Loan thorugh a local library, pending author/copyright holder's permission.
dcterms.accessRightsThis item may be protected under Title 17 of the U.S. Copyright Law. It is made available by UMBC for non-commercial research and education. For permission to publish or reproduce, please see http://aok.lib.umbc.edu/specoll/repro.php or contact Special Collections at speccoll(at)umbc.edu

Files

Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
Joyce_umbc_0434M_12166.pdf
Size:
331.12 KB
Format:
Adobe Portable Document Format
Download

Collections

UMBC Theses and Dissertations
UMBC Computer Science and Electrical Engineering Department
UMBC Graduate School
UMBC Student Collection