Main
Dr. Abhik Roy
Education
Western Michigan University
Ph.D. in Program Evaluation
Kalamazoo, MI
N/A
Dissertation. Building an Evaluation Model of Academic Advising’s Impact on Progression, Persistence, and Retention Within University Settings
Michigan Technological University
M.S. in Mathematics
Houghton, MI
N/A
Thesis. Quotient Rings of the Eisenstein Integers
West Virginia Wesleyan College
B.S. in Mathematics
Buckhannon, WV
N/A
Terminal Study. 4-Cell Embedding on a \(n\)-genus Torus
Professional Experience
Associate Research Scientist
Indiana University
Bloomington, IN
Current - 2024
- Applying Advanced Research Techniques. Utilizing both quantitative and qualitative research methodologies to design studies that accurately address complex social, behavioral, and policy questions.
- Conducting Comprehensive Data Analysis. Employing statistical software and qualitative analysis tools to extract, process, and interpret data, leading to evidence-based conclusions and recommendations.
- Crafting and Delivering Impactful Presentations. Developing presentations and reports that clearly articulate research findings, methodologies, and implications to both academic and non-academic audiences.
- Driving Stakeholder Engagement. Actively engaging with partners and clients to understand their needs, ensuring research findings are directly applicable to policy and program decisions.
- Guiding Projects to Successful Completion. Strategically managing the execution of research projects, ensuring they remain on schedule, within budget, and achieve intended outcomes.
- Initiating and Managing Research Proposals. Spearheading the creation of detailed research proposals, including objectives, methodology, budget, and timelines, to secure funding and support.
- Liaising with Diverse Stakeholders. Building and maintaining strong relationships with project partners, funders, and community stakeholders to ensure research activities are aligned with broader goals and expectations.
- Mentoring and Developing Team Talent. Providing leadership and guidance to research staff, fostering professional growth and ensuring the team’s skills are effectively utilized in project tasks.
- Promoting Collaborative Innovation. Leading initiatives that encourage team members to contribute unique insights and solutions, enhancing the overall quality and impact of research projects.
- Translating Research into Actionable Strategies. Analyzing data to produce insights that inform the development, implementation, and refinement of policies and programs.
Associate Professor
West Virginia University
Morgantown, WV
2023 - 2016
- Authored Publications. Contributed to various fields including program evaluation through authoring and co-authoring over 10 peer-reviewed publications.
- Conducted Rigorous Evaluations. Led more than 20 evaluations that resulted in increases in program effectiveness and capacity building.
- Designed and Deployed Interactive Surveys. Utilized HTML, CSS, and JavaScript on the Qualtrics platform to create multiple surveys, garnering approximately 7,000 responses.
- Extracted and Cleaned Data for Visualization. Pulled data from various remote sources; cleaned and wrangled it, then developed and published over 20 interactive visualizations for public reporting and client dissemination using RMarkdown and Shiny.
- Mentored Students. Guided 4 Masters and Doctoral students to the successful completion of their academic research projects.
- Taught Methods Based Courses. Instructed over 500 students in evaluation, measurement, research methods, and survey design, emphasizing data science techniques.
Data Analyst
University of Kansas
Lawrence, KS
2016 - 2014
- Conducted Comprehensive Studies. Utilized qualitative, quantitative, and mixed-methods approaches, presenting findings to senior management to influence policy changes.
- Crafted Tailored Evaluations. Met the needs of diverse stakeholders, resulting in actionable insights for multiple non-academic departments.
- Designed and Administered Surveys. Reached over 300 university students and staff, achieving response rates ranging from 47% to 91%.
- Engineered Web Applications. Utilized R and Tableau to streamline the processes of data extraction and visualization.
- Spearheaded Research Initiatives. Increased student retention rates for specific at-risk groups by 3% to 5% through predictive statistical modeling.
Select Publications
Experiences and Perceptions of Engineering Students Towards Cross-Disciplinary Course Using Sentiment Analysis
Journal of Civil Engineering Education
N/A
2024
Dey, K., Rahman, M.T., Roy, A., Pyrialakou, V.D., Martinelli, D., Fraustino, J.D., Deskins, J, Rambo-Hernandez, K.E., & Plein, L.C. https://doi.org/10.1061/JCEECD/EIENG-1976
Teaching science via computational thinking? Enabling future science teachers’ access to computational thinking.
Contemporary Issues in Technology and Teacher Education
N/A
2023
Kale U., Kooken, A., Yuan, J., & Roy, A. https://citejournal.org/volume-23/issue-3-23/science/teaching-science-via-computational-thinking-enabling-future-science-teachers-access-to-computational-thinking
Program evaluation standards for utility facilitate stakeholder internalization of evaluative thinking in the West Virginia Clinical Translational Science Institute.
Journal of Multidisciplinary Evaluation
N/A
2023
Curtis, R., Roy, A., Lewis, N., Dooty, E. N., & Mikalik, T. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/831
Thinking processes in code.org: A relational analysis approach to computational thinking computer science education.
Computer Science Education
N/A
2022
Kale, U., Yuan, J., & Roy, A. https://doi.org/10.1080/08993408.2022.2145549
There’s so much to do and not enough time to do it! A case for sentiment analysis to derive meaning from open text using student reflections of engineering activities.
American Journal of Evaluation, 42 (4), 559–576.
N/A
2021
Roy, A., & Rambo-Hernandez, K.E. https://doi.org/10.1177/1098214020962576
To design or to integrate? Instructional design versus technology integration in developing learning interventions.
Education Tech Research Dev 68, 2473–2504.
N/A
2020
Kale, U., Roy, A. & Yuan, J. https://doi.org/10.1007/s11423-020-09771-8
Using interactive theater to promote inclusive behaviors in teams for first-year engineering students: A sustainable approach.
Development of A Holistic Cross-Disciplinary Project Course Experience as a Research Platform for the Professional Formation of Engineers.
N/A
2020
Dey, K.C., & Rahman, M.T., & Pyrialakou, V.D., & Martinelli, D., & Rambo-Hernandez, K.E., & Fraustino, J.D., & Deskins, J., & Plein, L.C., & Roy, A.R. https://peer.asee.org/29592
General chemistry student attitudes and success with use of online homework: Traditional-Responsive versus Adaptive-Responsive.
Journal of Chemical Education, 95 (5), 691-699.
N/A
2018
Richard-Babb, M., Curtis, R., Ratcliff, B., Roy, A.R., & Mikalik, T. https://doi.org/10.1021/acs.jchemed.7b00829
To go virtual or not to go virtual, that is the question: A comparative study of face-to-face versus virtual laboratories in a physical science course.
Journal of College Science Teaching, 48 (2), 59-67.
N/A
2018
Miller, T.A., Carver, J.S., & Roy, A.R. https://www.jstor.org/stable/26616271
Reasoning-and-proving in algebra: The case of two reform-oriented U.S. textbooks.
International Journal of Educational Research 64., 92-106
N/A
2014
Davis, J.D., Smith, D.O., Roy, A.R., & Bilgic, Y. K. https://doi.org/10.1016/j.ijer.2013.06.012
What’s in a Scriven number?
Journal of MultiDisciplinary Evaluation, 8 (19), 41-45.
N/A
2012
Roy, A.R., Hobson, K.A., & Coryn, C.L.S. https://doi.org/10.56645/jmde.v8i19.372
Book Chapters
Invited Contributions
What is a Scriven number?
The American Evaluation Association Newsletter
N/A
2012
Roy, A.R., Hobson, K.A., & Coryn, C.L.S.
Evaluations
Active
Lead Program Evaluator & Methodologist - Healthy Start Initiative (HRSA-19-049 Total Award: $5,470,000)
West Virginia University Research Corporation
Morgantown, WV
Current - 2023
- Conducted process, monitoring, and impact evaluations at the program level.
- Designed and validated survey tools, evaluating the experiences of more than 50 participants.
- Generated two comprehensive internal evaluation documents and prepared an evaluation proposal aimed at federal funding renewal.
- Utilized a mixed-methods design to independently analyze both longitudinal and cross-sectional data, focusing on the smoking cessation efforts of mothers and the experiences of new or distant caretakers/fathers.
Program Evaluator & Mixed Methods Analyst - Teaching Science with Computational Thinking: Preparing Preservice Elementary Educators of the Future STEM Workforce (2019-NSF 2142274 Total Award: $294,958.00)
West Virginia University
Morgantown, WV
Current - 2022
- Administered process evaluations for all program activities.
- Created and employed a longitudinal mixed-method study to reduce and analyze more than 50 open-ended survey responses and six interview session transcripts, leveraging thematic/content analyses and techniques including HDBSCAN/t-SNE, k-Means, and PCA.
- Designed and validated two survey tools for assessing progress and evaluating programmatic impact.
- Produced one in-depth internal evaluation document and crafted evaluation summaries for federal reporting purposes.
Completed
Community Program Evaluator & Data Scientist - WVCTSI: West Virginia Clinical and Translational Science Institute (2017-NIH 2U54GM104942-02 Total Award: $20,000,000)
West Virginia University
Morgantown, WV
2022 - 2017
- Analyzed extensive datasets using both frequentist and Bayesian methodologies, guiding the direction and activities of eight distinct programs.
- Authored quarterly reports as well as internal and external annual evaluation documents, disseminated both in print and through interactive formats developed in Rmarkdown.
- Crafted over 100 data visualizations and developed over four Shiny applications for internal and public data exploration, including research collaborations using social network analysis, grant activities pulled from the NCBI API crossed with WVCTSI grant numbers, and dissemination of result and changes in practice within and beyond West Virginia.
- Designed and disseminated tailored Qualtrics surveys enhanced with HTML, CSS, and JavaScript, reaching an audience of over 5,000 individuals.
- Led local and multi-site, multi-cluster evaluation studies that influenced five core medical research and community engagement units.
- Mentored six aspiring graduate students in social data science, steering them through successful research endeavors.
Program Evaluator - Appalachian Gerontology Experiences - Advancing Diversity in Aging Research (2020-NIH 1R25AG059558-01A1 Total Award: $678,000)
West Virginia University
Morgantown, WV
2020 - 2021
- Developed and distributed two customized interactive surveys using Qualtrics, enhanced with CSS and JavaScript, to gather feedback on programmatic activities from a specific group of 24 students.
- Spearheaded three distinct evaluative studies focusing on student efficacy, engagement, and motivation.
- Produced an external evaluation summary for federal reporting.
Research Methods Advisor & Specialist - Research Initiative: A Holistic Cross-Disciplinary Project Experience as a Platform to Advance the Professional Formation of Engineers (2019-NSF 1927232 Total Award: $200,000)
West Virginia University
Morgantown, WV
2020 - 2022
- Conducted longitudinal studies on the experiences of 30 undergraduate students in specialized interdisciplinary courses that integrated social science and engineering, using both surveys and focus group discussions.
- Mentored 10 engineering faculty members and graduate students in the implementation of research methodologies.
Program Evaluator - Stepping UP with Avenue: Progress Monitoring: A Software Suite Helping Teachers Improve Literacy Progress For Deaf/Hard Of Hearing Students (2017-ED H327S170012 Total Award: $2,470,440)
Pennsylvania State University
State College, PA
2018 - 2017
- Administered in-depth evaluations of five tools designed for assessment and engagement of the deaf and hard of hearing.
- Conducted a multi-site, multi-cluster evaluation on a large scale for all programmatic activities.
- Produced a detailed external evaluation brief for federal reporting.
Lead Program Evaluator - Cultivating Inclusive Identities of Engineers and Computer Scientists: Expanding Efforts to Infuse Inclusive Excellence in Undergraduate Curricula (2017-NSF 1725880 Total Award: $2,000,000)
West Virginia University
Morgantown, WV
2018 - 2017
- Administered longitudinal surveys to over 50 participating faculty members to gauge expectations, gather feedback for improvement, and monitor shifts in DEI attitudes and perceptions.
- Applied longitudinal NLP text mining techniques such as concordance, LDA topic modeling, and sentiment analysis to analyze and summarize feedback from over 3,000 first-year engineering students concerning grant-related class activities.
- Contributed to the creation of more than 10 journal publications and academic conference presentation materials.
- Developed more than 150 static and interactive data visualizations for stakeholder exploration, both internal and external reporting, as well as for presentations and publications.
- Evaluated all four principal investigators through personnel assessments.
- Produced two in-depth internal evaluation documents and crafted summaries for external stakeholders and federal reporting purposes.
Program Evaluator - GAUSSI: Generating, Analyzing, and Understanding Sensory and Sequencing Information: A Trans-Disciplinary Graduate Training Program in Biosensing and Computational Biology (2017-NSF 1450032 Total Award: $3,013,779)
Colorado State University
Fort Collins, CO
2020 - 2017
- Administered 32 interviews and focus groups, both cross-sectional and longitudinal, using unstructured and semi-structured formats to gauge the experiences of students and faculty.
- Carried out process evaluations for four grant-associated programs, leading to enhanced member tracking, increased program efficiency, and heightened participant satisfaction.
- Constructed and validated tools to measure students’ ability to convey research findings to a lay audience.
- Designed semi-annual adaptive and interactive Qualtrics surveys enhanced with HTML/CSS/JavaScript, securing feedback from over 100 students and faculty with a 95% response rate.
- Developed predictive models targeting the improvement of student engagement, experience, and retention.
- Generated data visualizations for assessment and longitudinal studies, bolstering inferential statistical analyses to identify trends and support programmatic enhancements, retention strategies, and satisfaction initiatives.
- Produced nine in-depth internal evaluation documents and crafted summaries for external stakeholders and federal reporting purposes.
Presentations
Let’s get sentimental: Machine learning aided data analysis for large qualitative data sets
American Evaluation Association Annual Conference
New Orleans, LA
2022
Seidel, T., Ferguson, C.F., & Roy, A.R.
Best of both worlds: Affordances of mixing machine learning and qualitative content analysis
American Educational Research Association Annual Meeting
San Diego, CA
2022
Roy, A.R., Ferguson, C.F., Curtis, R., & Babb-Richards, M.
These aren’t random words just strung together?: Using machine learning and pretty visualizations to discover topics in articles.
American Evaluation Association Annual Conference
virtual
2020
Roy, A.R.
Little fish in a big pond, only fish in a little pond: How roles shape our identities as evaluators.
American Evaluation Association Annual Conference
Minneapolis, MN
2019
Loomis, D.L., Mikalik, T.L., Curtis, R., Roy, A.R., & Bernstein, M.
Evolving program logic models to meet shifting program needs: The case of WV Clinical Translational Science Institute.
American Evaluation Association Annual Conference
Minneapolis, MN
2019
Curtis, R., Roy, A.R., Bernstein, M, Loomis, D.L., & Mikalik, T.L.
The value of external evaluators when building clinical translational research infrastructure.
American Evaluation Association Annual Conference
Minneapolis, MN
2019
Curtis, R., Roy, A.R., Bernstein, M, Loomis, D.L., & Mikalik, T.L.
Using associated networks to evaluate content within courses.
American Evaluation Association Annual Conference
Minneapolis, MN
2019
Roy, A.R., Kale, U, & Yuan, J.
Why is it that writers write but fingers don’t fing? Using machine learning and lexemes to make sense of nonsense.
American Evaluation Association Annual Conference
Minneapolis, MN
2019
Roy, A.R., Curtis, R., Mikalik, T.L., Loomis, D.L., & Bernstein, M.
Iscovering the underlying meaning behind get me off your f***ing mailing list? and most other narratives.
American Evaluation Association Annual Conference
Minneapolis, MN
2019
Roy, A.R., Curtis, R., Mikalik, T.L., Loomis, D.L., & Bernstein, M.
Assessing for improvement: The use of artificial intelligence to uncover potential differential impact of assignments.
American Evaluation Association Annual Conference
Toronto, CN
2019
Roy, A.R. & Rambo-Hernandez, K.
Using a mixed methods evaluation to discover how an interactive theater based model stimulates inclusive behaviors in engineering.
American Evaluation Association Annual Conference
Cleveland, OH
2018
Roy, A.R., Rambo-Hernandez, K., Hensel, R.A., & Morris, M.L.
Examining the past and looking forward: The future of evaluation theory and use.
American Evaluation Association Annual Conference
Cleveland, OH
2018
Roy, A.R. & Hobson, K.A.
Transforming graduate STEM Education: A theory-driven evaluation of the GAUSSI National Science Foundation Research Training (NRT) Program.
American Evaluation Association Annual Conference
Washington, DC
2017
Roy, A.R., Hernandez, P.A., Chen, T., & Paguyo, C.
Program evaluation for everyone! - Constructing an online foundational course for capacity building using theorists as a focus.
American Evaluation Association Annual Conference
Washington, DC
2017
Roy, A.R. & Curtis, R.P.
Content in the background: Using evaluation theorists as the principal motivator for foundational evaluation courses.
Hawaii-Pacific Evaluation Association Annual Conference
Kane’ohe, HI
2017
Roy, A.R. & Curtis, R.P.
Survey says! Students getting tired of surveys.
National Academic Advising Association Annual Conference
Las Vegas, NV
2015
Roy, A.R. & Goetz, H.L.
Influences of Hierarchical Linear Modeling in evaluation.
Aotearoa New Zealand Evaluation Association Annual Conference
Auckland, NZ
2013
Hobson, K.A., Roy, A.R. & Coryn, C.L.S.
Survey sample methods: Evaluators’ toolbox refreshment.
American Evaluation Association Annual Conference
Minneapolis, MN
2012
Hobson, K.A., Roy, A.R. & Coryn, C.L.S.
Teaching Experience
Evaluation, Measurement, and Research Methods (2016 - 2023)
Educational Psychology
West Virginia University
Morgantown, WV
2017
Educational Research
West Virginia University
Morgantown, WV
2016
Measurement/Evaluation in Educational Psychology
West Virginia University
Morgantown, WV
2020 - 2018
2022, 2020, 2018
Mixing Research Methodologies
West Virginia University
Morgantown, WV
2019 - 2017
2022, 2019, 2018, 2017
Program Evaluation
West Virginia University
Morgantown, WV
2023 - 2017
2023, 2022, 2021, 2020, 2019, 2018, 2017
Statistical Methods 1
West Virginia University
Morgantown, WV
2021 - 2017
2021, 2020, 2019, 2018, 2017
Mathematics (2005 - 2015)
Business Calculus
Central Michigan University
Mount Pleasant, MI
2008
College Algebra
Central Michigan University
Mount Pleasant, MI
2009
Discrete Mathematics
Central Michigan University
Pittsburgh, KS
2012
Elementary Statistics
Central Michigan University
Pittsburgh, KS
2014
Foundations of Statistics
Central Michigan University
Pittsburgh, KS
2010 - 2009
2009, 2010
Intermediate Algebra
Central Michigan University
Mount Pleasant, MI
2008 - 2007
2007, 2008
Integral Calculus
Michigan Technological University
Houghton, MI
2007
Linear Algebra
University of Kansas
Lawrence, KS
2015 - 2014
2014, 2015
Mathematical Thinking Grades 6-12
Western Michigan University
Kalamazoo, MI
2013
Mathematics Curriculum Grades 6-12
Western Michigan University
Kalamazoo, MI
2014 - 2013
2013, 2014
Multivariable Calculus
Michigan Technological University
Houghton, MI
2005
Single Variable Calculus
Michigan Technological University
Houghton, MI
2007 - 2006
2006, 2007
Service
Memberships
American Evaluation Association
N/A
N/A
2022 - 2012
Social network analysis: Finding meaning in connections.
Nova Science Publishers, Inc.
N/A
2014
Roy, A.R. 978-1-53612-734-8