Human-Subject Experiment For Design Research

1. Function Optimization Game for Studying the Effect of Cost on Design Decisions Under Competition

Introduction and Related publications

This game is introduced in the following papers.

Z. Sha, K. N. Kannan, J. H. Panchal, “Behavioral Experimentation and Game Theory in Engineering Systems Design”, Journal of Mechanical Design, Transactions of the ASME, volume 137, issue 5, pp: 051405 (10), 2015.
J. H. Panchal, Z. Sha, K. N. Kannan, “Understanding Design Decisions under Competition using Games with Information Acquisition and a Function Optimization Experiment“, Journal of Mechanical Design, Transactions of the ASME, volume 139, issue 9, pp: 091402 (12), 2017.

Experimental platform

The game was developed using z-Tree, a software package for developing and carrying out economic experiments. Our developed function optimization game can be downloaded here.

screenshot

Protocols, instructions and other supporting documents

The following is a list of documents as the references for setting up the experiment for data collection.

2. Energy Systems Design Experiment for Studying Sequential Design Behaviors and Systems Thinking

Introduction and related publications

J. Clay, M. H. Rahman, D. Zabelina, C. Xie, X. Li, Z. Sha, “Systems Thinking Factors as Predictors of Success in Engineering Design“, The Ninth International Conference On Design Computing and Cognition (DCC), 29 June – 1 July 2020, Atlanta, GA. Poster abstract and presentation.
M. Rahman, C. Schimpf, C. Xie, Z. Sha, “A Computer Aided Design Based Research Platform for Design Thinking Studies“, Journal of Mechanical Design, Transactions of the ASME, volume 141, issue 12, pp: 121102 (12), 2019.
M. Rahman, C. Xie, Z. Sha, “A Deep Learning Based Approach to Predicting Sequential Design Decisions”, ASME 2019 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, Anaheim, CA, Aug. 18-21, 2019. (Robert E. Fulton ASME CIE SEIKM Best Paper Award). Also see the poster here.
M. Rahman, C. Xie, Z. Sha, “Towards Building an AI-Integrated Computer-Aided Design Platform for Design Research”, NSF Research Poster Competition, The International Mechanical Engineering Congress and Exposition (IMECE), Salt Lake City, UT, Nov. 8-14, 2019. Poster.
M. Rahman, M. Gashler, C. Xie, Z. Sha, “Automatic Clustering of Sequential Design Behaviors“, ASME 2018 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, Quebec City, Canada. 2018. Paper number: DETC2018-86300.
Z. Sha, L. Godfrey, M. Gashler, “Modeling Sequential Design Decisions Using Fine-Grained Empirical Data”, 2018 Design Science Research (DSR) Workshop, Montreal, Canada, August 23-25, 2018. Peer-reviewed extended abstract.

In this study, we have designed two experiments. The design context is solar energy systems design and the experiments are conducted using a CAD software for solar energy systems as the research platform – Energy3D, which is developed by our collaborator Dr. Charles Xie. Energy3D is an NSF-funded free software. The design behavioral data collected in this set of experiments consists of questionnaires, design actions logs, intermediate CAD files, video recording, design journals, and interview questions. The sequential design behavioral study mainly relies on the fine design action data automatically logged by Energy3D.

Experiment 1: Energy-Plus Home Design Challenge

Experiment 2: Solarizing UARK Parking Lot Design Challenge

For both experiments, the following documents will also provide support.

3. Design Experiment for Studying Human-AI Collaboration in Engineering Design

Introduction and related publications

In this project, we have been working with Dr. Charles Xie and his group from the Concord Consortium to develop experimental platform enabling the study of AI’s instructional role in engineering design education as well as AI’s capability in facilitating designers in engineering design practice. So far, we have developed two experiments. The first one can be used to support students’ science learning behaviors with the intervention of intelligent design tutor. The second experiment can be used to research designers’ design behaviors and sequential decision-making strategies in collaboration with AI design agent. Both experiments are conducted in the context of solar energy system design, and the design is performed within the CAD software Energy3D equipped with GA-based AI agents. Some preliminary results from the first experiment are summarized in the following paper. More will be coming soon.