3D computer-aided design (CAD) models are essential to many engineering simulation and analyses, such as finite element analysis (FEA). We (work led by Dr. Xingang Li and the PhD student, Yuewan Sun) recently explored the potential and capabilities of leveraging LLMs to directly generate 3D CAD models from various design modalities, including text, sketches, and images. We published a series of two articles on these LLM4CAD methodologies in the ASME Transactions: https://doi.org/10.1115/1.4067085 and https://doi.org/10.1115/1.4067713.

In the first paper, we utilized LLM as both a generator and debugger to iteratively improve CAD program generation for 3D models. We examined the relationship between part complexity and LLM4CAD performance (e.g., intersection over union (IoU) of the generated 3D shapes). In the second paper, we fine-tuned four small models using different data sampling strategies based on the length of a CAD program. Our study compared these models with one another and also with GPT-4 without fine-tuning. The insights gained can guide the selection of sampling strategies for building training datasets in the fine-tuning practices of LLMs for text-to-CAD generation, while considering the trade-off between part complexity, model performance, and cost. Additionally, we tested the generalizability of our methodology on more complex, real-world mechanical components drawn from the ABC dataset. We open-sourced our LLM4CAD dataset at the Texas Data Repository as part of a recent initiative led by the special issue “Design by Data: Cultivating Datasets for Engineering Design” from the Journal of Mechanical Design.