Forthcoming Subject-Area Review

Harnessing the Multimodal Features of Generative AI to Advance Ecological Engineering

by David Tilley

Abstract

This paper explores the integration and potential of generative AI, specifically ChatGPT-4 by OpenAI, to advance the field of ecological engineering. The multimodal capabilities—conversation, image creation & interpretation, spatial reasoning, customization and many forms of data analysis using code writing—add abundantly to the profession’s toolbox. This work briefly reviews the mathematical foundations of GPT-4, including word embeddings and transformer algorithms, to highlight the contrast with popular internet search. The paper demonstrates GPT-4’s ability to create cartoons our of news articles, detect insect infestation of plant leaves, count stems in a forest image, spatial reason from text and use as a virtual teaching assistant by transcribing handwriting and giving personalized feedback to students. Furthermore, the ability to tailor GPT-4 with OpenAI’s CustomGPT feature offers countless possibilities for harnessing GPT4’s global knowledge and capabilities into a focused and specialized format that corresponds to a user’s particular task and goals. The paper concludes by suggesting that the creation of AI Agents is the next phase for harnessing GPT4’s multimodal abilities to efficiently and effectively advance ecological engineering. The integration of generative AI into ecological engineering practice and academia can enable the field to reach new heights of impact, productivity, and innovation.