Designing organic molecules lies at the heart of solving numerous chemistry-related challenges, necessitating effective collaboration between human intuition and computational power. This study demonstrates how general-purpose Large Language Models (LLMs) such as GPT-4 can facilitate the design of potent molecules, leveraging feedback from experiments and empirical knowledge through natural language. We used this approach to design organic structure-directing agents (OSDAs) that guide the crystallization of zeolites. A computational workflow was developed, wherein the LLM proposed novel OSDAs to stabilize targeted zeolites. The suggested candidates underwent evaluation through empirical screening criteria and atomistic simulation. Feedback was then provided to the LLM in natural language to refine subsequent proposals, thus progressively enhancing the proposed OSDAs and promoting the exploration of chemical space. The predicted candidates encompassed experimentally validated OSDAs, structurally analogous ones, and novel ones with superior affinity scores, underscoring the robust capability of the LLM. The collaborations between humans and machines, utilizing natural language as the communication interface, hold potential for application in other molecular design tasks, including drug design.