Architectural researchers have introduced FUGenerator, an integrated multimodal artificial intelligence platform designed to bridge persistent gaps in domain knowledge, cross-disciplinary collaboration, and software interoperability across the building design lifecycle. Developed by a research team comprising Xinhui Xu, Tinghao Feng, Yulu Zhang, Zhengcheng He, and Philip F. Yuan, the open-access system responds to fundamental limitations in contemporary machine-learning tools utilized by designers. Conventional architectural practices have historically moved through a linear progression from design intention to drawing, representation, and final construction. Recent computational developments, however, have propelled an automated design paradigm organized around intention, generation, optimization, fabrication, and evaluation. While text-to-image and text-to-video generators offer initial aesthetic inspiration, existing implementations frequently encounter severe friction, including format incompatibilities, irreversible data loss, version inconsistencies across transfers, and opaque generative processes that prevent practitioners from evaluating or guiding underlying algorithmic logic.
To resolve these interoperability and specialization deficits, the research team structured FUGenerator around three core operational pillars: a dual-subgraph architectural knowledge graph, an expansive multimodal algorithm library, and a node-based traceable workflow editor. The architecture of multimodal machine learning fundamentally depends on five functional elements, incorporating input encoders for diverse modalities, input projectors, core generative models, output projectors, and modality-specific output decoders. While commercial models such as DALL-E, Midjourney, Stable Diffusion, and Runway Gen-2 perform isolated single-modality conversions or lack domain-specific architectural depth, the authors established a unified platform where natural language serves as an interactive semantic channel connecting text, two-dimensional imagery, parametric formats, and physical fabrication data. By integrating specialized datasets and fine-tuning pipelines into an interconnected software environment, the platform enables designers to navigate iterative multidimensional exploration while retaining operational control across distinct phases of project development.
The underlying knowledge foundation consists of an architectural knowledge graph split into complementary semantic and visual subgraphs, implemented as a semantic network using an Extensible Markup Language structure under Resource Description Framework principles. The semantic subgraph incorporates more than 13,200 building regulation clauses and 283 peer-reviewed academic papers spanning parametric design, digital construction, and artificial intelligence. Regulatory coverage centers on structural integrity, building fire prevention, general safety, and sustainable construction practices. To construct this resource, researchers executed manual entity annotation through Label Studio utilizing double cross-validation to maintain data reliability. Named entity recognition models were subsequently trained via the spaCy pipeline using the zh_core_web_trf pre-trained transformer base. Graph relationships were established through scikit-learn computations of term frequency-inverse document frequency semantic similarity, keyword matching, and section citation links, allowing practitioners to query entity nodes and relational edges through standard SPARQL retrieval.
Hierarchical organization within the semantic subgraph spans four distinct architectural levels, ensuring systematic traceability and future maintenance. The primary level classifies foundational document categories, encompassing building codes and academic literature. The secondary level specifies defined regulatory codes and keyword indices, such as the Building Fire Prevention Design Code and its specific fire prevention requirements. The tertiary tier indexes authors, formal citations, and thematic section headings, including firefighting equipment specifications, while the fourth level records the individual paragraph contents within each legislative clause or paper section. In parallel, the visual knowledge subgraph aggregates curated imagery from public architectural repositories, design firm archives, blueprints, diagrams, competition schemes, and construction photographs. Entity nodes store visual assets in base64 format with metadata detailing project titles, design firms, geographical locations, building typologies, and material categories. Initial descriptions generated by the BLIP multimodal model were manually verified before keyword tokenization established links across visual categories.
Operational execution within the platform relies on a multimodal algorithm library configured across text, image, and three-dimensional model generation modules. The text generation module operates through a Retrieval-Augmented Generation pipeline to answer technical inquiries and synthesize regulatory documentation. Architectural codes, academic manuscripts, and project reports are structured, assigned metadata, and vectorized into dense embeddings indexed via Facebook AI Similarity Search. The system delivers generated responses using ChatGLM-6B, fine-tuned specifically for architectural comprehension using Low-Rank Adaptation. The fine-tuning corpus comprised approximately 20,000 regulatory question-and-answer pairs and case-based architectural dialogues. Training was completed on an NVIDIA A100 graphics processing unit in approximately two hours, during which the core Transformer layers were frozen while newly introduced low-rank matrices received parameter updates, preserving general linguistic fluency while instilling precise domain expertise.
Visual synthesis is handled through Denoising Diffusion Probabilistic Models augmented by ControlNet, allowing users to apply explicit structural and boundary conditions during image generation. The ControlNet layers underwent domain-specific fine-tuning on roughly 4,500 building-specialist architectural images to maintain geometric realism under conditional guidance. The system integrates BLIP and Git-Clip models featuring Multimodal Encoder-Decoder structures to support image-to-text feature extraction and multitask alignment. Natural language text prompts and image embeddings achieve cross-modal alignment through dedicated input and output projector training. To address resolution deficits, the platform integrates the Enhanced Super-Resolution Generative Adversarial Network, termed ESRGAN, upgrading low-resolution outputs into detailed renderings. These integrated algorithmic tools support diverse interaction pathways, including text-to-image, image-to-image synthesis, sketch-to-rendering transformations, and localized inpainting, which designers iteratively guide through semantic re-prompting and parameter adjustments.
For spatial geometry, FUGenerator incorporates dual generation mechanisms converting both descriptive text and two-dimensional images into three-dimensional models. Textual prompts utilize large language models to interpret parametric format files and modify internal architectural volume parameters. For image-based reconstruction, the platform utilizes TripoSR, deploying pre-trained weights to reconstruct complex spatial three-dimensional meshes from single image inputs without requiring additional training steps. The interactive interface is driven by a Self-Assembly-Node architecture developed on the rete.js framework, providing an interpretable, drag-and-drop visual programming workspace. Workflow nodes are categorized into input, algorithm, and output stages. Input nodes accept three distinct data streams: text specifying spatial relationships, compositional rules, or descriptions; image references providing visual inspiration; and control parameters adjusting algorithmic impact factors and model constraints. Front-end reactivity is built on Vue.js utilizing virtual Document Object Model rendering, while RabbitMQ manages asynchronous message queues to balance computational loads across back-end server clusters.
To assess operational performance, researchers deployed the platform within a university design curriculum involving 21 fourth-year undergraduate architecture students over eight weeks. The educational experiment focused on furniture design and physical fabrication, assigning the chair as an architectural prototype across four distinct material fabrication groups: timber tectonics with five students, bamboo weaving with six students, three-dimensional printing with four students, and mixed reality construction with six students. Individual students were tasked with independently designing and fabricating a functional chair using their team’s specialized construction technique. The research team examined design streamline decomposition data across eight representative students, monitoring their progression through text, image, control, and three-dimensional model inputs and outputs. Across all eight evaluated subjects, students established a shared workflow pattern, starting with conceptual text and image prompts, conducting multiple iterative modifications via algorithmic controls, and finally exporting selected two-dimensional imagery into three-dimensional geometric modeling modules.
Despite common baseline workflows, material constraints prompted diverging interaction behaviors among the design cohorts. Students in the timber tectonics and bamboo weaving teams faced substantial physical and structural material limitations, leading them to rely predominantly on sketch-to-rendering image interactions to retain feasible physical geometry. Conversely, students working with three-dimensional printing and mixed reality fabrication operated under fewer immediate material constraints, experimenting with highly divergent text-to-image and text-to-video iterations. Platform telemetry revealed that students achieved peak generation bursts exceeding 180 design iterations in a single hour, completing initial furniture schemes within the first 20 days. This accelerated conceptual cycle afforded students extended time to concentrate on structural optimization and material assembly. Usage metrics showed a heavy preference for image-to-image generation and localized inpainting due to tight spatial constraints, whereas direct image-to-3D generation experienced low adoption due to technical limitations in mesh quality.
A subsequent eight-week testing phase tasked the student cohorts with scaling their workflows to complex architectural spatial prototypes, highlighted by a multi-story timber tower and an undulating bamboo footbridge. The timber tectonics team translated structural logic from the historical Yingxian Wooden Pagoda into modern timber architecture, fine-tuning two custom LoRA models before applying text-to-image and image-to-image workflows to match traditional layered structural principles for robotic fabrication. In contrast, the bamboo bridge cohort optimized a structure across an irregular site characterized by elevation differences at both ends, utilizing an AI-assisted workflow following structural analysis. By training a dedicated bamboo weaving LoRA model and applying localized control adjustments, the bridge designers integrated non-standard interwoven patterns, translating generated images into three-dimensional models for joint detailing. These implementations validated the platform’s capacity to navigate varied spatial typologies, shortening conceptual cycles while transitioning directly from digital proposals to automated physical construction.
Post-course qualitative interviews revealed that visual node management clarified design thinking and provided enhanced procedural control compared to commercial AI generators. However, participants noted operational hurdles, emphasizing that FUGenerator lacks a foundational model capable of generating highly creative spatial solutions without extensive domain fine-tuning. Navigating algorithm nodes required repeated iterative adjustments, while the multifaceted interface presented a steep initial learning curve. The researchers acknowledged that mapping complex, non-quantifiable architectural requirements into latent machine spaces risks information loss, compounded by the scarcity of publicly available multimodal architectural datasets. Current system capabilities remain bounded by a developing algorithm library and curated training data. Future development aims to expand open-source architectural datasets, broaden real-world professional case studies, and refine node workflows to support diverse building typologies across commercial architectural practice.
Subject of Research: Multimodal artificial intelligence integration platform for architectural generative design
Article Title: FUGenerator: multimodal-AI platform for architectural design
Article References: Xu, X., Feng, T., Zhang, Y., He, Z., & Yuan, P. F. (2025). FUGenerator: multimodal-AI platform for architectural design. Architectural Intelligence, 4(1), Article 14. https://doi.org/10.1007/s44223-025-00093-4
Image Credits: AI Generated
DOI: 10.1007/s44223-025-00093-4
Keywords: multimodal artificial intelligence, architectural design, knowledge graphs, generative design, diffusion models, digital fabrication, parametric modeling, retrieval augmented generation, FUGenerator, multimodal-AI, platform, architectural
