IT 13 - AI-Powered Virtual Try-On: Transforming Fashion with Augmented Reality

With the rapid growth of e-commerce in the fashion industry, one of the most common challenges faced by online retailers and consumers alike is the inability to try on clothes before purchasing. This often results in customer dissatisfaction, increased product return rates, and missed opportunities for sales conversions. To address these issues, our project explores the implementation and evaluation of a Virtual Try-On System using a state-of-the-art deep learning model known as IDM-VTON (Improving Diffusion Models for Authentic Virtual Try-On in the Wild). Rather than developing the model from scratch, our objective was to understand, utilize, and apply this existing model to real-world scenarios, testing its capabilities in generating realistic try-on results for various types of clothing and user images.

The core functionality of our project involves using IDM-VTON, a powerful diffusion-based model specifically designed for the virtual try-on task. This model takes a clothing item image and a person image as inputs and generates a highly realistic output that shows the person wearing the selected clothing item. The model is capable of preserving complex visual features such as texture, color, folds, and alignment while adapting the clothing to different body poses and structures. By leveraging this pre-trained model, we were able to build a virtual try-on pipeline that simulates the online shopping experience in a much more interactive and user-centric way.

Our work began with the preparation and curation of datasets, which included high-quality images of clothing items and human models. We applied essential preprocessing steps such as human parsing (segmenting the person from the background), pose estimation (identifying body landmarks), and garment masking to ensure that the input data met the requirements of the IDM-VTON pipeline. We experimented with a variety of clothing types and person images to test the robustness and realism of the model’s output across different scenarios.

While we did not train the model ourselves, our contribution lies in configuring, and testing the pre-trained architecture, as well as critically analyzing its performance. We evaluated the outputs based on visual realism, preservation of clothing details, and natural alignment with the human body. Additionally, we explored how the model handled edge cases such as complex poses, occlusions, and overlapping accessories, which are often encountered in real-world applications. In doing so, we gained a deep understanding of how diffusion-based models can be applied in virtual fashion, and identified the limitations and potential improvements in future virtual try-on systems.

This project has practical implications for fashion retailers looking to enhance user experience and reduce return rates, as well as for developers and researchers working on AI-driven fashion technologies. By offering customers a chance to virtually “try on” clothes with accurate and visually convincing results, this solution can significantly increase consumer confidence, streamline the decision-making process, and contribute to more sustainable shopping behavior. Our project serves as a demonstration of how pre-trained AI models, when applied effectively, can bridge the gap between technology and user needs in a real-world commercial context.