AI-Driven Automotive Photo Editing for Realistic Shadows and Window Effects

Customer Overview

A US-based automotive photography giant offered an app that enabled auto dealers to capture professional-grade interior and exterior car photos using 37 standardized angles, which were polished using automated tools and manual editing to deliver high-definition 4K images. Processing over 70,000 images daily, the client’s reliance on a third-party tool for image background removal and replacement increased operational costs. They wanted a scalable, cost-effective, and IP-owned solution.

Project Overview

In our project’s Phase 1, we helped our client replace a costly third-party tool with a custom-built, in-house image background removal and replacement solution, meeting their needs for both images and videos. Building on this success, in Phase 2, the client aimed to automate a key manual photo editing workflow: creating realistic shadows in automotive images. Crafted by professionals, these shadows added visual depth and precision to both interior and exterior shots. The goal was to replicate this human-level creative capability within the tool, add new features, and enhance existing features to show realistic window effects.

Challenges

Building a custom automotive photo editing solution to replicate professional shadow design, show realistic window effects, and adapt to varied car models and angles, while balancing AI with vision engineering and maintaining sub-second processing speeds.

• Consider both the industry standards and our client’s established process for professionally designing car shadows. Generate two types of shadows: a general shadow for the entire car, and separate tire shadows that align with it to make the vehicle appear properly grounded and the overall shadow effect realistic.
• A single shadow design wasn’t viable for all car images, as different car models and camera angles affected shadow shape and direction. Our solution must account for this variability to generate realistic shadows.
• While a hybrid approach combining vision engineering with AI models was necessary, determining the right balance to achieve accurate results was the challenge.
• Existing image annotation tools proved inadequate for the shadow generation use case. Photoshop could help, but using it at scale would increase costs. We needed a cost-effective solution.
• In Phase 1, the system applied the same transparency level to all windows after background removal. The challenge was to detect which windows were tinted or less tinted in the original images, and then apply the correct level of tinting effect to each one.
• Our initial window detection algorithm misclassified the car carrier as part of the window, causing frosted effects to be incorrectly applied to non-window areas.
• Even if the solution uses multiple AI models, maintain response time under one second and ensure fast graphics processing for scalability and performance.

Solution

We designed a hybrid solution for automotive photo editing, combining two custom-built AI models with vision engineering techniques to deliver accurate shadow generation, realistic grounding effects, and precise window effects across all car models and camera angles.

• Leveraging our client’s standards and industry best practices, we defined output quality benchmarks for shadow generation with grounding effect. To achieve this, we used two custom-built AI models and vision engineering techniques.
• Our chosen AI model must generate shadows for all car sizes & photo angles. So, we ran experiments with pix2pix, CycleGAN, Stable Diffusion img2img & Segmentation models, training them on 5000 images from 37 compositions & building PoCs to compare outputs. Stable Diffusion delivered top quality, followed by Segmentation model.
• While the Stable Diffusion model needed minimal vision engineering to meet our quality goals, it was costly to host and too large to support sub-second response times. Contrarily, the Segmentation model needed more engineering support but proved faster and cost-effective. Given our client’s need for performance and scalability within budget, we selected the Segmentation model.
• We fully trained and customized two segmentation models: one to suggest shadow shape and one to identify tire-grounding zones. Later, vision engineering techniques like blur and opacity adjustments were applied to create realistic effects. This decoupled setup enabled independent fine-tuning and balanced computational load.
• To replicate Photoshop-like shadow curves without licensing costs, we adopted Inkscape, an open-source tool. We trained the annotation team on Inkscape, achieving high-quality visual effects within budget and tight timelines.
• To achieve realistic tinted and frosted window effects, we applied vision engineering techniques, like hue and saturation adjustments, blurring, and layered transparency. Built an algorithm to analyze window brightness in the original image and adjust effects accordingly.
• We refined our window identification algorithm such that it now accurately distinguishes windows from non-window areas across car models and compositions.
• Even while employing three AI models for background removal, shadow, and grounding effects, we maintained sub-second speeds by invoking models in parallel. Used GPUs for graphics tasks to support high-volume processing.

Benefits

The AI automotive photo editing solution we developed to generate realistic shadows and window effects delivered the following key advantages:

• Automated a skilled manual workflow, reducing reliance on human editors and improving turnaround time, operational efficiency, and maintainability.
• Delivered consistent, high-quality visual output at scale through a modular system, giving the client a competitive edge in automotive product photography.
• Added advanced visual features like realistic shadows, grounding effects, and tinted windows, increasing the value and extensibility of the IP.
• Enabled real-time responsiveness with sub-second processing, critical for high-throughput, dealer-facing applications and scalable system performance.

Technology

• BiRefNet
• OpenCV
• Python Imaging Library (PIL)
• Python
• Hugging Face
• FastAPI
• Vast.ai
• CVAT.ai
• AWS Sagemaker

Industry

• Automobile/Automotive