StableYolo: Optimizing Image Generation for Large Language Models

Harel Berger, Aidan Dakhama, Zishuo Ding, Karine Even-Mendoza, David Kelly, Hector Menendez, Rebecca Moussa, Federica Sarro

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

AI-based image generation is bounded by system parameters and the way users define prompts. Both prompt engineering and AI tuning configuration are current open research challenges and they require a significant amount of manual effort to generate good quality images. We tackle this problem by applying evolutionary computation to Stable Diffusion, tuning both prompts and model parameters simultaneously. We guide our search process by using Yolo. Our experiments show that our system, dubbed StableYolo, significantly improves image quality (52% on average compared to the baseline), helps identify relevant words for prompts, reduces the number of GPU inference steps per image (from 100 to 45 on average), and keeps the length of the prompt short (≈ 7 keywords).

Original languageEnglish
Title of host publicationSearch-Based Software Engineering - 15th International Symposium, SSBSE 2023, Proceedings
EditorsPaolo Arcaini, Tao Yue, Erik M. Fredericks
PublisherSpringer Science and Business Media Deutschland GmbH
Pages133-139
Number of pages7
ISBN (Print)9783031487958
DOIs
StatePublished - 2024
Externally publishedYes
Event15th International Symposium on Search-Based Software Engineering, SSBSE 2023 - San Francisco, United States
Duration: 8 Dec 20238 Dec 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14415 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference15th International Symposium on Search-Based Software Engineering, SSBSE 2023
Country/TerritoryUnited States
CitySan Francisco
Period8/12/238/12/23

Keywords

  • Image Generation
  • LLMS
  • SBSE
  • Stable Diffusion
  • Yolo

Fingerprint

Dive into the research topics of 'StableYolo: Optimizing Image Generation for Large Language Models'. Together they form a unique fingerprint.

Cite this