Using AI to Generate Art for CODE Magazine

by Markus Egger | First published: June 19, 2024 - Last Updated: July 23, 2024

At CODE, we use AI to generate art. This may be the cover of a magazine, but most of our art is crated for web uses. (Antoher example would be the header artwork that adorns this blog post). We need artwork for our web site, but - perhaps more importantly - we need art for social media sharing and similar scenarios. Unfortunately, it is cost prohibitive to generate individual works of original art for tens of thousands of articles, and do so in formats matching all the social media channels, no less. Fortunately, that equation changed with AI!

Generating Art for Articles

I recorded a short video that shows the whole sequence of how we generate art using a series of separate AIs. Here it is:

Here is the short version of the steps we take:

  1. We have an autor who submits an article, which - after going through our editing process - ends up in our article management system, which is shown in this video.
  2. This doesn't have anything to do with art generation as such, but it is still interesting that the next step we usually take is have AI generate a short synopsis of the article, which we then use in scenarios such as article searches, when we want to show a list of articles with a short description.
  3. The actual art generation process kicks off when the user uses our art-generation interface to enter a prompt used to have AI generate an image. (Example prompt: “A rocket ship leaving earth orbit”)
  4. Most of the time, it is hard to think of a good prompt, since our articles are rather abstract technical articles, with (usually) no obvious visual representation. (What does database performance optimization look like exactly, to pick out a representative example?). This is where the magic kicks in! We have an AI that can read the article, and not just understand it in technically accurate terms, but it can then creatively come up with ideas that go well with that article. This includes not just a general idea, but it suggests art styles, useful metaphors, pop-culture references that might be helpful, and more. The user can tweak that prompt if desired.
  5. The next step is to then ask an image generation AI to use this prompt to generate an original image for it. We use different AIs for that. The video shows the use of OpenAI's Dall-E 3 for this, but we often switch the model, use local models, let models compete with each other, and so forth.
  6. We have the ability to not just take a detailed look at the generated image, but also see what internal suggestions the image generation AI made (see below), which can be useful to further fine-tune the prompt and generate new versions of the iamge
  7. When we find an image we like, we use yet another AI that uses an internal tool called “Olympus Image Composer” to create different format versions of the image and add other appropriate elements, like the article title, the name of the author(s), logos, and more. The user can change the details of the compositions, if desired. When the user chooses to save the compositions, they are automatically attached/associated with the article, and voila! We now have production ready artwork for the article!

The following is an example of the image generator creating art for the example prompt used above:

However, a more advanced, AI generated prompt may produce something like this (in this example, the art is for a Docker development article and the artwork makes more sense for that than a rocket ship):

The prompt generated in this example (which changes every time the AI generates a prompt) is this:

Create an image of a sleek, futuristic workstation floating on a cloud island. The workstation should include modular elements like gears, containers, and digital blueprints assembling around it in a whimsical, steampunk style. The background should have a serene blue sky with friendly clouds, subtly integrated with high-tech elements like glowing circuits and holographic displays. The scene should be lighthearted, professional, and cool, with bold colors but not overly cluttered. Instruct DALL-E to not use letters and numbers, or any other writing, unless absolutely necessary. - Possible Art Style(s): futuristic and sleek, blending elements of cyberpunk and minimalism with a touch of whimsical steampunk. Possibly useful inspirations: futuristic cityscapes, modular technology, assembly lines, mechanical gears and cogs. Possibly useful metaphors: container ships, LEGO bricks, modular robots, digital blueprints, cloud islands. Possibly useful pop-culture references: The Matrix, Tron, Back to the Future (specifically the DeLorean time machine), Iron Man's suit assembly scenes. Possibly useful visual ideas that could be included: containers, gears, clouds, floating islands, developer tools, sleek laptop, futuristic workstation, vibrant colors.

Note that this includes a creative idea for what the image should be (based on the article contents) as well as other potentially helpful ideas.

Here is a closer look at one of the images as well as what the AI suggests for it:

Note how much more detailed the AI's revised prompt suggestion is, compared to the original prompt that was used to generate the image in the first place.

Finally, here is a screen shot from our Image Composer UI which uses the art to generate individual designs (this one is for Instagram sharing, which is one of several compositions):

Note how this composition contains the original image created by the image-creation AI, as well as other elements, such as an appropriate logo, the title of the article, and the name of the author. This is done by another AI-driven process that creates image compositions based on usage.

Note: The artwork used as a header for this blog post was also created with a very similar process. Other things generated by AI on this blog post include the abstract (shown on codemag.com/blog) as well as the color used for the header and menu, which was chosen by an AI to be pleasingly accompanying the artwork when seen by a human reader.

Eliminating Jobs?

On a side-note: People often ask us whether we think it is OK that this eliminates jobs because we do not need artists anymore. This is actually an incorrect assessment of the situation. Yes, we now generate thousands of original pieces of art using AI. However, this is not something we ever did before, because it simply wasn't a cost-effective use of our resources. Therefore, no artists were removed from previously existing jobs. Quite the contrary: We are now in the business of producing art for all kinds of things we do (and didn't do before), and not all that art can be created by AI. Even if we just produce a small percentage of art with human artists, in the big picture, that is still more than before. Therefore, we now spend more money on art-related jobs than before. We spend more, yet it is worth it, because the equation now works overall. (Note: This is a pattern we also see with many of our other uses of AI, whether it is our own our scenarios our customers encounter).