Diffusion models produce high-quality, diverse images from text or image prompts, while GANs remain fast and useful for realistic generation.
2
Creative AI tools can help people make images, melodies, transformed audio, memes, and stories, but their value depends on human input and iteration.
3
Copyright, attribution, consent, accessibility, and job displacement remain unresolved problems because models often train on publicly available data without clear permission.
Summary
Suyash Joshi surveys creative uses of machine learning across image generation, music, humor, and storytelling. He compares GANs, variational autoencoders, flow-based models, and diffusion models, then demonstrates tools such as Stable Diffusion, StyleGAN, Google Magenta, and MusicVAE-related projects. The examples include text-to-image generation, style transfer, melody completion, audio style transfer, a journal that turns sentiment into music, and models that produce memes or puns. Joshi is skeptical of calling someone an artist when their main action is entering a prompt and selecting an output. He also discusses the disputes around attribution, copyright, scraped training data, explicit content, and the effect of automated creative work on artists and writers. His conclusion is that these systems are useful when people provide direction, make choices, and iterate, while the legal and social questions remain open.
Diffusion models have become strong image generators because they combine quality with diversity
Joshi introduces GANs, variational autoencoders, flow-based models, and diffusion models as different families for generating images, text, music, and audio. He describes diffusion models as the current leaders for fidelity, quality, diversity, and coverage. GANs generate quickly and can produce high-quality results, but their outputs may be less diverse. Variational autoencoders and flow-based models offer fast sampling, with tradeoffs in quality or diversity. The comparison gives newcomers a practical way to think about why different generative models behave differently rather than treating every image model as interchangeable.
Text and image prompts let people generate realistic images without writing a model
Joshi explains that systems such as DALL-E, Midjourney, and Stable Diffusion accept prompts that are usually text, although an image can also provide the input. The system turns that input into an image, and the prompt can specify a subject, person, place, or artistic style. In the live Stable Diffusion demo, the audience suggests a Tower of Babel reaching the Vault of Heaven, followed by an animal prompt. Joshi says users can adjust image size, model choice, the number of images, and the level of detail in the prompt. He describes the practical workflow as forming an idea, expressing it in words, generating several images, and choosing or refining the result.
Generative art existed before neural networks and often gave the artist more direct control
Joshi traces creative computation back to Harold Cohen's AARON system in the 1960s. He then discusses generative art made with code before deep neural networks became common. He admires Zach Lieberman's work because the artist writes logic that controls the generation, including randomness, instead of only entering a prompt into a black-box model. This distinction shapes his view of authorship. Prompt-based systems still involve human decisions and curation, but Joshi questions whether selecting a generated image makes someone an AI artist when the model performs most of the production.
AI image generation raises direct questions about attribution and copyright
Joshi discusses an AI-generated artwork that sold at Christie's for almost half a million dollars even though the person who created the open-source technology received no money. He also describes an AI-generated image winning first place at the Colorado State Fair, which upset artists who did not expect a machine-generated entry to compete successfully. In another case, a person sought copyright for an image produced autonomously by an AI model, but the claim was rejected because the work lacked the human authorship required by US copyright law. Joshi adds that Getty has banned AI-generated images because their training sources and originality can be difficult to establish.
Music models can extend a person's input into melodies and new instrumental styles
Joshi turns to music generation with projects from Google's Magenta research group. A melody-completion tool fills missing parts of a four-part musical sequence, while parameters such as temperature change how conservative or random the result is. He also demonstrates AudioCipher, which converts a word into a melody and exports MIDI for use in a digital audio workstation. Magenta's audio style-transfer projects combine an input sound with another style, such as changing a saxophone-like sound into a different instrument or character. He explains that some systems use spectrograms as image-like representations, while others work with raw digital audio and differentiable signal processing.
A creative application can combine sentiment analysis with a generative music model
Joshi presents Musical Diary, an application he built with two friends during the pandemic as part of the Google Magenta and Gray Area Music AI Hackathon. Users write journal entries, and the application creates notes as they type. A sentiment analysis model scores the text, then the application uses Magenta.js and a variational autoencoder to interpolate between happier and sadder melodies. The result is a playable musical interpretation of the entry that users can replay and share. Joshi uses this project to show why he finds creative AI applications interesting: the model becomes part of a larger product with an interface, interaction, and human-written behavior.
Humor is harder for models because a joke depends on context and an exact relationship between setup and punch line
Joshi shows projects that generate memes and puns, including a web and iOS application that uses TensorFlow Lite and Core ML for mobile deployment. He is candid that the jokes he found were often weak. A joke requires common sense, unstated information, and a connection between its setup and punch line. A model may combine the opening of one joke with the ending of another without understanding why the combination fails. He connects this problem to earlier text-generation systems that produced locally sensible sentences without maintaining meaning across a longer passage. Story generation has the same problem when a game needs consistent characters, plot points, and player choices.
Creative AI will change creative work while leaving access and employment questions unresolved
Joshi closes with questions about ownership, credit, training data, accessibility, and job displacement. Models may be trained on data scraped from websites, and the permission to use that data is often unclear. Although some tools now run locally or through hosted websites, creative users still need varying levels of technical knowledge to access them. Joshi asks what happens to concept artists, illustrators, animators, musicians, and writers as models improve. He supports using the technology, while acknowledging that people may need new skills or new ways of working and that the legal and social answers are still unsettled.
"The really hard thing this year especially if you are on social media I'm sure you've seen this, realistically looking images, artful images, realistic images, illustrations that have kind of taken over almost the web."Suyash Joshi07:32
Who should watch
You are an engineer or designer who wants working examples of generative models applied to images, music, jokes, or interactive applications.
You want a practical introduction to the differences between GANs, variational autoencoders, and diffusion models before trying creative AI tools.
You need to think through copyright, training data, authorship, or the effect of generated media on creative jobs.
