Unlocking new creative frontiers

Transcript

Unlocking new creative frontiers

Sebastian: Hi, so I'm going to take you through a talk I gave recently to the creative faculty at Kingston University. It's called Unlocking New Creative Frontiers. It's my point of view on what this moment in AI means for the creative industry.

I can now build the things I design

I can now build the things that I design. I spent 20 years designing products before I could build software myself. That's what is so meaningful about this moment. These days, I run an AI company with my co-founder. We help companies make sense of this moment in AI.

How good is AI now?

I want to anchor us in how good AI is now. There's a lot of talk about AI slop and there's also a lot of talk about AI doom and how it might end all things. But I want to take us to a shorter focal point of the next three to six months. Last week or a week before, OpenAI reported a solution to the roughly 90-year-old Navier–Stokes problem. It's a mathematics problem around how fluids move. It's a big deal in the math community.

I think similarly there's going to be a lot of big deals like this across a lot of different industries. I think the thing that's difficult right now is it's like, well, okay, I'm not solving any unsolved math theorems. So why do I need this level of intelligence?

What smarter models unlock

What I notice is that the smarter these models get, the more capable they are becoming at dissolving technical boundaries within more and more domains. This started with code. Now everyday people can build software and games. Increasingly, new technical capabilities are becoming available to everyday folks. That can include making games, filmmaking, animation, video editing, visual effects, 3D, fabrication, hardware. At least in the creative world, that is the most foundational shift these models are unlocking.

You can build an agent and have it read your email, but the ability to work in these technical domains is, to me, the most transformative thing these tools are making possible.

Generative imagery and style transfer

I want to give you a whirlwind tour of creative capabilities that have emerged in the last few weeks. Starting off with imagery. So, ChatGPT Images 2.5, an update to OpenAI's generative imagery model, is incredibly good at style transfer. Style transfer is the idea of taking an image and a style and applying that style to the image. You can see here a metallic figure applied to a snake. Simple idea, but historically wasn't very good. And so, this is a very cool capability. You can imagine what else you might do with it.

Generative imagery with Midjourney

Next is generative imagery using Midjourney, a creative tool that can produce stunning imagery. This is an artist called Olga Volosin. It's beautiful work.

Turning images into editable graphics

Next, I want to talk about an emerging capability: tool use. What you're seeing here moves quite fast. It's a generated poster image, perhaps made with ChatGPT image generation, then turned into an editable graphic through a Figma connector. So you can go from a generative image to an editable graphic that can be put into print or published elsewhere.

The beauty of living frames

Next is Living Stills by an artist called Oak, using Midjourney to create beautiful stills and then using one of the best video models called Seedance 2 to bring them to life. It gives you a sense of the quality and realism these models can achieve. We are a long way away from the six fingers and imagery that just looks like AI slop. We can make beautiful creative work.

The Cully Hill Boys

Next up is the Cully Hill Boys. This is a project that is a full-on motion picture. It's two hours long and made entirely with AI. But the thing that's notable about this is that it is an open source project that was commissioned by Higgsfield, which means you can go into the platform and see every one of the 470,000 assets that they created to make this video or this film. It's an incredible look behind the scenes at the techniques and capabilities involved.

Creature animation

Next is an artist called Shin, exploring bringing extinct species to life, starting off with generative imagery, and then using motion capture to animate and bring them to life. Here is another example of what newer models can do. This one uses GPT-6 Astra.

Video editing

Here is GPT-6 Astra using a computer to edit video in CapCut. I think video editing matters in a moment when people are reading less. The ability to craft stories through video is increasingly powerful, but it has been hard to access. Video editing is very hard, and the tools to do it are very complex. I'm excited about AI helping more people tell stories through video.

Gesture-based video editing

Here you have the Higgsfield team playing around with gesture-based video editing, because, why not?

Colour grading

Next, Higgsfield is doing colour grading. Again, colour grading is an incredibly difficult craft, something that takes a long time to master. Having AI use these complex tools to help with colour grading is powerful.

Music and composition

Here you see how it comes together. Now, again, this is maybe a taste of the future rather than something that's going to be incredibly powerful right now. But here you have Premiere Pro, DaVinci Resolve and Ableton Live, all working together to kind of create a composition. So video editing, colour grading, and sound and music composition, all being supported by GPT-6 Astra. This is a world we're going to live in. And these capabilities are going to be available to you.

Packaging from a dieline

Next up, we have Salma. She is an incredibly talented product designer, works in brand for cosmetics. And here she is exploring bringing a dieline to life. A dieline is the flat, unfolded plan of a package. Using Blender, an open-source 3D tool, and its MCP connector, she brought it to life and tested how it would work in the real world.

3D model and animation

Next up, we have a designer who took a static image that they found and wanted to bring it to life. Hey, can you create this in 3D, rotate it, and give me an exploded keypad view? And they made this. Again, I'm looking at some of these things and I can't believe that this is actually possible now. That's something I could go off and do, but it is here. And it's only getting better.

Garment and material visualisation

So here again, Salma from before, exploring material and fashion design inside Blender using the Blender MCP. It's remarkable.

Interface components

Next up, we jump into product design and interfaces. Here is a designer who created a design system and then is using Astra to basically procedurally generate infinite iterations of the system that they've designed. I think again in product, it's increasingly going to be the way that we work, in which we work on designing the systems and then use AI to procedurally generate these iterations.

Interactive product configuration

Next up we have a designer called Daria who does incredible e-commerce experiences for product and interior. And so this is all enabled by Three.js, a 3D library built for the web and allows for these incredible e-commerce experiences where you can dynamically reshape furniture, play around and transition between different variants. It's just really amazing stuff that I haven't seen before, but again, is now within our capabilities. This is something we could build together. Let's do it.

A Catan-inspired game

Next up, we have Meng To, a designer who has built out and rebuilt Catan as a game. And I think that the reason gaming is just a very powerful example of capability is because it combines so many different sorts of technical capabilities from sound design, storytelling, 3D, graphics. It's an incredibly challenging thing to do technically. So the capabilities of these models to be able to generate games is increasingly very impressive.

Vesper

Here too, another game created by a creator. And again, you can just see what's possible with using tools like Blender and the latest AI models.

God’s Eye View

And then we have Bilawal Sidhu, who has recently open sourced this incredible project called God's Eye View. His thing was, what if I created a vibe-coded Palantir? Obviously, this isn't Palantir, but what he's done is taken dozens of open source datasets from flight data, shipping data, satellite, earthquakes, you name it. He has a background in sort of mapping and spatial 3D and has been able to create this very responsive worldview that you can play around with and see what happens when you overlay data.

What is incredible is that this is an open-source project, right? So you can go to GitHub, grab this project, download it, and then if you wanted to do something that isn't a Palantir duplicate, but it's some kind of interesting project mapping datasets onto maps, you can just point your agent at this dataset and be like, hey, look at how they did it. This is what I want to do.

And so this idea of being able to take codebases and remix them to do new creative things, I really implore you to check out this project. It's really incredible.

Solaris

So now just a couple of tastes of the future. This is a speculative project by a company called Runway. They're similar to Higgsfield, a generative video and editing platform. And what they've done is create a thing called an interface world model. So you have world models, which are essentially infinitely generated worlds. You know, you could be a chipmunk in a giant garden, and it procedurally generates everything as you go. But what's different here is this is an interface world model. And so what you can do is essentially create any scene.

Take an image of a salad. Then you create your parameters around how the interface is going to work and then you're able to interact with it in real time. This is really being unlocked by high quality generative video now being able to be generated in real time. So if you think about this conceptually, right, is if you can generate faster than real time, then you can essentially have totally interactive customizable video, right? So a different company called fal.ai has a website called fal.live where essentially you have interactive video.

It just runs and whatever you decide, it becomes the thing. But the reason I bring this up is to give a taste of what future digital experiences might look like, how rich, how high fidelity those experiences might be, is just something that's pretty crazy to think about. And just provides a lot of possibilities. We've seen this demo, Ikea has tried to do this before, and all sorts, and it's possible today, but technically very complicated. Within the system, you could do anything like this. So the possibilities are endless.

Here, you want to change the time of day, scrub to the right.

Digital jigs and bespoke creative tools

And so finally, I want to kind of talk a little bit about this concept of digital jigs. And this is the idea, you know, what I've shown so far is basically creating, using AI tools to create the final products. But in order to get to the final products, there's often times where you're having to create tooling. And this becomes more and more possible, right?

So this idea that instead of having to use your existing creative tooling in order to get the results you're trying to get, you can actually create off the cuff throwaway tooling to get that thing. You're creating a brand that has an incredible wavy pattern or glass effect or mosaic pattern. You can create tools just for that specific problem to be able to solve that creative solution. So I think that is very exciting.

One idea across several media

Okay, so I’ve shown you different capabilities. Let me just quickly do a run through of what that looks like in an everyday creative process. So we had a workshop recently where I wanted to demo the ability to create 3D objects. So in order to do that, I had an idea around creating a vinyl object inspired by kind of sculpture and colour theory.

Came up with this guy that I thought was quite interesting, was able to generate a bunch of different ones, and then was able to then turn it into a 3D object using some of the 3D models. So, you know, you have one object, ask it to create a 360 view of the different ones, and then upload it to a model to turn it into 3D. But it's like, oh, okay, well, maybe we can take it further. What if we had a commercial for it?

Then you can take it into a storyboard. Here, the idea was like, okay, what if we crossed an Apple promo video for a new piece of hardware with like a hip-hop music video? Like, what would that look like? So again, now you can then quickly mock that up as storyboards. And then you think, why not animate that? You upload the 360 view, the storyboard, and your concept, and then you can have a full-on ad.

This was done in about 20 or 30 minutes before a workshop. And the reason I say that is it's like the time between an idea and something that's tangible has collapsed so dramatically. And I think that's such an exciting thing because if you think about feedback loops, right, if you're trying to learn an instrument, say the piano, and you had to wait hours until you heard the note that you press, yes, you can learn it, but it takes a long time, right? Because you don't get that feedback loop.

Condensing creative feedback loops is powerful. I wanted to show what that looks like: taking ideas and sketching them with current capabilities.

Conceiving and making are coming together

But basically, conceiving and making are coming together, right? So an illustrator can animate an illustration. An illustrator making artwork for a game can explore building that game, right? So why not just make the game? And then in a world where an illustrator can animate and an illustrator can make a game, what does the label illustrator actually serve that person? And so I think increasingly this idea of like I'm a graphic designer, I'm an illustrator, I'm a fashion designer, these boundaries are going to start to dissolve.

And we're seeing that in product teams already, right? So in product, you have the boundaries between what a designer does, what a UX researcher does, what a product manager does, and what an engineer does are all kind of getting fuzzy. Everyone's starting to live in their lane. And I think creatively also, you want to be careful about these identities because they could be a limiting belief in a world where your neighboring technical capabilities are actually also totally accessible to you.

Creative direction and authorship

Creative direction and authorship. So I think this one is super interesting. So I can guide work and I can stand behind it. I can't really claim that I did it myself. Now, I have a background in art direction. And so for me, that hasn't been such a difficult shift because transitioning from directing creative by people to doing hybrid situations where I'm directing AIs or combinations of AIs is just not such a conceptual shift. But for a lot of creative domains, there is a lot of emphasis on authorship.

And so, in a world where you are collaborating with AI, you know, this sort of idea around authorship is an interesting one. I think that's going to have to evolve.

Clients can commission a creative system

So, increasingly, you're going to have clients that can commission a creative system rather than individual creative artifacts, right? I've started to see this happening already. So, some of the last creative projects that I art directed at Google were for creative systems rather than artifacts. What you'll get is these systems that you can then generate infinite things. So, like, to make that tangible, imagine it's an illustration system, right?

So, rather than getting five or ten illustrations for specific use cases, I'm getting that illustration system that I can then create infinite types of illustrations from that. And so, that's a different business model. It's a different licensing. It's just a whole different kind of ballgame. But I think increasingly, creatives are going to be commissioned for systems rather than artifacts.

Become more ambitious with your ideas

Next is becoming more ambitious with your ideas. The thing here is we've all been trained with what is possible over the last 20 years. And that has been dramatically redefined. And so I find this for myself. I'll have an idea, and it just feels too big, too complicated, and I just let it drift. It happens almost before I notice it. But we now live in a world where I could take that idea and just throw it at an AI partner and see what happens.

And so it's kind of a subtle thing, but it's not just training yourself to be more ambitious, but it's also training yourself to catch ambitious ideas before you throw them out because of prior assumptions about what is and isn't possible. So I think it's something we need to be really cognizant of.

Reevaluate what good creative work can achieve

I think then we're also going to have to reevaluate what good creative work can achieve. Again, in this sort of light of ambition, in a world where you can take an image, turn it into a film or an animation, turn it into a game. There's a lot more that is possible. And so especially in the context of young creatives, what I would push them to do is to strive much bigger. Like we all need to get more ambitious about what is possible.

And the industry is also going to put that pressure on us in the expectations of what is possible. And so it's something that I think we need to kind of reevaluate.

I feel both anxious and excited

So the next. step is I feel both anxious and excited And I think this is a really important thing to be able to hold that cognitive dissonance of being both anxious because, hey, it is incredible amounts of change moving faster than we have ever experienced in our lifetimes. And then on the other hand, it's incredible opportunity. And so I think we need to hold both of them. And that's something that isn't necessarily that easy

Building my first website and software

I think back to the first time building my first piece of software or my first website by hand after 20 years of building products. It's just such an intoxicating feeling to have that kind of agency to bring things into the world with your own hands. And so, again, just, it's an exciting, it's an exciting moment to be a creative. It's an exciting moment to be somebody with curiosity.

Storytelling and building

And then finally, when I ever get lost in this, like, where does a human fit into all of this? I come back to two things, storytelling and building. Since we've called ourselves humans, we have been storytellers and we've been building things. And there's no world in which those two skill sets and developing those two skill sets are not incredibly important. So whenever you get lost, fall back on storytelling and building.

AI takes practice

And then finally, AI takes practice. AI is an experiential technology, similar to playing an instrument or learning a language. And so there is no way to create that sense or that proficiency with this technology without practicing it. So even if you're anxious about it, especially if you're anxious about it, get in there, experiment, play, learn. It's the only way to build a relationship and an understanding of this technology that will shape our futures.

Let’s try it

So I implore you: let's try it. Try some image making, jump into some generative imagery, try building with AI and see how that goes. Thanks.