I’m starting off a new project. And this one is inspired by a recent UNIMA Youth Commission session about AI & Puppetry. The session prompted thinking about the faces of AI, and one day perhaps AI could be “person” with whom we will collaborate, not just a “fake” creator or tool for creators. While selfaware AGI (artificial general intelligence) is likely a long way off, I think we could try that collaboration right now. Or at least I’d like to know how far we can push it now and what sort of result might be possible. So here’s the rough plan…

  • Create a short story involving 2 characters, one created by me, the other by the AI.
  • Find a way for us both to manipulate elements of a scene. I’ll use Blender for this, having learned a lot during my last project.
  • Finally see if I get AI to choreograph puppet movements on screen. This could be the most interesting challenge, will take some thinking.

If others were interested or are experimenting themselves, leave a comment. How do feel about “tinkering with the garage door open”, letting us all know what you’re up to?

I’ll start by sharing a quote shared by Matt Smith from the UNIMA AI session.

My digital puppet collaborators are not machines, but they do learn about, and defer to, my human preferences while – at times – rebelling against my controlling nature. They experience glitches, and have accidents, and it is these that give them credibility. For me, a “successful collaboration between humans and machines” looks like artificial digital puppet life and intelligence

Lucy Childs (2022). The Quest for Life and Intelligence in Digital Puppets (Doctoral dissertation). p186-187

Choosing an AI (large language model)

I thought I’d read into this a bit, because you should know your tools, right? Well it got pretty technical and long-winded, so I’ve included what I found in a different blog post. In the end I felt pretty comfortable going with a later model of OpenAI’s GPT, specifically the gpt-4-0613 model. Oh, I also use the Android app ChatBoost because I can connect it to a paid OpenAI account and select a specific model to use for chat.

Part 1 – Creating a short story

Below I’ve included the discussion I had using gpt-4 to co-create a short story or scene. I think it went pretty well. Towards the end it stopped taking turns, perhaps decided to get a bit pushy with me 😉 but we got back on track.

You: Hi there. I'd like to try an experiment where we co-create the script for a scene together. The scene will involve two characters, one will be played by me, the other by you. The characters will be played by shadow puppets so they could be human, animals or fantasy creatures. I'll start by describing my character, then you can describe yours, then we can take turns saying what our character does next. Let me know if that makes sense to you.

Bot: Absolutely, that sounds like a fun and creative experiment! I'm ready to co-create this shadow puppet scene with you. Go ahead and describe your character.

You: My character is called Sandy. Sandy is a woodcarver who lives alone in a cabin by the woods. By day Sandy loves to walk through the woods, whistle tunes and work on carvings. At night Sandy loves to sit by the fire and eat fish caught from a stream. Your turn.

Bot: My character is named Whiskers. Whiskers is an adventurous squirrel who resides high in a tree just on the edge of the woods. During the day, he explores the woods, gathering nuts and playing with the other forest creatures. At night, he enjoys nestling in the branches of his tree, munching on acorns and listening to the sounds of the night. Now, let's start our scene.

You: Sandy is sitting on a log by a stream fishing. He is whistling a tune.

Bot: Whiskers, from high in his tree, notices Sandy's tuneful whistling. His curiosity piqued, he decides to descend from his tree and scampers towards the stream, rustling some leaves as he goes.

You: Sandy stops and say, "Well hello little squirrel. What brings you to this part of the woods on such a lovely Autumn day?"

Bot: Whiskers looks up at Sandy, eyes gleaming with curiosity. He makes a series of chittering noises and gestures towards the fishing line dancing on the surface of the stream. His way of asking what Sandy is doing.

You: "Ah, you mean my fishing line." Sandy holds up his line. "Well I'm trying to catch a fish for my supper. But I also don't mind sitting by the stream and enjoying the sun. Would you like to join me?"

Bot: Whiskers tilts his head, considering the invitation. After a moment, he scampers onto the log next to Sandy, settling down comfortably. He continues to watch the dancing fishing line with interest, every now and then glancing up at Sandy with a curious chirrup.

You: After a short while relaxing in the sun Sandy turns and says, "I have enjoyed your company little squirrel. And I would stay for longer, on any other day, but I'm afraid today I must get back to work. You see, the King has asked me to carve a statue as a present for his daughter. I have found the perfect piece of wood. But I still do not know what to carve."

Bot: Whiskers perks up at the mention of carving. He chitters excitedly, scampering off the log and running towards one of the nearby trees. He comes back moment later, holding a fallen acorn in his paws. Whiskers drops the acorn in Sandy's lap, looking at it and then Sandy expectantly. Was his suggestion of carving an acorn statue too ridiculous?

You: Sandy says, "My dear little friend, what a perfect idea! So simple and beautiful. Would you mind if I carved a statue of you holding the acorn?"

Bot: Whiskers looks at Sandy, surprise clear in his eyes. He chitters excitedly, scampering in a little circle before settling down again. He then poses with the acorn, as if giving Sandy his approval to proceed with the new project.

You: Sandy races off to get his tools and the wood to carve. In a moment he is back and working with joy and purpose on the block of wood. In what seems like no time at all, the wood is carved away, and what remains is a perfect statue of squirrel proudly holding a shiny acorn. Sandy sighs and steps back to look at his work.

Bot: Whiskers climbs onto the log, standing beside the statue. He chitters approvingly, his eyes wide with admiration. He runs his paws over the wooden version of himself, his tiny fingers tracing the intricate details that Sandy had carved.

"Well done, my friend," Sandy says to Whiskers, patting him gently. "This is a masterpiece, thanks to your inspiration."

The sun sets, painting the sky in hues of orange and purple. Happiness filled the air as Sandy packed up his tools and Whiskers scampered around the statue. It was a day well spent, and an unlikely friendship formed in the heart of the forest.

You: Hey, I was supposed to say what Sandy says. Couldn't hold back there, huh? It really does seem like a good place to finish the scene. But I'll add on more thing. Sandy says, "Would you like to come back to my cabin for supper? It's small but cozy with a warm fireplace."

Bot: With a swift nod, Whiskers accepts the invitation. He clings onto Sandy's shoulder as they make their way back to the cabin, their journey lit by the dwindling sunlight filtering through the autumn leaves. Their friendship promises many more shared adventures and warm evenings by the fireplace.

You: The end. Well done, you! It has been fun co-creating this story with you.

Bot: I enjoyed it too! Thank you for the creative scenario. I'm here if you want to create another story or need assistance with anything else.