Category: Uncategorized

So in  part 1 we came up with our 2 characters, a wood carver (Sandy) and a squirrel (Whiskers). We co-created a script for the scene. And part 2 explored ways the characters could communicate (subtitles) and ways an AI could direct the movement of a character by describing what should happen. So for part 3 I want to share what happened when I asked the AI to describe the squirrel’s movements.

I confess I entered into this discussion by thinking of what questions I should ask, how to describe the things I wanted, and what question order would be best for getting what I want. I was thinking to deliberately lead the AI towards a successful conversation, a set of outputs. However, I had no idea what would result from the conversation. And I’ll explain my reaction, the AI provided far more than I expected, and it really challenged the way I imagined the story. As a result, I’m definitely rethinking the layout of the scene (which currently exists only in my mind), how objects in the scene will be located such as the statue, and how I imagined the squirrel moving around the scene. So this really does feel like a collaborative co-creation.

The following chat session continues on from the start of the chat session in part 1.

You: I think we should discuss the scenery in which the above scene takes place. I imagine the scene is framed by tall trees to the left and right. A large log is on the left of the screen and the stream is visible in the background. At the beginning Sandy is sitting on that log with a fishing rod and line in the stream. The ground is clear across the front of the scene where our characters will perform their interaction. Do you think that would suit our story? Is there anything else we need for either Sandy or the squirrel to act out the scene?

Bot: I think that sounds like a wonderfully picturesque setting for our story. The tall trees provide a sense of enclosure, making the scene feel intimate and private. The log becomes not only a seat for Sandy but also a stage where Whiskers can show off his acorn. The stream in the background contributes a peaceful ambiance, complementing the relaxed tone of the story.

For the characters, I think Sandy would need a bag or box for his carving tools and the block of wood. As for Whiskers, maybe there could be a little pile of acorns nearby at the beginning, which could be the source of the one he gifts to Sandy. This could signify his willingness to share what he has with his new friend.

You: Wonderful. So thinking of that scene can you list all of the locations, essentially the stage marks, that the little squirrel would move between?

Bot: Absolutely! Here are the stage marks for Whiskers, the squirrel:

1. Acorn Pile: To the right of the log, this is where Whiskers grabs one of his precious acorns to give to Sandy.

2. Log: This is where Whiskers gives the acorn to Sandy. It's also where they converse and where Whiskers poses for the statue.

3. Ground in Front of the Log: This is where Whiskers excitedly scampers in a circle when Sandy suggests making a statue.

4. Statue: Positioned near the log, this is where the statue ends up once Sandy's done carving it, and Whiskers comes here to admire the final masterpiece.

5. Sandy's Shoulder: Whiskers clambers up here when Sandy invites him to his cabin for supper.

Remember, Whiskers is a small creature and likes to dart around quickly, so he could also pop up on the log, around the ground near Sandy, or even in the trees at times to make the scene lively and dynamic.

You: What types of movement do you think would help bring the squirrel to life? You've mentioned scampering and darting around before.

Bot: Whiskers the squirrel is a ball of energy and curiosity, so his movements should reflect his lively character. Here are some types of movements that can help bring Whiskers to life:

1. Twitching: Squirrels are known for their constant twitching, especially their tails. This twitching can be used to show excitement or agitation.

2. Leaping: Squirrels are great acrobats. Leaping, whether it's from the ground to the log or among the trees, will showcase his agility.

3. Quick Head Movements: Squirrels are always alert and attentive. Quick, jerky head movements can reflect their high energy and constant vigilance.

4. Flicking Tail: Whiskers can flick his tail in excitement or use it for balance while he's on the move. 

5. Nibbling: To illustrate his love for acorns, Whiskers could occasionally be seen nibbling his acorn, turning it around in his tiny paws.

6. Clasping: When Whiskers is holding the acorn or clinging onto Sandy's shoulder, his little paws can clasp and unclasp, highlighting his careful handling or secure grip.

These movements, combined with our little squirrel's expressive eyes and chittering sounds, will surely make Whiskers come alive in our story.


You: Considering this part of the above scene:
Whiskers, from high in his tree, notices Sandy's tuneful whistling. His curiosity piqued, he decides to descend from his tree and scampers towards the stream, rustling some leaves as he goes.

Can you list Whiskers' movements as a series of steps, including moves from one of the above stage marks to another, any pauses, or any of the other movements you mentioned above?

Bot: Absolutely, let's break it down:

1. Whiskers begins in the tree (Stage Mark: Tree), his attention grabbed by Sandy's whistling. He twitches his tail in anticipation and flicks his ears towards the sound.

2. He decides to descend, leaping from branch to branch with agility and quickness.

3. He makes a pause on the last branch before the ground, his head making a quick back and forth movement as he calculates his jump.

4. He jumps from the tree onto the ground (Stage Mark: Ground near Tree). 

5. Hearing the rustling leaves as he lands, he freezes for a moment, his tail twitching nervously.

6. After a pause to ensure all is safe, he briskly moves towards the log where Sandy is seated (Stage Mark: Log), scampering across the ground and rustling the leaves.

7. Arriving at the log, he pauses again, his quick head movements surveying Sandy before making his next move. 

8. Finally gathering his courage, he leaps onto the log to interact with Sandy.

These series of movements not only showcase Whiskers' physical agility but also highlight his cautious, curious nature.

You: Brilliant. Now considering the next part of the above scene:
Whiskers tilts his head, considering the invitation. After a moment, he scampers onto the log next to Sandy, settling down comfortably. He continues to watch the dancing fishing line with interest, every now and then glancing up at Sandy with a curious chirrup.

Can you list Whiskers' movements as a series of steps for this part, again including moves from one of the above stage marks to another, any pauses, or any of the other movements you mentioned above?

Bot: Of course! Let's break down Whiskers' movements for this part:

1. After hearing Sandy's invitation, Whiskers pauses on the log (Stage Mark: Log), tilting his head in contemplation. His quick, darting eyes show he's thinking.

2. Deciding to accept the invitation, he scampers along the log towards Sandy, his tail flicking in excitement.

3. Once next to Sandy, he settles down, taking a comfortable position. His body relaxes but his tail continues to twitch, indicating he's still alert.

4. His eyes fixate on the dancing fishing line in the stream (Stage Mark: Stream), reflecting his intrigue. He leans forward slightly, showing the intensity of his interest.

5. Occasionally, he breaks his gaze from the fishing line to glance up at Sandy, giving a curious chirrup. With each glance, he tilts his head a bit, signifying his curiosity.

6. His tiny paws clasp and unclasp, reflecting his desire to touch the fishing line but maintaining respectful distance from Sandy's activity.

These movements showcase Whiskers' curious and respectful nature. They demonstrate that Whiskers is both interested in Sandy's activity and aware of the boundary between them.

You: Great. Now consider this next part of the above scene:
Whiskers perks up at the mention of carving. He chitters excitedly, scampering off the log and running towards one of the nearby trees. He comes back moment later, holding a fallen acorn in his paws. Whiskers drops the acorn in Sandy's lap, looking at it and then Sandy expectantly. Was his suggestion of carving an acorn statue too ridiculous?

Can you list Whiskers' movements as a series of steps for this part, once again including moves from one of the above stage marks to another, any pauses, or any of the other movements you mentioned above? It might be tricky for Whiskers to drop an acorn in Sandy's lap, so he could just hold on to it, presenting it to Sandy.

Bot: Absolutely, let's break down the movements for this part:

1. Whiskers perks up on the log (Stage Mark: Log), his ears pricking up and his tail twitching at the mention of carving.

2. He chitters excitedly, scampering off the log towards the acorn pile (Stage Mark: Acorn Pile).

3. After a quick search, he picks up an acorn, holding it carefully in his paws. His tail flicks as he looks at the acorn, pleased with his find.

4. He runs back to the log, his body low and tail streaming behind him.

5. When at the log again, he stands on his hind legs, lifting the acorn in his forepaws to present it to Sandy. He pauses in this position, his body a little tense as he waits for Sandy's reaction.

6. His eyes dart quickly between the acorn and Sandy, indicating his hope that Sandy will get his suggestion of carving an acorn statue. His tail gives a nervous twitch with the anticipation.

These actions clearly demonstrate Whiskers' excitement at the idea of an acorn carving and his hopeful anticipation of Sandy's response.

You: Wonderful. Ok, now consider the next part of the above scene:
Whiskers looks at Sandy, surprise clear in his eyes. He chitters excitedly, scampering in a little circle before settling down again. He then poses with the acorn, as if giving Sandy his approval to proceed with the new project.

For this part can you list Whiskers' movements as a series of steps, including moves from one of the above stage marks to another, include any pauses, or any of the other movements you've mentioned?

Bot: Certainly! Here are Whiskers' movements for this scene:

1. Whiskers starts on the log (Stage Mark: Log), looking at Sandy with surprise in his eyes. His tail flicks and his head tilts slightly, showcasing his astonishment.

2. He releases an excited chitter and scrambles off the log onto the ground, scampering in a small circle (Stage Mark: Ground in Front of the Log). This movement emphasizes his uncontained joy.

3. After his brief bout of excitement, he pauses for a moment to catch his breath, his sides slightly heaving and his tail twitching.

4. He then bounds back on the log next to Sandy, settling down near him and the acorn.

5. He poses with the acorn, delicately holding it in his paws as he looks at Sandy, giving a pause as if to get Sandy's attention.

6. He then continues to hold the pose, his body stiffening a bit as if giving Sandy his approval for the new project, his tail flicking in excitement.

These movements show a range of emotions from Whiskers - his surprise at Sandy's acceptance, his joy at the prospect of an acorn statue, and his implicit approval for Sandy to begin.

You: Very good. So consider this next part of the above scene:
Whiskers climbs onto the log, standing beside the statue. He chitters approvingly, his eyes wide with admiration. He runs his paws over the wooden version of himself, his tiny fingers tracing the intricate details that Sandy had carved.

Could you list Whiskers' movements as a series of steps for this part, including moves from one of the above stage marks to another, any pauses, and any other movements you mentioned above?

Bot: Certainly, let's break down Whiskers' movements:

1. Whiskers begins on the ground (Stage Mark: Ground in Front of the Log), looking at the statue on the log.

2. He clambers onto the log (Stage Mark: Log), his movements quick and agile as he approaches the statue.

3. Standing next to the statue, he pauses and looks it over, his tail twitching and eyes wide with admiration. His head may tilt from side to side as he takes in the details.

4. He then lets out an approving chitter, his rapid head movements indicating his excitement.

5. Next, he reaches out with his paws to touch the statue, pausing momentarily as if asking for permission.

6. He then runs his tiny fingers over the statue, tracing the intricate details carefully. His paws clasp and unclasp as they explore the carved wood.

7. Throughout this, he occasionally glances at Sandy, his eyes still wide with admiration and gratitude.

These movements show Whiskers' admiration for the statue and his recognition of Sandy's talent and effort.

You: Nice. And now consider the final part of the scene.
With a swift nod, Whiskers accepts the invitation. He clings onto Sandy's shoulder as they make their way back to the cabin, their journey lit by the dwindling sunlight filtering through the autumn leaves. Their friendship promises many more shared adventures and warm evenings by the fireplace.

Once more can you list Whiskers' movements as a series of steps for this part, including moves from one of the above stage marks to another, any pauses, or any of the other movements you mentioned above? Perhaps have Whiskers follow Sandy and not cling to his shoulder as that might be hard for us to co-ordinate.

Bot: Absolutely, let's consider this final part:

1. Whiskers begins on the log (Stage Mark: Log), facing Sandy as he is invited for a warm evening by the fireplace.

2. He responds with a swift nod, his ears flicking and tail twitching to indicate his acceptance.

3. He then carefully hops down from the log onto the ground (Stage Mark: Ground).

4. Whiskers pauses for a moment, as if to gauge Sandy's direction.

5. Once Sandy begins moving, Whiskers starts to follow him closely, his movements quick and light as he scampers across the ground.

6. While following Sandy, he occasionally pauses and looks around, his movements cautious yet excited, capturing the dwindling sunlight and the autumn leaves around them.

7. As they reach the cabin (Stage Mark: Cabin), Whiskers pauses one last time, taking in the welcoming sight before scampering up the stairs after Sandy.

These movements show Whiskers' acceptance of the invitation and his anticipation for a warm evening by the fireplace, depicting the start of an exciting friendship full of shared adventures.


You: Wow, that's an awesome description of all the movements. I'll try to animate Whiskers for you following each of these steps and instructions, as much as I can. And I'll choreograph Sandy to perform together with Whiskers in the scene. Any final words of advice or things you'd like to see?

Bot: I'm glad to hear you found the descriptions helpful! For the animation, focus on capturing Whiskers' characteristics and emotions through his movements. He's described as curious, excited, respectful, and appreciative. Express these traits through his actions.

Keep his movements quick and nimble to reflect his energetic and lively nature. However, while he is active, he is also cautious and considerate, so remember to include moments of hesitations and pauses.

Lastly, it would be wonderful to see the bond between Whiskers and Sandy being portrayed strongly in the animation. Their friendship is a key element of the story.

I'm looking forward to seeing your animation. Good luck!

(Feature image courtesy of Alan D Thompson. https://lifearchitect.ai/models)

As a part of a larger project I thought I’d read a bit into AI rather than just pick a tool and not know what I was doing. Well I learned a lot, which I’ll share here, but there’s probably still a lot more to learn. And I’ve found some references which I think are handy, or at least interesting for you to also browse, so I’ve included them below. Let’s discuss!

The number of parameters in a model seems roughly like the number of neurons or brain-power size, probably not a direct comparison between products as their approach differs. The training dataset size might make a big difference and also the make up of the training data used, as some focus on carefully selected content (expertise?) over broader content (wider human culture?). The majority of recent and accelerated growth in AI models seems to be due to these 2, the growing number of parameters and increasingly larger datasets. The number of tokens in a model I believe can be likened to it’s vocabulary size. And many models are trained on internet data available up to a certain date, or training data available date, so be aware of this if recent world information is important. Something else to be aware of, some interfaces and products say they will choose from different underlying models depending on the situation, in principle to optimise the outcome, but I imagine this might also depend the developer or user’s subscription budget. So you may not always know what you’re using. I’ve found you can try ask a model for more information about itself, but OpenAI for example will only tell you the training data available date, but it doesn’t seem able to (or willing to) provide all the details.

OpenAI’s GPT4 seems to still be a go-to for industry comparison, consistently at or near the top with the most complex model (largest number of parameters) and perhaps it’s the easiest to get started with (and/or pay for). Or perhaps that’s OpenAI’s marketing at work, as ChatGPT seems to be a name lots of people recognise and many people have tried, experts and non-experts. There are heaps of different benchmarks, but GPT4 seems consistently of high quality and often a benchmark that everyone aims to beat. But there is definitely no “right answer” as many AI’s are designed to be stronger in different areas.

Try to be aware of an AI’s capabilities, for example use this link to look up OpenAI model capabilities. Notice that if you just select GPT-4 on their web site or perhaps in other apps, that is currently shorthand for gpt-4-0613 and this model has 8k context tokens (8×1024, or roughly 8,000) and uses training data available up to Sept 2021. (correct at the time of writing this) A better alternative for long form creativity could be to choose a model with 32k tokens. OpenAI says that 1 token is the equivalent of about 0.75 words. So 8k is 6,144 words which is about 22 pages of a novel (assuming 280 words per page average), and 32k at 4 times larger would be about 88 pages of a novel. But what is context? AI is essentially “stateless”, meaning it has no memory, so without you seeing it happen the recent memory or history of your conversation (both prompts and responses) is sent each time to the AI as the “context” for your next prompt. That’s why you can come back to any chat conversation at a later time, the AI doesn’t magically remember where you left off, the software stores and sends the full context to the AI again. I hope I explained that well enough. So the maximum number of context tokens is essentially the maximum size of the AI’s short-term memory. Now, depending on how each chat algorithm works, as the context limit is reached they could either remove (forget) the oldest messages, or they could summarise (shorten) a section of the older conversations. If you’re having a quick chat or writing a short poem this probably doesn’t matter. But if you’re getting the AI to help write a manuscript having over “22 pages” of short-term memory could start to matter.

References:
A Comparative Analysis of Leading LLMs, updated Mar 2024. https://mindsdb.com/blog/navigating-the-llm-landscape-a-comparative-analysis-of-leading-large-language-models
A collection of model comparisons can be found here. https://lifearchitect.ai/models/
And you can even see some benchmarking results as people submit them, eg. the LMSys Chatbot Arena Leaderboard. https://huggingface.co/collections/open-llm-leaderboard/the-big-benchmarks-collection-64faca6335a7fc7d4ffe974a
And the description of some benchmarks can be found here. https://www.vellum.ai/blog/llm-benchmarks-overview-limits-and-model-comparison

Responsible Use of AI Technology

Maybe some final AI threads for others to follow, if interested, on the responsible development and/or use of AI. I think these things we still need to watch unfold, and see if the discussions, policies, companies and the world converge or diverge.

In 2021 UNESCO adopted an agreement to support 193 member states in implementing and reporting on a set of recommendations on the ethics of AI. Several companies have signed an agreement with UNESCO at the Global Forum on Ethics of AI in February 2024 to develop more ethical AI (Microsoft, Mastercard, Lenovo, GSMA, INNIT, LG, Salesforce, Telefonica). While OpenAI has a major partnership with Microsoft, they don’t appear to have a position on the UNESCO’s recommendations. In fact a recent UNESCO study seems to point fingers at biases within OpenAI’s older models and their closed development approach. The results however, for the more recent ChatGPT model (see figure 1) showed a significant shift towards more positive and neutral content generation. I’m sure UNESCO are hoping studies such as this draw more attention to their recommendations and may even influence global AI benchmarking. So I think it’s fair to say there has been a lot of effort put into OpenAI’s approach to safety and the pressure is not coming off anytime soon.

The final thread I’ll mention is the environmental impact of the computing power necessary to train these models. I think it’s clear that while the AI buzz is booming this will be an issue. This article suggests that in 2019 the power used for training one AI model is 20 times larger than the power an average American uses in a year, and the power used just keeps increasing for larger and larger models. Sound like a lot? Sure, but how many people then use that one model, millions? An AI once trained can be used over and over again, and in fact for some significant tasks AI is being used to significantly reduce the need for computational power (engineering and simulation, architecture and rendering). The buzz may be driving up speculative investment and growth, but eventually companies and models will only remain viable if their cost of growth remains lower than what people are willing to pay. The more a pre-trained model is used the better, and the more people who re-use it the better, and the more AI companies can reduce their training costs the better. So the end point of this thread isn’t clear to me, I don’t think the market will tolerate infinite growth, and I think there’s a chance the benefits could look similar to cloud computing – where the total size and growth is still very large, but the net benefit for applications moving to use the cloud is an 87% decrease in energy consumption (see this article).