Claude Code Took Over Blender and Created This... transcript

I equipped my Claude Cote with blender skill. Also I gave him access to best APIs such as 3d generation, image generation, computer vision and much more, enabled bypass permission mode and told him to do one task to create me this location in 3d. I let this guy cook as much as he can and what happened is not what you expected guys. I gonna reveal to you soon but first why Claude Cote. Claude himself is absolute beast in coding and blender behind the scene is basically python scripts and we also got an amazing blender mcp recently.

But Claude Cote allow you to scale it even further and remove all of that headache with ui setting up skills and stuff. You can simply run Claude with dangerously skip permission and let this guy cook which we're gonna do in this video. I gonna show you some cool really usable examples. So first how do you install it. What you see on my screen is blender 5.1 open. So that's basically the latest version that we need. We need blender and official blender mcp.

Again this is like a new official mcp which is released just recently. Do not mess it up with the community developed mcp. All links to full tutorial with all skills and stuff will be in the description guys absolutely for free. So here we need to install the add -on and activate it in blender. That's extremely simple we can even just drag and drop this guy and drop him to the blender like this. My extension is already installed but to verify that it installed you're gonna go to edit preferences.

Then in add-ons you're gonna see this mcp. So what you need to make sure that the server is running and the auto start is set to active. So whenever you will run the blender the mcp will be automatically start. Very convenient you do it once and forget. The second part is cloud code. So I believe guys you know how to install cloud code. I'm gonna skip that part but here is how you connect it to the blender and all the skills.

The cursor now is in empty folder and I'm gonna go and open a terminal. So in my terminal I'm gonna start it with dangerously skip permission guys because I don't want it to ask any stuff. I'm gonna give him a task and I'm gonna just wait until it's done. Disclaimer I do not recommend to do that but I will do that anyway. If you're not sure about the risk maybe start with cloud and approve actions manually. You'll get the same results but in second case you're gonna need to approve actions.

Now I'm gonna need to install blender mcp server. How it's done super easy. I prepared that md file with instruction that you can drag and drop into the folder with the cloud and it will be installing everything manually. Let's do that here you can see the instruction and this instruction also includes full ai which is optional but if you want it to be connected to the best apis such as image generation 3d generation vision model segmentation all that stuff on full ai but you have to replenish credits first.

I'm gonna do that for you guys but note this is optional. Okay so now we're gonna just drop this file to cloud and say install all stuff and we just need to wait. All right it's done and here we can see the output. So it actually suggests to install the add -on if you didn't we just did and then we do have a full ai key again it's optional. So you're gonna go in .inf that's a protected file and you're gonna place it here.

So that's how it will be accessed by cloud code. This is the best practice to keep your keys only in .inf file. Now after mcp are created you need to restart the cloud session once to get it to see blender. Let's do that i'm gonna kill previous terminal and open it again with the continue command that will actually get me back to my previous session. All right so we are back and we restarted. Now let's test the connection simple command to check if it can see my scene and yes we are connected and we can see some stuff.

So that was all set up if you add full ai key and it will also have the access to full ai apis. Now let's do some serious tests. The first test will be to do an animation of a character that will be split in parts. The first test will be to create animation. Animation of a character that will be an angel. So here is my character. I actually created it in triple then split into the parts in union and then texture it again in triple.

So you'll see the whole the process and the idea is simple. We're simulate how it drops on the floor so the parts are scattered naturally. So we need him to run simulation. The second part of the thing is to do animation but not just simple assembly animation but as it like a tornado animation. So are you ready to look at this guy cooking? I am. Also I prepared all the prompts. They are here. The complete prompt on the screen you can pause to take a look but I also created it with cloud code and reviewed it a little bit.

Okay so let's run and guys I already created one video about it. I was showing you how to do it on the cloud desktop and I also showed quite a lot of cool cases. There we were doing like keyboard animation but it was simple and I really wanted to try something more. Again I read all your comments and I decided like to give it like another try and make it more complicated. See like how far I can actually push it and all examples in this video will be way more difficult than there.

But if you use cloud desktop also take a look on that video. The first attempt was not successful because it actually not simulated the falling apart but the tornado animation was more or less. So I actually provided with the feedback and it started to do the next round but now the animation was sort of like fragmented also not good. So I gave it like one more feedback to see if it actually can fix it and here you see it actually start cooking again.

It was thinking for a long time but not spending that much tokens. So I don't know like it was actually evaluating quite a lot of stuff and then actually I got very interesting results. But I was not happy because it was like very slow tornado assembly and after just two more iteration one to fix the animation consistency and another to add one more extra spin in the very final we have this beautiful animation. And I think it's actually look very nice.

So there was a proper simulation from assembly so it like scattered parts like it falls apart and here you can see the results. This required total five iteration initial prompt and four revision and we got those results. To be honest I would not be able to do that manually. It's not the perfect animation still but the fact it was done so fast is actually fascinating. Now let me show you one more example and this is how I actually using my work.

This is about geometry nodes and by far I think this is like the most useful case so far. So geometry nodes is anything like that you can add to any object and stuff and I already showed like in my previous video how we scattered objects around the plane. So we saved enormous amount of time like positioning them and that was super fun and easy and I actually did it in one prompt. Now let's try to push it a little bit further.

I do have this scene and there is a water surface. It's not animated it's just a plane and here is the prompt that you will see on the screen. The idea is to animate it so it will be animated as a bubbles and I also want geometry nodes where I see parameters so I can control how big the bubbles is like how frequent and stuff. Let's get to it. So this one was easier at the beginning but when it created me the first version it was too big and too pixelated.

Here you can see the results. Okay and yeah I mean the bubbles was extremely big so it was not taking into account the world scale but it was impressive that it actually did like almost perfectly like in terms of like the task except the sizes. Okay so I actually provided the feedback that we need more resolution and bubbles should be like three times smaller and then it starts to do debugging. It was actually very fun to watch because it was enabling disabling the rendered hiding objects to see the water because it was covered by other objects.

It did not delete anything at all. Everything's safe, no problems so far guys and I tested like quite a lot in the recent days and nothing was deleted. And here you can see that the results starting to get better and better and better and finally with a few tips from me I was able to get quite interesting result. Here in the end I had to simplify shading so you can see it better because my scene was initially set for cycles render and yeah here we go.

So the animation run and we can control the parameters like the bubbles strength, the frequency and the spread speed and in this mode you can see it more obvious. I think that's very cool guys right. Look I did not touch anything, any geometry node. I don't even know like how it created that. Two prompts and I have this amazing result. So what about that island that our cloth cookie. So first of all how did I come up with that test.

Well actually I tried something like this before but recently I saw some guy build up the tool called image blast. That's basically a skill for cloth code again that uses world marble and full AI with Hoonion to create these amazing worlds. Look and I was like oh that's cool and I actually tried out. The setup is actually very similar and I actually forked it a little bit to do my test. So here's how it works. So you basically clone that image blast.

The link will be somewhere in the description and you need to provide it with two keys. Full AI API that will be used to create 3d objects inside the scene and world marble to create that scene. World labs is about to create 3d ghosts and splats and recently they added something cool that they can actually create a collider, the real mesh and that is actually what that image blast about. Anyway here's how it works. It uses quite a lot of credits right.

So each world generation might cost you up to five dollar right. So you don't need to test it out. I'm gonna test it and here's the result and it really works. It creates the world with world labs and also it opens a viewer in web and here's the thing it only works in web because there's a spark gs that runs this splats and to render it nicely in blender it's a bit more difficult. So just like to cut the whole part. Okay and after a few minutes it generated me the world and I was able to preview it in blender and yeah it looked nice but the coolest thing is again like when you create worlds with another objects in it and here's one more test.

I created this japan style and you can see the objects and the objects actually sort of interact with the environment. So behind the scene there is like a lighter but yeah in fact this is just a goose and splat. It can look impressive but in fact that's just as splats. So I decided build up my own version. I will try to do something like that in blender and here's the result. Well it's not what I expected really. I was experimenting a lot guys.

I was like I don't know I spent like maybe like two days trying to give it like all possible tools like some 3d for segmentation top view and stuff but it seems like cloud vision is terrible. It's so nice for coding and all these operations and stuff like writing code but with the vision so like I was trying to say like okay do the revision like do it again like move objects around and stuff. It's just I don't want. I can actually drop on the github what I coded so far.

If some of you can improve this it will be cool to see but the bad thing is that each world generation like this cost me quite a lot. So I was using triple and average I was paying eight bucks to create all the objects and place them around. I also connected patina so you can see the water material it's like a patina model but this is the best result I could get. I also tried with another location which is like ice location similar concept and it was disappointing like it could not even place the object sometimes like I mean some objects they were rotated 90 degree I don't know why.

I think triple generates it okay nicely but yeah so here's the final result. It's definitely not what I expect and I tried quite a lot. I gave him like so much credits like how much tokens I burned you better don't know and guys after I recorded the video actually I decided to give it a try with codecs. The good thing the same folder with same skills work with codec. I asked codecs to get acknowledged about all of this and make those skills codecs.

So basically skills.agents files right. Not a big thing it actually worked well so guys if you use codecs you can pretty much use the same setup right. You can use my same files to install the mcp here or blah blah blah. This pretty much works it will just take a bit more time but codecs will guide you through it and what about the results. To be honest I think it's slightly better and I liked how it actually went through the process.

Why I actually wanted to do that is that codecs has way better vision model and it was sort of like these modalities are inside the core model and what I can say that the scale of the ice it was generated with patina and the scale is it's a bit of mess here but it added like a mapping and it added the mapping and did the right scale which is great and objects positioned nicely. I actually set the logic and you can see in even small pieces boom boom boom they are on the ground.

Codecs actually did it slightly better well not significant as you can see this is still not usable just if you wondered like how codecs will do that here you see the result and I will drop this project on the git with all the skills that I used to create this. You can check it yourself it's quite expensive to experiment almost eight dollars this is for generate 3d models call image generation different passes and generate like patina pbr materials and stuff and if you have ideas how to improve that you can also take a look.

But we are not done testing so here's one more complex test we're gonna try to retarget animation. So I created this character in triple so we're gonna go to akarig and rig the character there. Akarig is free by the way and it does quite nice rig. You can preview the animation here and it looks not that bad actually I really start using akarig quite a lot but there is a downside right the akarig animations are mostly paid.

So I was thinking like what if I write my retargeter and I will just take mixamo animation and apply it for my akarig character. I mean easy money save right. So let's bring my character to blender and I also go to mixamo and here I found like this animation will of course twerk dance why not right and we're gonna download it as fbx and here you can see the mannequin from mixamo and our character. So it is actually challenging task these skeletons are not the same you can see there's difference in bones as character creator skeleton has more bones mixamo has less bones and also there is quite different rotation and this by far was most challenging task.

The prompt will be on the screen and also like some backstage story is that first I did a very wrong prompt I said like they do have the same rest pose. The rest pose is the original pose but I was mistaking because character creator exported t -pose as animation frame. So I spent like an hour and eventually Claude just said to me wait but the rest pose is not as you said and that was so funny because I just realized oh my days right he's right and I had to rewrite the prompt and start it over.

So here is the second attempt and it did quite a lot it was double checking himself he was doing like some poses retarget and stuff there was problems with the arms and stuff and it was trying to fix himself. I provided the feedback sometimes mainly there was the issue with the hands and actually one time blender just crashed but I don't know how but whenever I opened the blender it was restoring it the same moment I just told Claude that whatever you do is crashing blender here as you can see the prompt and it stopped doing it then he realized that he broke the model and reverted it to some working condition.

Okay so here's the final result so overall it actually managed to do 90% of it right so except the arms something with the shoulder there's a difference in the bone as I said like and the wrist right the wrists are completely something like the fingers are mirrored or something but all the rest like such as leg and body and head actually retargeted very nice right here you can see the skeletons they are quite different right so even like in arms I tried to fix it a lot it wasn't able to do it maybe we need Claude's 4.95 or mid test model he will definitely manage to do that but my idea was like if I made it work then the next thing I tell Claude is to create me an add-on so I can select two skeletons and retarget the animation because I already see like that it works but speaking of add-ons that can be actually very useful and I saw some guy on LinkedIn showing how he created an add-on for blender that actually tracks his head and create that parallax effect he also using the same setup here you can see what he created quite fun and drop the comment guys what you created or how you tested it I'm super excited about this now I doing a lot of tests it will be cool to learn more that was stuff like and subscribe and I see you in the next video