Thanks for this. This idea is most interesting. I mostly work in 3D but the same thing could apply. Although it’s debatable whether the effort of training a LORA outweighs the effort of further 3D work.
Some context:
I don’t know whether this is true of other people but it is important for me to design the key plot characters immediately after developing the initial game concept. And I need to visualise them. This is because those characters, their look and personalities are essential for me to write the story and especially the lines of dialogue.
When i design a character I design how they look as well as their personality. The look is very important. I consider their ethnicity, their face, their hair, clothes and accessories. Small details are very important.
This is why i like to design them in 3D. If you’re a writer, consider a few of your lead characters and ask yourself, “What color are her eyes?” or “What do his shoes look like?” Irrelevant maybe? Maybe not. But you might be surprised what you’d never considered, when you thought you knew everything about a character.
The idea would be to make renders in key poses as source training data for a LORA.
Let’s take an example:
I have a “superhero” game concept that stars two characters - “Max” and his friend “Zu”.
précis:
The Italian Maxman is not your regular superhero. He’s not super strong. He can’t dodge bullets or leap tall buildings. He’s a bit overweight, but nevertheless, he’s not one bit less the all action superhero. Saving the world is a full time job. And even though he’s not paid, he always sees the job through.
Let’s look at some character design:
“Max” needs to be an atypical super-hero. He’s a bit plump but also cool. He has a great sense of knowing what to do and can get things done. He’s not a fake super-hero but he has no special abilities other than an uncanny ability to “solve problems”.
The “Zu” character needed to be a bit larger than life, be sharp, a bit funky and a wild sense of humour. She’s also the “in game help” if you get stuck. She’s also super-clever and resourceful.
Once these characters are designed, i could easily generate some “key poses” so that an AI could train on them. They’d consistently be in their regular outfits so that would be ideal for the training. Only the poses would change. There would have to be poses from the back and ones covering the full body including shoes etc. Backgrounds could be plain or absent, which would help avoid distraction.
For example, consider Zu’s hair style. It’s asymmetric and you have to see it from different angles to get training data.
eg.
Anyway, it’s an interesting idea. And additionally, there are no ethical issues training from your own creations.