Neural network-generated custom portraits (aka "Why haven't I seen anyone suggest this before?")

[Swatcher]

Context: I've been using DALL-E mini since it's been popular this past week, and I noticed that the image resolutions(256x256) are surprisingly close to portrait resolution(210x330). I slopped together a quick test.

AI Example (generated by DALL-E mini):

Disregard the PoE1. That's just what I had installed at the moment. I didn't bother refining the portrait much. Only spent like 10 min on all this to prove how quick it can be. 

In the forums, custom portraits are something I see come up fairly often, so I'm wondering why no one here ever considered neural networks to do it for them. I haven't seen anyone ever suggest it, and I just searched for anyone too. Hell I made an account just to post this. I guess it is pretty recent tech though.

I've been watching different AI/neural network stuff for the past couple years. There's been some pretty impressive leaps in general from the last year alone (And just wait until next year and the year after!).

(To the mods--Wasn't sure if this'd get buried under the main portraits thread or whether the AI stuff is directly relevant to it, so new topic it is. I hope that's okay)

[Swatcher]

Maybe this is a bit of an off-kilter topic, but I hope people see it, haha. AI-generated custom RPG portraits in one minute. How ridiculous is that for real

Could also use it for tabletop characters too

 

Here's what I did in DALL-E mini. btw this isn't the only website or software out there.

1. I entered my own prompt "Black woman with white hair, Pillars of Eternity portrait" (don't need an account, eyy). Waited ~1 minute, and this was the very first result.

2. NOTE: You can probably come up with something decent-looking after doing enough runs, but I used GIMP to speed things along and for my personal adjustments. Chose the top-right creature for its outfit and the top-left square bc I preferred the head(look at the other choices)

3. The actual work. I went into GIMP and quickly just slapped on the left's face to the right one. Warped the stank mouth, bit of nose, slight brightness adjustments, and that's it, image done in like 10 minutes total. Probably could've been even faster.

Spoiler

Kinda reminds me of Jade Cargill now that I look at it

Can someone make something better than my rushed test job? Maybe. Probably. Better yet, make a quick AI portrait and commission an artist to use that as a base/reference to make something better and less crusty-looking. Artists could use the money anyway. Either way, it's easier than making something on your own from scratch, esp. for multiple characters.

Edited July 6, 2022 by Swatcher

[Swatcher]

One more thing: Note that this AI is mostly good for humanoids or elves. I tried some Aumaua, but it just gave me more humanish people. A different more specific prompt COULD work for Aumaua maybe, but I only tried "blue-skinned shark man portrait" and got THIS lmao. Jesus. 

Spoiler

And SOMEHOW, out of all these interspecies abominations, the middle right one. It's not a "sharkman". It's a shark trying to be a man. Even has the tail poking out a hole in its pants.

 

Spoiler

Hold on, breaking news! For your eyes only. My uncle that works at Obsidian is now confirming, Pillars of Eternity is not actually dead! Pillars 4 is in development!!1 Goku is in it too. First gameplay teaser, you didn't hear it from me

 

 

Edited July 6, 2022 by Swatcher

[thelee]

DALL-E mini is technologically impressive, but seems frustrating for a use case like this IMO. Though honestly I hadn't considered this use case before, so I'm excited at the possibility! I'm surprised you actually got something legible out of it (that you're even using as a profile pic).

 

What i'd *really* want to see is someone with DALL-E access generating these portraits. Would also help solve my frustration at the lack of halfling portraits in WOTR.

[Swatcher]

DALL-E mini is only just how I got the custom portraits idea, but honestly, using it really wasn't as frustrating as you think. The first grid I posted was from literally my first attempt after I got the idea. Mini's faces could use some work, but just editing a new face on top like I did speeds it up; there's a lot of faces to choose from. Pretty fast and headache free(if you're not picky), which I like.

And I'm also surprised! The quality, just how far has this tech come in the past year... I've been watching neural networks rapidly progress since Google DeepDream made the waves 7 years ago. You wanna see something ridiculous? Instead of DALL-E like you said, check out DALL-E 2, the new closed-beta. Just look at some of the AI-generated prompts on the r/dalle2 subreddit (link. also their top posts of all time). The output quality is insane.

'Midjourney' AI is also good (and quicker to get into the closed-beta, I hear). Also could try 'DiscoDiffusion', not closed but requires a gmail plus some set-up work; check the wiki on the same-name subreddit for guides.

 

https://huggingface.co/spaces/dalle-mini/dalle-mini (Here, if anyone wants to try DALL-E mini for portraits or otherwise. Just enter a prompt. Supposedly they're moving to craiyon.com, so the first url might not work for future people)

Edited July 3, 2022 by Swatcher

[Swatcher]

Decided to try a few more portraits since this is fun

Black-haired girl in leather armor

And the properly cropped versions

Spoiler

  

SOURCES: 

Spoiler

I only used the second one for the tired eye.

 

Edited July 6, 2022 by Swatcher

[Swatcher]

Woman in decorative plate w/ thick wavy black hair

And properly cropped:

Spoiler

 

EDIT: Decided to make a slightly smaller-hair version since I wasn't overly fond of big 80's hair in my crpg

And like I said, plenty of faces to choose from. For this one and only this one, I chose a face from a real existing portrait (well, someone else's edit of an existing portrait. Same difference). For the rest, I'm using AI results only, just to test DALL-E mini.

Some of the AI results can be perturbing.

Spoiler

Yeah.

 

SOURCES:

Spoiler

1st as a base. 2nd (actual portrait) for face. The goofy 3rd one was only for its hairline.

 

 

Edited July 6, 2022 by Swatcher

[Swatcher]

Pale elf man in plate armor

Armor seems a tiny bit too sharp compared to his softly lit features. Might try flattening it out. 

And cropped to size:

Spoiler

 

SOURCES:

Spoiler

1st one as a base. 2nd for the face.

 

 

EDIT: Slight armor lighting adjust

Spoiler

 

Edited July 5, 2022 by Swatcher

[Swatcher]

Old pale elf man

Seems a little scarred. Might've been the balrog that got his face.

Cropped:

Spoiler

 

SOURCES:

Spoiler

#1 for body. #2 for face. #3 for mouth only. #4 for ear only.

The process can be frankenstein-y, just a little bit.

 

Edited July 6, 2022 by Swatcher

[Swatcher]

A couple observations I made

1. Men seem to be harder to get good faces for. Women usually only take me a few prompts. Maybe it's not as weird as I think. Probably less photos of men than women online, so a weaker sample size to train the AI. Sucks

2. Close-up is good. Especially for faces. The further away & smaller anything gets, the worse the AI's detailing gets (this is also true for other AI models, in my experience. Not a DALL-E thing). Including terms like 'close-up' yielded better results. Try something similar.

3. Faces are the hardest for a lot of AI, so try to use a face from an existing RPG portrait. Of course, as of this year there's better AI models like DALL-E 2 that can do highly accurate faces, IF you have access... Anyway, if you're really gonna use DALL-E mini's faces, focus on getting a good face first, then get the body/armor later. Don't be picky. It's harder to find a face that fits a body than the other way around. Angles are the absolute worst.

And speaking of picky, the biggest weakness of this: You can't get too specific about what you want your results to look like, or it'll take forever. I just picked whatever looked good. Don't look for a tanned face at a 45° angle eyes cast down with curly black hair unless you wanna spend five years re-running prompts.

Edited July 3, 2022 by Swatcher

[Swatcher]

Even if there's better choices, DALL-E mini is pretty fun to use.

Sometimes the AI even decides to have a good day, like on my 1st try this knight dude is just straight up usable

 An actual face! Minus the snail ear, which is only minor. Just Mike Tyson a better ear from someone.

For some reason "knight" alone just produced a lot of decent results.

 

 

 

By the way, this guy

Spoiler

                

 

Edited July 6, 2022 by Swatcher

[Swatcher]

Trying anything that isn't "man", "woman", or non-human terms is iffy at best. Currently struggling with "Female Dwarf" and Orlan prompts.

Related: After fiddling with keywords, I think I discovered the secret keywords for Coastal Aumaua. The best description of Aumaua:

Spoiler

 Josh is in shambles

 

Edited July 5, 2022 by Swatcher

[Swatcher]

I tried doing just "Dark Souls" for portraits and got some unexpectedly good results. This was the 1st attempt

Half-considering fixing up one of these into a portrait. Man if I had the patience, a lot of these would make a good base for drawing a portrait from scratch.

Two more

Spoiler

Surprising that Pillars portrait x Dark Souls actually worked. I wonder what other video games would make good mashups?

Even though DALL-E mini isn't the best quality this is too much fun. I should probably go set up DiscoDiffusion or anything with Diffusion Models, but I just can't help but love this one's sheer variety and speed.

Still though, resolution is meh. If anyone here somehow has access to one of the better AI models, I'd like to see what they can make with it.

Edited July 10, 2022 by Swatcher

[Swatcher]

UPDATE: I have access to DALLE-2! Man, this thing is cool. However, my credits expire the 17th 18th of every month. They gave me a bunch of starter credits that'll expire by Sep 17 18. I haven't really toyed around with it much, so I still have 50 starter credits (50 'uses'). Every month after, I'll only get 15 free credits (the prev. month's credits don't roll over), and extra costs $$$. If anyone has any ideas, feel free to shoot them now!

I can only try so many times by myself. If only there were more DALLE2 people here to try this stuff out and do requests (or one person with a big wallet and a big heart)

So, anyone have any ideas for prompts? Portraits?

Edited September 23, 2022 by Swatcher

[thelee]

how good is the corpus? if you ask for "hearth orlan" will it actually generate some decent hearth orlan pics?

[Swatcher]

I'd be amazed if it did, because DALL-E Mini/Craiyon couldn't at all, although DALL-E 2 was trained on different data sets. Still tho
But, I have to do some stuff irl. I'll get back to this soon

 

edit: In the meanwhile, check this out https://dallery.gallery/the-dalle-2-prompt-book/, either for funni pics or to see how a good prompt for DALL-E 2 is made.

Edited September 23, 2022 by Swatcher

[Boeroer]

12 hours ago, thelee said:

if you ask for "hearth orlan" will it actually generate some decent hearth orlan pics?

"orang-utan with ears of a lynx" should do the trick.  

[Torm51]

Hey this is great work!

[Swatcher]

Thanks! Making these is pretty entertaining.

Okay we barely even have a week left until my credits reset. Probably should've started earlier. Let's hurry and get some ideas in! I'll start with orlan and halfing, since those were the first two ideas from thelee.

 

This is probably going to take some experimentation. And unlike DALL-E Mini, we can't keep retrying freely (can't use Mini to practice this either, because they're two different AIs with different training data = different results)

Edited September 23, 2022 by Swatcher

[Swatcher]

THE FIRST BATCH (7 done, 43 tries left)

Alright, this batch was total trash. They LOOK good, but they're the wrong things/wrong style. I need to change the wording/prompts. Take a look:

Warning: Long image (had to stitch it together as one long image bc of the inconvenient 1 MB limit. Anyone know a way around that?)

EDIT: This website keeps shrinking it down so it's unreadable. I guess I'll just have to screencap & upload it one at a time, post by post, all meticulously. See you guys in a bit

Edited September 10, 2022 by Swatcher

[Swatcher]

In chronological order of how I made them:

1 Black woman with white hair, Pillars of Eternity portrait

Tried to see if I could recreate my very first picture in one go. Not bad but the styyyyyle . That sh!t ain't Pillars.

Edited September 17, 2022 by Swatcher

[Swatcher]

Trash batch

2 portrait of an orlan from Pillars of Eternity

Doesn't seem to recognize orlan

Edited September 9, 2022 by Swatcher

[Swatcher]

btw, note that the resolutions are higher than they look. Here's a full image from the first one

1024x1024 .png files (converted to a jpeg tho to fit here)

Edited September 23, 2022 by Swatcher

[Swatcher]

Trash batch

3 Pillars of Eternity (lmao)

Doesn't seem to even recognize Pillars of Eternity as a game.

DALL-E Mini/Craiyon does, tho.

Repeating what I said, they learned from different training datasets. One AI was fed Pillars of Eternity images, and one wasn't. Big neural networks basically take in a giant feeding-trough worth of data--like in the order of millions--and they're picking what to feed it, which varies.

That's why the AIs have different results.

Edited September 14, 2022 by Swatcher

[Swatcher]

Trash batch

4 a halfing in the style of a CRPG portrait

Left, I-... Is that Vivec? Is he a coomer???

Edited September 23, 2022 by Swatcher