Jump to content

Neural network-generated custom portraits (aka "Why haven't I seen anyone suggest this before?")


Recommended Posts

Context: I've been using DALL-E mini since it's been popular this past week, and I noticed that the image resolutions(256x256) are surprisingly close to portrait resolution(210x330). I slopped together a quick test.

AI Example (generated by DALL-E mini):

portraitTest.thumb.jpg.27d06cb5752862a0e00804d46c60ae8e.jpgportraitTest2.jpg.6015e91bcd90aa604520841a203a6bc0.jpg

Disregard the PoE1. That's just what I had installed at the moment. I didn't bother refining the portrait much. Only spent like 10 min on all this to prove how quick it can be. 

In the forums, custom portraits are something I see come up fairly often, so I'm wondering why no one here ever considered neural networks to do it for them. I haven't seen anyone ever suggest it, and I just searched for anyone too. Hell I made an account just to post this. I guess it is pretty recent tech though.

I've been watching different AI/neural network stuff for the past couple years. There's been some pretty impressive leaps in general from the last year alone (And just wait until next year and the year after!).

(To the mods--Wasn't sure if this'd get buried under the main portraits thread or whether the AI stuff is directly relevant to it, so new topic it is. I hope that's okay)

  • Like 3
Link to comment
Share on other sites

Maybe this is a bit of an off-kilter topic, but I hope people see it, haha. AI-generated custom RPG portraits in one minute. How ridiculous is that for real

Could also use it for tabletop characters too

 

Here's what I did in DALL-E mini. btw this isn't the only website or software out there.

dallemini_2022-6-27_19-29-49.thumb.jpg.91f55e0f1697001cb6979a2af94ee742.jpg

1. I entered my own prompt "Black woman with white hair, Pillars of Eternity portrait" (don't need an account, eyy). Waited ~1 minute, and this was the very first result.

2. NOTE: You can probably come up with something decent-looking after doing enough runs, but I used GIMP to speed things along and for my personal adjustments. Chose the top-right creature for its outfit and the top-left square bc I preferred the head(look at the other choices)

3. The actual work. I went into GIMP and quickly just slapped on the left's face to the right one. Warped the stank mouth, bit of nose, slight brightness adjustments, and that's it, image done in like 10 minutes total. Probably could've been even faster.

Spoiler

1277781132_female_human_aiFullVersion.png.f50691f9092e7f4cb847a6466a10059c.png

Kinda reminds me of Jade Cargill now that I look at it

Can someone make something better than my rushed test job? Maybe. Probably. Better yet, make a quick AI portrait and commission an artist to use that as a base/reference to make something better and less crusty-looking. Artists could use the money anyway. Either way, it's easier than making something on your own from scratch, esp. for multiple characters.

Edited by Swatcher
  • Like 3
Link to comment
Share on other sites

One more thing: Note that this AI is mostly good for humanoids or elves. I tried some Aumaua, but it just gave me more humanish people. A different more specific prompt COULD work for Aumaua maybe, but I only tried "blue-skinned shark man portrait" and got THIS lmao. Jesus. 

Spoiler

dallemini_2022-6-28_19-45-57.thumb.jpg.f5c099a770e7874bef2feda8c2c91e18.jpg

And SOMEHOW, out of all these interspecies abominations, the middle right one. It's not a "sharkman". It's a shark trying to be a man. Even has the tail poking out a hole in its pants.

 

Spoiler

Hold on, breaking news! For your eyes only. My uncle that works at Obsidian is now confirming, Pillars of Eternity is not actually dead! Pillars 4 is in development!!1 Goku is in it too. First gameplay teaser, you didn't hear it from me

dallemini_2022-6-28_20-4-58.thumb.jpg.904eb13af4b0b5621b3dfd3033732634.jpg

 

 

Edited by Swatcher
  • Haha 1
Link to comment
Share on other sites

DALL-E mini is technologically impressive, but seems frustrating for a use case like this IMO. Though honestly I hadn't considered this use case before, so I'm excited at the possibility! I'm surprised you actually got something legible out of it (that you're even using as a profile pic).

 

What i'd *really* want to see is someone with DALL-E access generating these portraits. Would also help solve my frustration at the lack of halfling portraits in WOTR.

Link to comment
Share on other sites

DALL-E mini is only just how I got the custom portraits idea, but honestly, using it really wasn't as frustrating as you think. The first grid I posted was from literally my first attempt after I got the idea. Mini's faces could use some work, but just editing a new face on top like I did speeds it up; there's a lot of faces to choose from. Pretty fast and headache free(if you're not picky), which I like.

And I'm also surprised! The quality, just how far has this tech come in the past year... I've been watching neural networks rapidly progress since Google DeepDream made the waves 7 years ago. You wanna see something ridiculous? Instead of DALL-E like you said, check out DALL-E 2, the new closed-beta. Just look at some of the AI-generated prompts on the r/dalle2 subreddit (link. also their top posts of all time). The output quality is insane.

'Midjourney' AI is also good (and quicker to get into the closed-beta, I hear). Also could try 'DiscoDiffusion', not closed but requires a gmail plus some set-up work; check the wiki on the same-name subreddit for guides.

 

https://huggingface.co/spaces/dalle-mini/dalle-mini (Here, if anyone wants to try DALL-E mini for portraits or otherwise. Just enter a prompt. Supposedly they're moving to craiyon.com, so the first url might not work for future people)

Edited by Swatcher
  • Like 2
Link to comment
Share on other sites

Decided to try a few more portraits since this is fun

Black-haired girl in leather armor

Untitled.png.216779367cffb1c5aa26408126d8a339.png

And the properly cropped versions

Spoiler

 Untitled2.png.2b1fca7fda978f291a6252aff4ffe7bb.png Untitled3.png.ec8e09edc97452881ae651314a68fdf0.png

SOURCES: 

Spoiler

Untitled.png.28fa4e1fd3b3a6052d16ab7a675a0b9a.pngI only used the second one for the tired eye.

 

Edited by Swatcher
  • Like 1
Link to comment
Share on other sites

Woman in decorative plate w/ thick wavy black hair

Untitled.png.a457e598c414e3aeab02c32b710fd182.png

And properly cropped:

Spoiler

Untitled2.png.0a551e7611955e52348cfd5618923e8e.png Untitled3.png.29bece1da9b10f8f1d46fb21eb43fb89.png

EDIT: Decided to make a slightly smaller-hair version since I wasn't overly fond of big 80's hair in my crpg

1969960549_smolhair.png.071d3f623cd1c108da3f3461dd1dca30.png

And like I said, plenty of faces to choose from. For this one and only this one, I chose a face from a real existing portrait (well, someone else's edit of an existing portrait. Same difference). For the rest, I'm using AI results only, just to test DALL-E mini.

Some of the AI results can be perturbing.

Spoiler

Untitled.png.be960d5e12980ac632524652b87e4021.pngYeah.

 

SOURCES:

Spoiler

Untitled.png.5572b2bb120607e989e90e85740965d5.png

1st as a base. 2nd (actual portrait) for face. The goofy 3rd one was only for its hairline.

 

 

Edited by Swatcher
  • Like 3
Link to comment
Share on other sites

Pale elf man in plate armor

Untitled.png.8c83c8ca822d2a520c461d98c9d82b6b.pngArmor seems a tiny bit too sharp compared to his softly lit features. Might try flattening it out. 

And cropped to size:

Spoiler

Untitled2.png.72dc25a21a5d999826dad84fb98aaed1.png Untitled3.png.2d3d9ef86cab882667a162019b54dbf5.png

SOURCES:

Spoiler

Untitled.png.4b99197b63caa57c12da6e064d7a1576.png1st one as a base. 2nd for the face.

 

 

EDIT: Slight armor lighting adjust

Spoiler

2030338433_fixedarmor.png.ec58ca5aad43b48683aea2958db7e2f5.png

 

Edited by Swatcher
  • Like 1
Link to comment
Share on other sites

Old pale elf man

Untitled.png.efbb393f039431d9d236957946b73ae6.pngSeems a little scarred. Might've been the balrog that got his face.

Cropped:

Spoiler

Untitled2.png.17e4a82ee792bfdcfdf5dcf0cebfa855.png Untitled3.png.483b4ea8d1fdcf2806cc618aad2d58ee.png

SOURCES:

Spoiler

Untitled.thumb.png.199f021a3793d09e1d032eaa9400d72c.png

#1 for body. #2 for face. #3 for mouth only. #4 for ear only.

The process can be frankenstein-y, just a little bit.

 

Edited by Swatcher
Link to comment
Share on other sites

A couple observations I made

1. Men seem to be harder to get good faces for. Women usually only take me a few prompts. Maybe it's not as weird as I think. Probably less photos of men than women online, so a weaker sample size to train the AI. Sucks

2. Close-up is good. Especially for faces. The further away & smaller anything gets, the worse the AI's detailing gets (this is also true for other AI models, in my experience. Not a DALL-E thing). Including terms like 'close-up' yielded better results. Try something similar.

3. Faces are the hardest for a lot of AI, so try to use a face from an existing RPG portrait. Of course, as of this year there's better AI models like DALL-E 2 that can do highly accurate faces, IF you have access... Anyway, if you're really gonna use DALL-E mini's faces, focus on getting a good face first, then get the body/armor later. Don't be picky. It's harder to find a face that fits a body than the other way around. Angles are the absolute worst.

And speaking of picky, the biggest weakness of this: You can't get too specific about what you want your results to look like, or it'll take forever. I just picked whatever looked good. Don't look for a tanned face at a 45° angle eyes cast down with curly black hair unless you wanna spend five years re-running prompts.

Edited by Swatcher
  • Thanks 1
Link to comment
Share on other sites

Even if there's better choices, DALL-E mini is pretty fun to use.

Sometimes the AI even decides to have a good day, like on my 1st try this knight dude is just straight up usable

Untitled.png.ac0a1d7253e7331ff96f167a23b27135.png An actual face! Minus the snail ear, which is only minor. Just Mike Tyson a better ear from someone.

For some reason "knight" alone just produced a lot of decent results.

Untitled.thumb.jpg.ad7309802cebc78bb4631087bba444e1.jpg

 

 

 

By the way, this guy

Spoiler

Untitled.thumb.jpg.0638266b630576f972cf00cec8ba6dd3.jpg

                Untitled.jpg.5197b2074f4bcc96612bdacf46133ac5.jpg

 

Edited by Swatcher
  • Like 1
Link to comment
Share on other sites

Trying anything that isn't "man", "woman", or non-human terms is iffy at best. Currently struggling with "Female Dwarf" and Orlan prompts.

Related: After fiddling with keywords, I think I discovered the secret keywords for Coastal Aumaua. The best description of Aumaua:

Spoiler

Untitled.thumb.jpg.a2c52f7c7b8ce2747c0a361b695eaa30.jpg Josh is in shambles

 

Edited by Swatcher
  • Thanks 1
Link to comment
Share on other sites

I tried doing just "Dark Souls" for portraits and got some unexpectedly good results. This was the 1st attempt

Untitled.thumb.jpg.1dff97e6e34ceec2463ced4e1ab909c4.jpg

Half-considering fixing up one of these into a portrait. Man if I had the patience, a lot of these would make a good base for drawing a portrait from scratch.

Two more

Spoiler

Untitled2.thumb.jpg.5066b0f049d4d21e2909b4cb86ef41ed.jpgUntitled3.thumb.jpg.2411a978d78b0e17a05d6a1e46df1a02.jpg

Surprising that Pillars portrait x Dark Souls actually worked. I wonder what other video games would make good mashups?

Even though DALL-E mini isn't the best quality this is too much fun. I should probably go set up DiscoDiffusion or anything with Diffusion Models, but I just can't help but love this one's sheer variety and speed.

Still though, resolution is meh. If anyone here somehow has access to one of the better AI models, I'd like to see what they can make with it.

Edited by Swatcher
  • Like 1
Link to comment
Share on other sites

  • 2 months later...

UPDATE: I have access to DALLE-2! Man, this thing is cool. However, my credits expire the 17th 18th of every month. They gave me a bunch of starter credits that'll expire by Sep 17 18. I haven't really toyed around with it much, so I still have 50 starter credits (50 'uses'). Every month after, I'll only get 15 free credits (the prev. month's credits don't roll over), and extra costs $$$. If anyone has any ideas, feel free to shoot them now!

I can only try so many times by myself. If only there were more DALLE2 people here to try this stuff out and do requests (or one person with a big wallet and a big heart)

So, anyone have any ideas for prompts? Portraits?

Edited by Swatcher
  • Like 1
Link to comment
Share on other sites

I'd be amazed if it did, because DALL-E Mini/Craiyon couldn't at all, although DALL-E 2 was trained on different data sets. Still tho
But, I have to do some stuff irl. I'll get back to this soon

 

edit: In the meanwhile, check this out https://dallery.gallery/the-dalle-2-prompt-book/, either for funni pics or to see how a good prompt for DALL-E 2 is made.

Edited by Swatcher
Link to comment
Share on other sites

Thanks! Making these is pretty entertaining.

Okay we barely even have a week left until my credits reset. Probably should've started earlier. Let's hurry and get some ideas in! I'll start with orlan and halfing, since those were the first two ideas from thelee.

 

This is probably going to take some experimentation. And unlike DALL-E Mini, we can't keep retrying freely (can't use Mini to practice this either, because they're two different AIs with different training data = different results)

Edited by Swatcher
Link to comment
Share on other sites

THE FIRST BATCH (7 done, 43 tries left)

Alright, this batch was total trash. They LOOK good, but they're the wrong things/wrong style. I need to change the wording/prompts. Take a look:

Warning: Long image (had to stitch it together as one long image bc of the inconvenient 1 MB limit. Anyone know a way around that?)

8777613_1DALLE2FIRSTBATCH(7).thumb.jpg.c2b2508e87aabf97b6170c95e29c0df2.jpg

EDIT: This website keeps shrinking it down so it's unreadable. I guess I'll just have to screencap & upload it one at a time, post by post, all meticulously. See you guys in a bit

Edited by Swatcher
Link to comment
Share on other sites

In chronological order of how I made them:

1 Black woman with white hair, Pillars of Eternity portrait

1484398541_1BlackwomanwithwhitehairPillarsofEternityportrait.thumb.jpg.46f23b5d8136dac3ce4d942bd9b016ed.jpg

Tried to see if I could recreate my very first picture in one go. Not bad but the styyyyyle 😕. That sh!t ain't Pillars.

Edited by Swatcher
  • Like 1
Link to comment
Share on other sites

Trash batch

3 Pillars of Eternity (lmao)

673573218_3PillarsofEternity.thumb.jpg.cdda5c16c7f11a5ea43e7f7a45c5c826.jpg

Doesn't seem to even recognize Pillars of Eternity as a game.

DALL-E Mini/Craiyon does, tho.

craiyon_012311_pillars_of_eternity.thumb.jpg.ea336bd151c423739762b95d1a934948.jpg

Repeating what I said, they learned from different training datasets. One AI was fed Pillars of Eternity images, and one wasn't. Big neural networks basically take in a giant feeding-trough worth of data--like in the order of millions--and they're picking what to feed it, which varies.

That's why the AIs have different results.

Edited by Swatcher
  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...