1. Winter and seasonal lack of motivation has made things slow down quite a bit.
2. Most of what I have done is experimenting with new things, including:
Learning new ways to texture hard surface models with fake beveling. This makes objects look more pleasing and is pretty fast to make.
Text based approach to transform my game project into a more easy-to-develop mobile game.
This has led me to collaborate with a programmer (yei!) and I will give more details later.
(For once I really believe this project could be finished some day!)
Learning Fractal art.
This is really new and very powerful. As my ideas are often dark and abstract, being able to use fractal art to blend with 3d opens up new possibilities for visual FX.
Learning Visemes.
This is the new fancy thing for Vrchat avatars. Being able to speak into a microphone and get the characters lips to move accordingly is really cool.
It is still very early to show anything, but as soon as I master one or more of these new skills and progresses, pictures are coming :)
I have currenty a handful of scenes partially ready for my game. Also a handful of partially finished characters, but I have taken a break from characters to rest and get some focus on terrain for an outdoor scene.
So this is not obviously lit or detailed yet, but it is promising I think.
The terrain is created in L3DT and a heightmap is exported directly to Unity and then I used EasyRoads3d (free version) to make a road that passes through. The road is not really detailed as it will not actually be used but just be there.
Now I can make trees and rock to see how it will goes!
BTW. I have a new Instagram account that has more frequent smaller updates. apdanstudio
For the last few weeks (actually) I have tried again to improve my workflow for fast facial animation, and now I have reached a quality I believe is sufficient for my project, especially within time limits.
These few weeks I have remade the facial animation structure and bones dozens of times from skratch and tried many different methods for animating, yet only now do I find a quality I like.
My current project has swayed between various rendering styles throughout, ubt because "reasons", I now want to bring it into full 3d and not any prerendered styles.
The only limitation I have is that I am not a programmer, and thus any animation must be simple and bone driven. I do not care much for in-engine animation mixing, lip-sync addons and other cheap solutions. I want to bring the whole of an actors performance into an animation pipeline, bake it out to pure bone driven animation, and play in-engine on demand. Let's see how it goes.
First Testing
It started with an idea to use blender to capture markers from video in 2d and automatically translate to 3d. Everything on Youtube has shown to be awful and useless.
The first step was to analyze how each marker (a point that represent a focal point of deformation in the face) moved in that particular area. This maker was to be linked to the tip of a bone that had its root deeper in the skull and the rotation when the tip followed the marker should be good enough to deform the face properly. Wrong!
Not only is this very lazy, but it doesn't consider the translation from the actor to the character geometry very well, and tons of calibration was needed for each shot of video.
The end result made the face either stiff or with an unnatural rubbery feel.
And I found a real problem. The mouth.
The corners of the mouth and the thickness of the lips vary greatly when a person speaks. This translated poorly.
The tracker rig
A simple facial rig was created and duplicated for 2d tracking.
The tracker rig had bones that directly tracked the markers from video. To this rig was parented "balls" of 3d.
The tracker duplicate did not follow the markers from video but followd the balls. This meant that I could move the balls to calibrate the tracked animation to the chracter and compensate for bad symmetry and proportions.
The character rig followed the rotation from the tracker duplicate rig, and could lower or heighten the influence.
I kept this tech for animating eye brows, lower eye skin, upper cheeks and nose wing (area).
Second Testing
When searching for a way to deform the mouth I (again) looked up the FACS system for learning more about facial muscles. This led to the fact that this system relies on special analyzing software that recognizes facial poses on video and makes a graph for the use of muscle groups that is used by another special solving software. Expensive to say the least. Also it needed facial scans of combined FACS poses for reference so that the character could be deformed properly with "blend shapes" also known as Shape Keys in Blender. Making a dozen facial scans of an actor, cleaning them up and then making blend shapes on the character is an exptremely time consuming job only AAA developers can afford. And yes, the final result is of high quality and realism. I simply could no go there.
FACS tip
If you consider going the FACS route, there is a program called "Agisoft Stereo Scan" which lets you skip renting a Photo Dome to scan your actors.
Simply take two parallel pictures of a pose and Agisoft will make a simple 3d mesh with texture in a few clicks. It isn't highly detailed down to the pores, but it gives a general idea of the shape of the face in that pose. which can be useful I guess. It is also a great way to make character faces for background characters only seen from a distance, without any modeling or texturing at all.
Final Testing
I realised FACS was a little to high on the shelf for me. And besides I only needed Mouth deformation. The rest I could track from video.
I searched the web for a Blender addon or solution that could blend "Pose keys" freely. That did not exist, even though many point to an addon that lets you mix two poses. I need any number of mixes.
So I created the Piano Rig.
This is a rig (armature) that has simple rows of bones where each bone drives contraints on the character rig to deform.
Learning from FACS, I made a list of poses needed for all kinds of mouth deformations.
In a non-linear fashion the bones are not limited to single use. All kinds of poses can be mixed.
Mini discovery
The sound of the norwegian letter "Æ" heard in the american english word "At" looks like a letter made by A and E. To make this deformer, the Piano bone to drive the facial pose "Æ" also influences the pose bone for letter A and E.
This created a perfect Æ apperance. This didn't work for all poses, but was very helpful to limit the number of Piano bones needed. The phoneme list is as follows:
A
E
I
OU
Y
Æ/AE
Ø/OE
Å/AA
MBP
FV
KGDTS
Sticky Lips
Pucker tight
Pucker Kiss
Lip press
Jaw drop
Pout
Smile
Anger
(there might be more poses needed in the future but this is a good start)
So each bone in the character face that makes a smile, has a constraint listening to the smile bone for a 0-90 degreee influence, 90 being the strongest expression from neutral.
This has the following advantages:
1.- The character does not need any facial key frame animation. Everything is controlled by the piano rig.
2.- If an animation is made and a pose seems off, the pose constraints can be adjusted on the character rig, and ALL those poses repeated in the animation will be changed. Totally non destructive.
3.- ALL mouth poses can be mixed with various strength. With the rotation of a single bone, the character can turn from normal speaking to shouting or smiling gradually through the animation while expressing words. Although combining A with MBP is rather useless anyways.
In addition to the piano rig, the character Jaw and muscles around the eyes can be tracked directly from video. Also head movement. The eyes are rigged with smooth following eye lids, but the look direction needs to be keyframed by hand. Still simpler that keyframing a full face.
This is how the timeline scrubbing looks. The bones on the left are animated and the armature on the right follows.
Complexity
After all that hard work, setting up a list or tutorial on how to set the constraints on each mouth bone is a job in itself, and wasted development time. One facial bones has up to 11 constraints for rotation and translation from various other bone influences. It takes too long to explain how each of these work I want to make a game!
Final of testing. Much to improve still, but this is a proof of concept at least.
More or less done with part #1 of the current game project. It plays well, and I'm tired of looking at it.
It will be damn refreshing to start building new places. Keep in mind that I'm creating all 3d models, and textures by myself, not buying any premade stuff. I do this for the following reasons:
Integrity
Practice
Unique feel
I think this is very important nowadays for indie developers.
Snippet from some of the work.
Brushed up an old 3d model to more correct anatomy. This is a screen grab, showing the raw model without any materials or bump mapping.
Also I have written a full Striigatorium story all by myself (which is a first), but I'm not sure what to do with it yet. As it is not suitable for a game it would be cool to make into some sort of visual narrative, but style, quality etc. is still in the idea state.
The good thing about Youtube is that you no longer have to buy games to see if they are good. It is enough to watch snippets of gameplay and read comments.
And that is why Mass Effect: Andromeda will pass by. Why any developer can release such an awful package of junk is beyond me, especially when it comes from AAA developers.
Anyways, one thing is for sure: I will not attempt to "ulgify" the characters in my project that is based on real models to satisfy people with personal issues.
I am not saying that I am a better artist, by far, but I do not create through the "profit lens", and I'd rather put some passion into it and take my time.
This was just an example I found. But in reality it is way worse.
Really?
Disclaimer: Images from ME:A was found on google and do not belong to me, and credits go to the respected owners.
Slightly nervous character. The character Idle loops are only 50 frames so I have to make the best of it to combine expression in such a short time without getting that "wavy" loop feeling.
An last here is a crop of the latest scene I am working on to get a break from the first scene. There are still lots of missing objects, "grit" and filtering left.
Things have been moving slowly lately due to winter time and a wall hit on graphics.
The original idea was to have as much animation that the main project would look and behave like a real drama, but the quality of the animation was simply not on point within the capacity for production, and a good enough voice cast would be very expensive.
Besides I didn't like the idea of only producing for PC. This is a simple game in terms of mechanics, and that would appeal to a more casual audience too, and they often use tablets. So I decided to move over to a tablet friendly setup.
Developing for tablet reduces the need for heavy animation which needs hair physics, facial animation, and realistic movement to become believable enough. It also reduces the interface complexity with various cursors and functions down to a finger tap solution. This removed a heavily load from my time and I can now focus more on making pretty still pictures for a so called Visual Novel. I have seen plenty of these games and 99% of them are of asian origin with anime/manga style, and I think this is different by far.
I am now devloping for PC and Android, and possible other platforms as I am waiting for the release of Visionaire Studio 5 which will be more stable for building for various platforms. Visionaire has a very supportive forum and I highly recommend it for adventuregame developers.
So below is an early alpha stage of a scene, half size, badly rendered and not filtered, and there will be minor animated elements in the final scene. But it is faster to make and easy to implement into the engine.
For Animations where I do not need a full 3d scene or lack a sufficient photo, I simply build an image by expanding a part of a photo.
If I need to I can also add 3d objects into the final image just as easy as I add the animated character with shadows casting on the floor as they are calculated onto an invisible simplified geometry of the scene. Post filtering is done in Blender as usual.
Happy new year! May 2017 be better than the 2016 global disaster of shame and death!
The last 6 months or so I have worked on what I just call "The Hall Scene", on my project.
It is the first scene in the game, and workflows for the way I want to make the game had to be worked out in this scene, the reason I've spent much time on it.
However in the beginning of December I hit a road block and had to start over, and this is how it goes:
When the game was written the story required a game that was not made with a "toon" like style as seen in so many other games in this genre. The story had to be cinematic and portray facial closeups and a somber atmosphere. So I thought,; why not just make it like an interactive movie? I imagined handheld cameras, weird filming angles and tons of cool effects, and it seemed to work in the Hall scene. But it required lots of tweaking to get the render times down, and since the scene was so simple the real slap in the face came when I moved deeper into more complex scenes. The render times went through the roof (15 hours and more)! It simply was not feasable anymore.
So after a lot of thinking I came up with a more classic idea: Have more static cameras, and composite the 3d characters into the scene instead of rendering them with the scene.
The difference is not better but way faster, both when it comes to creating the scenes with more matte paintging and photos, and rendering it out. Also it is now possible to blend with real actors and video elements.
Keep in mind that the blurryness and abberation in the last image is how it is meant to be.
This "original" scene is fully 3d, and very dark and dramatic, but since the camera moved in the animation, the background had to be rendered for each frame and that took a long time.
Render time: 3 hours
This "new" scene is much less dramatic, but also serve its purpose. This is made with a static camera, and is actually a composition of photo (which makes it faster to create) and 3d elements. It is not perfect, but that's not super imortant in the end.
So, after tons of test rendering I stop where I am happy with the result. Perfectionism is compromised.
What have I learned?
In the beginning I used one phonetic pose for each letter in the script. This looked very hard and exaggerated after rendering compared to lipsync in modern games. So I thought I must tone down each pose to a smaller expression. Not working. It still looked fast and strange. At last I simply removed poses that was not clearly heard in the sound file:
In the words "Are you" the old was:
A R E Y O U
to
A R Y O U
to finally
A Y U
This is a very rough render with minimal samples.
The animation has only 15 fps and is not super smooth, but I am still unsure if I really want to go higher. But I have found rendering cheats good enough to increase the game resolution to full screen!
And, yeah, I have a voice actor now! Her name may be revealed later :)
