Mystic hut - a mysterious moonlit diorama

Want to know how to create a spooky stylized diorama in Unreal Engine? Then this breakdown by Nikita Vorobiov is perfect for you! Learn how Nikita utilises paintovers, silhouettes, lighting, colour and much more to craft a beautiful, eerie scene.

Intro

Hello guys! My name is Nikita Vorobiov and I live in Ukraine. I’ve been in 3D for a long time but always working in different fields - VFX, generalist, character, lighting, colour scripts, few crazy startups and some management - can’t even keep track of all of that.

Lately I’ve been lucky to find what I really like and where I can apply most of my skills I picked up along the way and it turns out environment art is awesome. This is where painting, lighting, composition, modelling and a lot of other things come together in proportions that suit my workflow the best: tasks change often enough so I don’t get bored, routine tasks alternate with creation, and the whole process is very flexible and iterative.

Composition

The important thing about composition is practice. The more you do without thinking, the easier it is. I’ve done quite a few compositional sketches including abstract ones in 2D and 3D practising various types of contrasts, form relationship, diagonals, focal points etc. It still sounds crazy to me, but moving boxes around and drawing shapes helps a lot! Here’s are few examples:

I wanted to do a small piece of location to be sure I could handle something bigger and practice what I already know. I chose to focus mostly on feeling and first impression because in my opinion no amount of materials will fix boring ideas and layouts.

For me, composition is mostly simple visual messages that communicate a mood or a feeling. Those messages should be very simple and obvious when pointed out, but at the same time subtle enough not to be recognised without some effort. Overthinking your composition may be harmful and very exhausting, so on the blockout stage I go with the flow and see where it takes me. Ability to know when you stumble upon something that works with your idea is very important.

Breakdown of my original sketch from Maya. Triangular, a bit unstable in terms of structure with high contrast was just right for the feeling I was going for. Having a strong silhouette against the moon was the main thing, everything else unfolded from that.

Composition changes a lot, so during the work you may lose something to gain something else. In the end, there’s no reason to hold on to any idea if it doesn’t make the image better. I had to give up on horns-hands because tree silhouettes worked with the rest of the shapes in the scene. Let’s compare the first sketch with the final image.

Obviously, all of the compositional elements I had in the blockout are still there: overall triangular theme, high contrast areas, areas of high and low detail, relations between big, medium and small shapes. More complex three quarter lighting now reveals not only silhouettes but also the forms inside them. Pines add more repetition of the same shape making unique elements (rectangular windows and a round moon) stand out even more. Tree silhouettes now are pointing down and leading you into the scene contrary to branches in the blockout pointing nowhere.

Light and Colour

On the blockout stage I move lights along with changing big shapes, then compare a lot of renders and pick one that works the best for mood and composition. It’s very easy with 3D and GPU rendering and happens almost in real time. I can try whatever lighting scheme I want in minutes and see what fits the mood. When I paint over my renders I simplify lighting and move accents to where I need them to be. With this particular piece I was worried about the light direction. Obviously, if the moon is in the back you can’t get lighting from the side. So I posted my painting to a few group chats and no one mentioned it as an issue. Amazing.

Colours are chosen more deliberately since there’s no tool that I can randomly yank around until I’m happy with the result. I did colour keys and some painting in the past, so most of my ad hoc decisions come from there. It's impossible for me to explain the whole colouring process in short, it has become too intuitive and entangled with everything else. But I'll go over some key points that could be useful even if you can’t paint at all.

I wanted something a little more subtle than an obvious teal-orange scheme. So I gathered references that in my opinion would work with my idea — lonely, not very inviting, mysterious house. Any particular reference doesn’t need to have everything right on point, but you must be able to tell what is it they have in common you like.

Most of those images have a strong single dominant colour, a little complementary accent, strong silhouettes and nice contrast between light and shadow. I tried to recreate those ideas in my painting. What I got in the end was a bunch of very close blueish-green desaturated shades. This allows moonlight to do its job in uniting things without being challenged by bright local colours.

Windows and torches are just lonely spots, they almost don’t give any light and this way the scene doesn’t become overly friendly with warmness. Now looking back after writing all of this, I think lights could be more yellowish-green to make the scene even less inviting and more mysterious. But it’s not an easy decision since the image becomes monochromatic and loses colour contrast.

Lighting in Unreal Engine

Lighting was very straightforward. I used as few lights as I could to keep changes quick and easy. For the same reason I used raytracing. Important thing here is to follow the concept and not to invent anything new. If I don’t like how it turns out I go back to 2D — it’s less restricting, much faster, and more flexible.

Story

Images can pass messages by their own means, so I'm not sure if an image needs a literary story. But it totally needs reasons to why it was put together this way and not the other. What I wanted is something that would be easy to tell a friend about, like “Hey have you seen that house with the moon?”. Believe it or not, that’s the whole story behind this scene. A lot (if not all) of modern popular art, including literature, utilise the same approach with great success. It proves to work better than the story I tried to put in my previous works with all the props and little hints about what happened and why. Looks like nobody got time for that.

Use of Silhouettes

To be honest, most of the silhouette work was done subconsciously. I’ve never even thought of this word and it has never caught my attention as a separate thing to work on until I heard the question. It was the basic element of the scene and I always thought of it as a part of the main idea. Looks like there's no hard line between silhouette, lighting and composition for me. Hope this helps, cheers!

Silhouettes Explained

Okay I’m kidding. Even with all of this being true I still can break down my thought process in retrospect and see if we can improve something.

Let’s start with the shape. Once again, mood and feeling will be our main reference points. I’ve mentioned “instability” as a thing this scene should have, and we can push it further. Let’s try and give a house impression it could be tipped over more easily.

Shapes became more distinct (notice the entrance), more unstable and exaggerated. Looks better to me. Now let’s add internal form and see how it will improve readability and add new information to the image.

Lighting draws another silhouette on top of the one we already have. Just moving a single light source around gives you a lot of options, but let’s take a look at two of them. There’s definitely a case when you would want to use the left image, but here it contradicts the mood of the scene — it’s too bright and open, something you would expect to see on a green field on a sunny day. Also frontal lighting makes things look more flat in general. So let’s go with the right one and add some texture.

It’s not much, but it describes materials and introduces the smallest level of detail to the scene.

As you can see, there’s always room for improvement at any stage of the work. We can repeat those steps as many times as needed to compare results and pick the combination of the best ones.

Textures

My basic concept for the scene is relying very heavily on lighting. I made the mistake of overlooking the role of the local colours in the past, put too much on lighting alone and the scene didn’t hold together as well as I thought it should. So this time I had to paint an unlit version of my materials. It doesn’t have direct light or shadows but still tinted to fit the setting and lighting better.

This was helpful for a few reasons. First, obviously, I got myself a reference. Second, I had a chance to breakdown materials layer by layer and understand them better while painting.

I don’t like designing something in Substance Designer, it's much easier to use it to copy something. It’s probably because I’m still very new to it and don’t understand the tool well enough to be efficient with it.

While working on everything from concept to real time scene I have to choose where I put the most effort and what must be simplified. So I tried to take the shortest, simplest route with my materials. It may not be the best choice for every piece of work because it limits the learning process, but I plan to catch up on that in my next work. Every material I did for this scene was extremely simple, here’s an example of the Substance Designer graph for the moss material:

I followed two general rules: start from the most important part and work with the whole scene together. This way I couldn’t dive into one material and work on it solo for 2 days, but I had to make very rough drafts and see how they work in terms of readability, detail, scale and colour. I constantly checked the materials in the scene on the actual geometry I was using.

Foliage

I didn’t know how to address this task at first, but had a few things to help me. First was my painting, the only thing I needed is to recreate that in the engine. The other one was pines in Overwatch - those guys have amazing ways to simplify things and it fits this image’s style. So I tried to recreate something similar. Had to approach it from two opposite sides, working on geometry and alpha in turns. Took me about 6 iterations to get there. I’m still not very happy with how it turned out — trees are a bit simplistic and not stylized enough for my taste, but in context of the scene they do their job. Gives me a room to grow, anyways.

Some of the alphas I had to try before I found the right one and used it in Substance Designer as a mask for Tile Sampler.

Two slightly different pines I used in the scene. One of the earlier versions is on the right.

Substance Designer, Substance Painter and Photoshop

I used Designer for every texture in the scene. I’m not a fan of any particular tool, they’re all amazing, so the deciding factor for what software I use is how quick it will give me what I want and how easily it fits into the process. For instance, I didn’t use ZBrush or Painter because sculpting/baking/importing/exporting would make iterations much slower, and for this scene I was happy with the level of detail and control I could get from Designer.

Inspiration

My inspiration comes from a lot of places: real life, books, games, art. When I come up with an idea I search if it has been done before. That’s when I go through Artstation, Pinterest and Google Images. With so many people working in visual arts you can always find a solution for your problem or something that will make you look at it from a different angle.

Feedback

Artistic community is being extremely nice and accepting nowadays, everybody just cheers up and gives compliments to everyone. I feel like feedback on early stages from right people could be sometimes even more helpful than any reference. So I have to ask for advice from a little group who I know will be honest about what they think and will force me to answer some questions about my decision making not only in context of one piece of work but rather in perspective of all my future plans and aspirations. But I don’t feel that’s enough for me, I definitely need to be more open about my process to get more diverse and professional feedback, that’s gonna be number one priority with my next work. Hopefully, EXP’s discord will help me with this.

Advice

As you may have noticed, I don’t refer to artistic decisions as good or bad, rather I always ask “does it work with my idea?”. So always remember the feeling and mood you are trying to put in your work and stick to it. Change what you don’t like, don’t be afraid to throw away what doesn’t work, and keep focus on the whole picture.

Future Work

Overall plan is to build a more focused and professional portfolio, so I’m going with a modular environment. I also want to continue working on stylization and push it much further and expand my understanding in that area. Feels like a third person boss fight level fits both of those criteria and gives a lot of room for memorable shots and nice ideas. But a lot can change after the first drafts so I can't promise anything.

Outro

Thank you for reading through all of this! Hope this article will help you to be more flexible and use more artistic tools in your works — it definitely helped me to re-evaluate my strengths and weaknesses and understand where I can improve. Message me right away on Discord or Telegram if you have any further questions.