This is a research through design project where I’m researching various aspects of making cinematic interactive experiences. For this I’m using Unreal Engine 5 as it provides a great set of tools from its advanced lighting and shader system to the ability to use Megascans and Metahumans. As a starting point for this project I’m referencing some of the (in my opinion) best cinematic games: The Uncharted series and the Tomb Raider reboot series. This is an ongoing documentation of my findings and research and will see consistent updates over the next couple of months. This is in no way comprehensive but more of a documentation of how I work and my findings along the way. It started out as a research project but turned into a game production where me and a team of 8 others are building a cinematic narrative game in the style of Uncharted. It will be a very short but high quality experience that shows that we have what it takes to do AAA development. I’m project manager, design lead and technical director and I’m learning a lot about what goes into producing this kind of game.
Current progress (finished)
The game is done and up on itch.io for anyone who wants to play! 😀
Background
First some background, I absolutely love the Uncharted franchise. My first interaction with uncharted was back around 2009. My brothers friends came over to our house for new years eve. They asked if they could use my PS3 and I was like sure. One of them had brought with him the new Uncharted 2. I watched in awe as they played the first few hours of the game. Not long after I got the first 2 uncharted games myself and played them all the way through. I played the 3rd game as soon as it came out and the 4th as soon as I could afford a PS4. I still replay them all every 1 or 2 years. Same thing for The Last of Us, also a Naughty Dog game and actually my favorite game franchise of all time with Uncharted coming is second. These games have had an immense influence on me and have in my opinion set a new standard of what games can be. I’ve always wanted to work on these games and decided a few months ago that I wanted to try and create a similar although obviously much shorter experience as my “Project Vrij II” project. A school assignment where we have 8 weeks and a team of up to 8 people to create a game. So I set out putting together a team and figuring out the possibilities and resources I had access to. I managed to put together a team of 2 Developers, 4 Artists, 2 Designers (me included) and one Audio Designer. Audio Designers luckily did not count towards the team size for this project. So I’ll be managing a team of 9 people.
Preliminary Research
Not interested in the research? Jump to development!
Motion Capture
On of the first things to solve was motion/performance capture. It is a big part of the cinematic experience in those games. Luckily at our school we have what we call the Blackbox, it’s a large room set up with an OptiTrack system. This is an amazing system to use for motion and performance capture. I have used it in the past to capture human performances and it works great. I honestly don’t think it will give us many issues (fingers crossed). It however doesn’t do facial performance capture. The room also isn’t great for voice acting. After looking through lot’s of different ways of doing facial capture for Unreal I’ve settled on a solution that works well enough for what we want to do. It’s called Face Studio and all it needs is a decent quality video of the face during a performance and will connect directly to Unreal Engine. It also works great with Metahuman faces. Face Studio is payed software but they offer a free 30-day trail which should suffice for our use case. At first I was thinking about strapping webcams and microphones to people’s faces but I figure that would get too janky and complicated. So I decided to separate the body and voice + face captures. I will first have actors act out the whole scene in the Blackbox then I’d put them in front of a camera and a high quality microphone and record their face and voice at the same time. Then we just play those in the simultaneously in the engine. It’s not the best solution but it should work well enough for what we are trying to do especially since our budget it basically non existent. After some more research and trying various solutions we’ve found that using the live link iPhone app works much better than Faceware Studio so we’ll use that instead.
Environments
Uncharted and Tomb Raider use very similar sorts of environments. Lush landscapes, ruins, temples, etc. are very common place. They also have scene that take place in modern environments however they are less common. For our experience we’ll realistic only be making 1 or 2 levels so we can decide to forgo those. I have some useful assets in my Unreal Library such as: Modular Lost Ruins Kit, City of Brass: Environments, Procedural Nature Pack Vol.1 and a lot more, as well as all the free Megascans content. That should provide the majority of our environmental models and props. We also have 4 Game Artists in our team so if we need anything we don’t have they can always make more.
Exploring Naughty Dog’s level design process
1. Concept phase
It all starts with the writers and directors working with concept artists in an iterative process make concept art that best represents the directors vision. This can take many iterations to get the style, look, feel etc. for a level right.
2. Level Design & Blockout
Now that the idea for a level is clear it is handed over to the level designers who again work iteratively in the engine designing the flow, layout, how the story progresses here etc. They do this working closely with the director to make sure it all makes sense within the larger scope of the game. Level designers will have varying degrees of creative freedom depending on the level and it’s role in the story. It is very important to balance the game flow with careful use of Geometry, Composition and Scripted events.
Geometry: The collision, physical interactive objects, composition of the scene, shape design all helping to guide the player in a natural way while providing challenges. Using focal points where you pull the players towards something using geometry and assets or using contrast through lighting, color, texture, etc.
Scripted Events: All of the scripted sequences in the level, enemies, companions, explosions, special set pieces etc. That all help guide the player and give feedback on what they’re doing.
When using assets to guide the player they can either be direct (like a sign) or indirect like a series of pickups that form something of a path.
An important tool to naturally guide the player is the use of shape design to guide the player. Lines, silhouettes and pathways can greatly influence the player.
An example
Here you can see an interesting use of those concepts. This is a screenshot from Uncharted 2. The red arrows show how the surrounding landscape lines guide the players eyes towards the city. The green shows a clear pathway that is also much brighter than the surrounding area thus catching your eye. The silhouette of the city is very detailed while the mountains behind it are in the fog, lacking detail and color. This makes the city pop. The blue circles show assets that help provide some more parallax, giving depth to the scene which is supposed to convey a grand scale. The purple lines show how some of the domed structures around the main temple are askew. This makes them more visually interesting as well as portraying a level of decay.
Another example of the same concepts, this time from Uncharted 4.
Also if you have a returning landmark it can help the player feel grounded, track their progress and stay connected to the environment. Using lines and silhouettes to accentuate those can help the player be aware of the landmarks.
Here’s an example of a landmark in Uncharted 4. This uniquely shaped mountain is part of the scenery on the island of Libertalia. It’s unique shape makes it instantly recognizable. The shape of it looks intimidating and scary which is foreboding but also makes it stick in the players mind quicker. It grounds the world and makes it feel more consistent as you progress through the various stages of the island.
Colors can also help guide the player. You can either teach them from the beginning that a certain color guides the way; think red in mirrors edge or yellow in uncharted. You can also have your environment be somewhat monochromatic or dull and have something stick out by being bright or a very contrasted color the the rest of the scene. Colors can also invoke emotion and meaning.
Level Design for Combat, shares many of the same principles as before however instead we’re trying the give the player plenty of line of sight and overview. Verticality can also really help make combat a lot more interesting. There are many occurrences in uncharted where Nathan is climbing on something while in combat and it’s really cool an dynamic when the systems work well together.
Line of sight: Uncharted is a cover based shooter so you want to have a large and varied amount of cover throughout the combat area. However it is important to make sure the player can keep a clear overview of where enemies are so they don’t feel overwhelmed.
Verticality: Verticality makes combat a lot more dynamic as it involves the jumping and climbing mechanics into the combat. This helps to avoid the classic problem that a lot of cover shooters have where it’s just, cover, shoot, move up, cover, shoot, move up, etc. Having these jumping and climbing mechanics allows this verticality to make combat far more dynamic. Climb up and pull an enemy off of a ledge. Jump down on an enemy and kill them that way, swing around the corner, etc. etc. It makes combat far more engaging.
Mechanics
Both Uncharted and Tomb Raider are third person shooters with climbing and stealth mechanics.
Movement
I’ve started with Advanced Locomotion System as my base. It is an amazing starting point and already supports pull ups on ledge and mantling. As well as foot IK, rolling, crouching, ragdolls and a bunch more. Which is great because it save me a lot of time. It also is very well made and easy to expand upon using C++ or blueprints. It still needs a cover system and a better climbing system.
Combat
For combat we need various weapons (mostly guns) and a cover mechanic. We want to try to include as much dynamic combat as we can, using the environment and movement mechanics. Although within the scope of this project that might be a bit much. I’ve already added a basic weapon system to the Advanced Locomotion System as a trial and to figure out how it works and how feasible the project is for when we need to add more mechanics to it later.
Cinematography
Uncharted uses a lot of the same techniques as regular filmmaking does when it comes to shot composition and framing. Screen shakes and camera movement are really important for conveying mood and impact. An important thing to keep in mind is that the camera is the player and the player is the character. By which I mean the camera is the lens through which the player experiences the character. (No pun intended) As such the camera should react in a similar way to the player. If you’re dying the colors might get more dull, the camera movement might get more sluggish or start “limping”. This helps greatly with immersion. Once again with cinematography colors, saturation and contrast come back as a very important tool of conveying information to the player. However more tools are added as well such as bokeh, camera angle, framing, etc. This is just an overview of some basic cinematic techniques as it’s way too big of a subject to discuss here.
Composition
Placing the objects and characters in a scene in a way that makes sense, is visual appealing etc. Composition is generally the first step when setting up a shot and can be very instrumental for the mood as well as readability of a shot.
Framing
Where are you gonna put the camera in your composition? Putting it very close to your subject can be very intimate but could also be claustrophobic. Context really matters here. Where are your characters? Are they in the frame or out of it? What does the silhouette of your shot look like? Are your characters centered in the shot or not?
Camera movement
I think it’s interesting to think about who the camera is. Like I said earlier the camera is the player and the player is the character. So in a sense the camera is both the player and the character and the movement should reflect that. Camera movement should change based on what is happening, how the character feels, how the player feels, etc. Having a sluggish camera with a lot of 6 axis noise can convey that the character is dying, sick, drunk, etc. Screen shakes when you get hit or fall from a height. As well as generally how the camera moves can add a lot of life and feeling.
Continuity
If a character is moving out of the frame on the right side. Make them enter then next from the left. If you wanna change sides add an intermediate from the front, back, top bottom, etc. Make your shots flow together well. Or intentionally don’t. If you want something to come across as chaotic or confusion you can play a lot with this. Generally though continuity really helps scenes mesh together well instead of being jarring.
Optics
With the tools we have nowadays we can make cameras in games with whatever type of lens we want. We can play with field of view, bokeh, zoom, focus, etc. all on the fly. This gives you an amazing set options for your scenes. Use bokeh to focus on a subject. Change the field of view for interesting effects. There’s just so much you can do.
Story & Writing
The player isn’t stupid, don’t act like they are. By which I mean, you don’t have to spell it out for players to understand a situation. Even more so preferably don’t explain things with dialog if they can be conveyed through other means. Narrators are often quite “cheap” and crude tools to set a scene when there are so many far better ways of doing exposition. You could have a narrator say “Layla is a tough girl” but instead it’s much cooler to have a shot of Layla being a badass, beating up some thugs, etc. It’s a similar message but one has much more impact and intrigue. Why was she there, what was happening, who is she? etc. and because the player isn’t told what to think it’s a much more enjoyable experience for them as well. There is a scene in the first episode of the show Wayne where he gets beat up then walks away bruised and bloody and he doesn’t care. It really stuck with me because it was an odd scene however it also made it clear that physical pain doesn’t bother Wayne, it said a lot about the character without directly telling the viewer anything. Another example would be providing more lore through npcs talking to each other or through signs and notes in the world. This way your not just telling the player directly. These not only help to fill the world and make it more real and alive but they also allow the player to explore the lore as much or as little as they want to in a more natural and passive manner. In a way it all comes down to show don’t tell. I’m not an expert on writing or narrative design in any way but it fascinates me and I’m trying my best to learn more as I go.
Time to start developing!
So the time has come where the project officially kicks off and we’re in full swing.
Writing the script
The first thing that needed to happen was to write the script. So I got together with everyone in the team to start discussing what direction we wanted to go in. I wanted everybody in the team to be able to voice their opinions and ideas at this first stage of the process. We outlined the main characters and groups as well as their goals and motivations. We talked about environments and styles as well as presentation. In the end we came up with a rough outline for all of those things. A few days later I sat down with Sabe (the other game designer), Jurre (the art director) and Max (the audio designer) and we started writing up a first draft of the script. During which we fleshed out the characters and world. Once we had finished we took this draft to Eline (our concept artist) who set out making concept art. In week 2 and 3 Sabe and me worked on refining the script, adding notes for various things like what is and isn’t a cutscene and where those are triggered. We went with a main character + companion setup for our story because it allows for interesting interactions between the characters. Our main character is Matthew he is a treasure hunter and is roughly 30 years old. His companion Samuel is a bit older (around 45) and is sick. They’ve been hunting treasures together for a long time but ever since Samuel got sick they’ve been search for a cure. The game starts with Matt and Sam arriving near the entrance of an old tomb on the side of a volcano. Old stories told of an ancient cure hidden inside. They venture into the tomb hoping to find the cure and save Sam.
Early prototypes and blockouts
Coming from a development background I immediately set out blocking out a level and building various interactions. I found that Unreal Engine 5 works amazingly well for rapid prototyping and I was speeding through the process. Also ALSv4 was a great base for our character controller. Within a day or two I had a testable block out. I tested the block out with various team members as well as my co-designer and we discussed what worked and what didn’t. After that I went back in and started adjusting. Going back and forth like that iterating on the level design.
As soon as the script was done I implemented a basic dialogue system. It allowed me to implement all the dialog from the script as text and display it as subtitles. There was no audio recorded yet but this allowed us to feel out the pacing of dialog and how the conversations flowed within their respective contexts. This helped immensely with refining the script before we went into the Mocap studio with our actors.
Performance Capture
Now that we had a mostly finalized script and our art director had drawn storyboards we were ready to head into the motion capture studio with our actors to get all our animations. We took 2 days to make sure we could get all our different animations and we got several takes of each scene and animation to make sure we had some options. It all went great and the animations looked very good. Our actors did a great job and we allowed them to alter things they didn’t like so we could get the most genuine performance. I’m really grateful for all the help and support from my team because it was very stressful but they were amazing! And of course our actors deserve a great amount of props for delivering 2 days of great performances when all we could offer them in return was some take out and our gratitude.
Environment design
Environment design was mostly handled by our lead artist Jurre. Who did an amazing job making the world feel vibrant and alive. Taking ques from the research we did we used various techniques to guide the player. Many of which are described above in the research section.