[Development] My first engine

[Manoo]

Hi guys, I have been making my 3d engine for past 3 months. It's my first try and honestly it's also my first c++ program (I don't count school educational programs). So I want to share my program, If u wonder why I'm doing this, I just want to know what u think . I know this engine still lacks lots of features like sky, particales and more... But as I said before just let me know what u think: maybe it is look crappy for your or not very optimized or maybe some ideas about developing.

Requirmanets:
-Windows 7/Vista not sure about 8 and 8.1 never tried.
-Video card that is capable Dx10

By the way I made it totatly from scratch with c++ ,directx 11 and I used two addional libraries bulletphysics and assimp. I don't know if it's worth it as I said it's my first try, because I saw unity now is most popular way to create games and sometimes I have second thoughts about continueing my engine. Maybe some one experienced could tell me, what is better to create game on your own engine or on other engines like unity, orge?

And in the future I'm planing to create a game with my engine like zombie shoter or smth. In this version I just put some simple models for showcase and there's also a child from warcraft3 animated by blinky if i'm not wrong .

Sorry for bad gramma , I hope it's readable .

Controls:

W - move forward
R - spawn a box that moves forward your look at point(100 model max, after this point u gona get error )
esc - close app

(Engine -> http://www.hiveworkshop.com/forums/pastebin.php?id=prhxng <-)

ScreenShot

ScreenShot

 

[edo494]

why 2010!!!!???

ogre is not game engine, google search:

"OGRE – Open Source 3D Graphics Engine" - Graphics engine, not game engine, it doesnt have any physics or collisions or anything like that, it just allows you to create 3D graphics with C++ level of abstraction

the advantage of unity is the fact that everything has been already made, and "decently" optimized, but most of the time you have to pay for commercial use, or for extensional packages.

Your own game engine will most likely not be as capable as something like Unity is, because Unity is result of thousand of hours of work,
However with your own engine, you can control all the stuff, and if something feels unconfortable, you can go and change it, which I doubt any big game engine allows you to. Also not all game engines are made for language you like the most(Unity uses JavaScript or C#), so if you want to use language of your choice, you may end up creating your own engine, as you did

At the end, its up to you to decide

 

[GhostWolf]

• You packed it in a RAR, this means you are a terrible person.
• You are using BMP files instead of PNG/DDS/Anything other than BMP. This means you are a terrible person (both DirectX and OpenGL know how to load DXT (and many other formats) compressed textures, most commonly stored in DDS files, so the fact that BMP files are easy to load isn't an excuse).
• Not only are you using X models, but in their text format. This means you are a terrible person (but I'll let this one slip, because finding a decent model format isn't hard, it's impossible, just change them to binary).
• You should use TTF files for fonts (for the sake of customization).
• The main loop is terribly inefficient, draining both the CPU and GPU.
  • Run the logic code at some fixed rate (e.g. 60FPS), I assume right now it's running as fast as it can, which is useless and hogs the CPU.
  • Enable by default vsync to not hog the GPU.

Can't say anything beyond that, because you didn't release the code.

Now about your question - what do you want to make? a game, or an engine?
If you want to make an engine for the sake of making an engine (e.g. as an educational experience), then go for it.
If you want to make a game, and you are making an engine for it, don't do that. Instead make a game.
This doesn't necessarily mean using another engine (but I encourage you to do that), this just means that you need only the code to run your specific game, as opposed to trying to make a fit-for-every-case engine code (which engines don't do either, by the way).

 

[Manoo]

Thank you for feedbacks )

edo494, I know orge is not a game engine, but visual part I think one of the most important part of the game. By the way I hate c#, because it is only works on Windows and coding on this languege only going to close door to other systems.

Why 2010 vs? Dunno, like new version would improve my coding :/... Just sometimes it's really pain in ass to download some microsoft programs, by they way I think newer version has some costs or no?

GhostWolf, I agree with everything just don't understand what Winrar done to you so horrible that u hate him so much .
Bmp format is easly changable i don;t see a big deal about it and yea I heard about compression thing, but I use bmp mostly only for hightmaps and even this part is for tempraly use . Any way basicly I tested dds, png and bmp without any compression they all size same, but blizzard uses dds for some reason so I think I'm going too.
Yea I totaly forgot to change x format back to binary :/, but is one click deal any way I'm using Assimp. About TTF could u please explain it more? I hope u said my inefficient loop only because of vsync , I left this part deliberately, if u wonder why, because I can see how fps reacts to new code. If I had costant 60 fps I wouldn't able to notice if my new code parts are slow. Don't get me wrong I don't regret programing wc3, it's actualy inspired me to program. However for me unity programing is very similar to wc3 it's more modding than programing.

By the way biggest reasons why I don't want to develop on other engines is that they have their own languege with all kind of assets, functions. And if u decide in the future to move on to other engines, It would been like lots of time wasted, I know it's not totatly right u still learn game logics. But for instance I have been developing wc3 for about 4 years and after i moved to c++, experiences I had with wc3 helpt me very little with c++ coding.

 

[edo494]

Thank you for feedbacks )

edo494, I know orge is not a game engine, but visual part I think one of the most important part of the game. By the way I hate c#, because it is only works on Windows and coding on this languege only going to close door to other systems.

Why 2010 vs? Dunno, like new version would improve my coding :/... Just sometimes it's really pain in ass to download some microsoft programs, by they way I think newer version has some costs or no?

well, if you install mono on Linux, I think you can interpret C# normally

And yea its pain to download, but the 2013 express is free, as all expresses, and it allows you to use most of the C++11 stuff, like lambdas, variadic templates, move semantics, std::nullptr_t and nullptr instead of shitty C++ NULL, standardized regex, threading, inteligent pointers and more(constexpr is not supported by Visual Studio, maybe 2014 if it even comes out), but well its up to you really, I dont blame you or anything just noticed you use 2010

If I had costant 60 fps I wouldn't able to notice if my new code parts are slow. Don't get me wrong I don't regret programing wc3, it's actualy inspired me to program. However for me unity programing is very similar to wc3 it's more modding than programing.

very valid opinion, and you can find out which parts of the code you implement are really slow, but as long as you have 60 FPS, its fine. Check my signature

But for instance I have been developing wc3 for about 4 years and after i moved to c++, experiences I had with wc3 helpt me very little with c++ coding.

that is true, but Jass and potentially vJass gives you the basics of programming if nothing else (Functions, local, global variables, vJass shows people OOP and more crappy useless(in wc3) stuff)

 

[GhostWolf]

RAR is a propriety format that requires one to install a 3rd party program. Zip is an open format supported natively by pretty much any OS and program.

DDS (specifically DXT5) has a constant 1:4 compression. This isn't very good compared to other lossless formats like PNG, but the important part is that it is a format that the graphics drivers know how to work with.
That is, if you compress a 100KB raw image (e.g. BMP, TGA) to a 10KB PNG, you'd still have those 100KB at the end in graphics memory. The DDS however, would be 25KB both on disk and in graphics memory.
In addition to this, DirectX and OpenGL can directly load DXT data. For DirectX, I assume there is a function that can load a DDS file, and for OpenGL you need only to do a little parsing to extract the DXT data inside it (which is very easy). This in fact makes loading DDS files as easy as BMP.

Your logic should run at a constant rate either way, if you want stable physics.
As to rendering, you should give the option to enable/disable vsync.
If you want an easy way to decouple the logic and rendering rates without going into multithreading, check this out.

TTF, or TrueType Font, is pretty much the de-facto standard when working with fonts. Never looked at how the format works, but you can always google it.

And as a side note, I suggest you to look at PhysFS. It's a very neat library.

 

[Dr Super Good]

•You are using BMP files instead of PNG/DDS/Anything other than BMP. This means you are a terrible person (both DirectX and OpenGL know how to load DXT (and many other formats) compressed textures, most commonly stored in DDS files, so the fact that BMP files are easy to load isn't an excuse).

One could argue all day as to whether or not using BMP is a dumb idea or that DDS is the ultimate real time graphic texel source. However the reality is that a good game engine system should be able to cope with any number of image formats as long as you are willing to write plugins for it. An adapter could be used to guarantee all required data is present (mipmaps, colour format, etc).

The best example of this in action is with images stored as jpeg chunks (like blp). Unlike DDS which has only native compression (can literally be imported into GPU 1:1 and work), JPEG requires considerable and costly processing to form an uncompressed image out of. Despite this it was used heavily for storing textures in old games where space was a major problem. Even WC3 .blp uses JPEG as it can often compress better than any other kind of compression. This is all done by first decompressing to an uncompressed image and then adapting that image so it can be fed as a texture.

•Not only are you using X models, but in their text format. This means you are a terrible person (but I'll let this one slip, because finding a decent model format isn't hard, it's impossible, just change them to binary).

Again this is an extension sort of problem. His engine might end up supporting many common model formats with various loaders that all convert them (adapt them) into the standard native data that the engine uses. Be it loading from a .mdl or.mdx or .obj or any kind of silly format imaginable. Each may have pros and cons but in the end support for whatever should never be a problem.

•The main loop is terribly inefficient, draining both the CPU and GPU.◦Run the logic code at some fixed rate (e.g. 60FPS), I assume right now it's running as fast as it can, which is useless and hogs the CPU.
◦Enable by default vsync to not hog the GPU.

You telling me it is not some multi threaded monster? I will look at it tomorrow but this sounds concerning.

If I had costant 60 fps I wouldn't able to notice if my new code parts are slow.

You profile the code and measure? I mean that is what the people over at 0A.D. did to find where the problems were.

 

[Manoo]

Thank you for your all answers .

Ghostwold, directx has implemented vsync, so I don't need to create my own.

And yea its pain to download, but the 2013 express is free, as all expresses, and it allows you to use most of the C++11 stuff, like lambdas, variadic templates, move semantics, std::nullptr_t and nullptr instead of shitty C++ NULL, standardized regex, threading, inteligent pointers and more(constexpr is not supported by Visual Studio, maybe 2014 if it even comes out), but well its up to you really, I dont blame you or anything just noticed you use 2010

NULL just a simple 0 , why do u need nullptr_t?

By the way I got one problem and is about networking, in wc3 I didn't had to do anything with player sync and movements, so that's why I'm lack of understanding how the all architecture works.

For example I saw some games creates theire networking differently.
In Hon, if u give an order, your client sends it to server and then your client waits for the answer.
In Diablo 3, everything is same but your client doesn't wait for server answer and if unit is not in place it get's synchronized.
This informations is not 100% accurate, but I saw this behavarios during lags or smth.

So my question would be, how server logic should work? Should I make this steps?
1. Then I interact with the game inputs, for example I pressed W (to move forward).
2. Then server gets a massage, it checks if everything alright no hacks or smth.
3. Then server sends to all clients that my hero is moving (or moved?), also should it send information with all game units position in case there was desync?
4. Client sees that my hero moving.

That's how I imagine networking, tell me if i'm wrong, I never tried it . Damn I love game programing ))

 

[GhostWolf]

@DSG

Graphics memory is more important than hard drive memory. I don't care about games that were made 12 years ago.
The best of both worlds is to put DXT containers (e.g. DDS) in an archive, of course.

It doesn't matter what an engine supports (though it should just support its own internal model structure natively), the DEMO uses terrible BMP and X resources.

@Manoo

I didn't say you have to implement vsync, you can't actually do that (it's on a driver level).
Just have a way to swap between enabled or disabled vsync.

Networking is a big subject, and it really depends on how you want to do it.
Generally speaking, the easiest form is a server-client model (unlike peer-to-peer).
Ideally, a client in a networked game is like a console - you send inputs to the server, the server runs the game simulation, and it sends you back an image that you render.
In reality, this isn't possible because of pings, bad connections, bandwidth, etc..
Games such as Diablo 3 (and really, almost any networked game from the past...many years) use a form of client-sided predictability.
This means that both the server and the client run the game simulation. The client sends actions to the server, and runs them immediately locally. The server goes and simulates the actions too, and returns a response. If the response was equivalent to the local action, everything is fine. If it isn't, the client has to correct itself to match the server (the server is always the correct one by definition, to avoid hackers, or just bad predictions, since the client doesn't necessarily have all the information that the server has (and it shouldn't, again, to avoid hackers)).

nullptr actually isn't the same as NULL, it's a better implementation that should have existed since day 0, but whatever, I don't like C++ so it can burn for all I care.

 

[edo494]

Thank you for your all answers .

Ghostwold, directx has implemented vsync, so I don't need to create my own.



NULL just a simple 0 , why do u need nullptr_t?

By the way I got one problem and is about networking, in wc3 I didn't had to do anything with player sync and movements, so that's why I'm lack of understanding how the all architecture works.

For example I saw some games creates theire networking differently.
In Hon, if u give an order, your client sends it to server and then your client waits for the answer.
In Diablo 3, everything is same but your client doesn't wait for server answer and if unit is not in place it get's synchronized.
This informations is not 100% accurate, but I saw this behavarios during lags or smth.

So my question would be, how server logic should work? Should I make this steps?
1. Then I interact with the game inputs, for example I pressed W (to move forward).
2. Then server gets a massage, it checks if everything alright no hacks or smth.
3. Then server sends to all clients that my hero is moving (or moved?), also should it send information with all game units position in case there was desync?
4. Client sees that my hero moving.

That's how I imagine networking, tell me if i'm wrong, I never tried it . Damn I love game programing ))

std::nullptr_t is special type that is convertable to any pointer type.
The problem with null is as follows:

code

void someF(int);

void someF(char*);

int main()
{
    char* c = NULL;
    someF(4); //calls someF(int)
    someF(c); //calls someF(char*)
    someF(NULL); //oi! this calls someF(int)
    someF(nullptr); //calls someF(char*)
}

edit: My idea about networking is not bad, but it has no desync protection, much like wc3

 

[Manoo]

Networking:

I think it is good to do it in wc3 fashion, you dont send positions of all units or anything like that, you just send the orders with all needed data in(in wc3, you would just send that this and this and this unit received this and this order with this and this target or at this and this position)

But I think, you still need to send unit positions to check and fix desync or not?

By the way what is better for terrain Bump or Normal? In my understanding:
Bump - represents vectors to show surfuce and it's create more realistic object. It's my current choice.
Normal - I don't get it, some say it's just like heightmap to imitate surfuce, if it is I guess it's much more faster than bump and doesn;t requires tangent ar binormal vectors.
Fix me if I'm wrong.

Yea and my goal is to create as much realistic graphics as possible so I guess bump would be the best choice .

 

[edo494]

yea, this idea is not really complete and I think GhostWolfs idea on networking is much more solid, I edited it already

 

[GhostWolf]

Bump mapping is a general name for a couple of techniques (normal mapping is one of them) to make planar geometry look...not planar.

Normal maps store geometry normals on a per-pixel basis to match the UV map.
Since lighting is completely based on normals, if you have normals that would have matched a much higher resolution model than your current one (and you have enough of these normals at high density, hence per-pixel), you can use the lighting to make your model look similar to the high resolution model.

This doesn't change the actual geometry, of course, which means that if, for example, you are looking at a wall with normal mapping, and you look at the edge of it, no matter how supposedly bumpy the wall is, the edge will be one straight line.
For this reason, normal mapping is mostly good for depressions rather than bumps.

Here's the example image from wiki:

Normal mapping is very cheap to calculate at run-time and it produces pretty decent results in most average cases, and so it is used in pretty much every game in existence for a very long time (I remember it was a cool new technique when Doom 3 was introduced, but it was used also before that).

To check the results, open up any 3D game with high quality graphics, and look closely at the edges of models.

What you refereed to as bump mapping might be displacement mapping, which stores displacement vectors that actually change the geometry itself. I never looked closely at this technique, so I can't really comment on it.

 

[Manoo]

So they both represent normal vectors for pixel this one I knew, but what's the differnce between them. Because in starcraft 2 normal maps are very very strange I could say they look more like a heightmap.

 

[GhostWolf]

You can use normal maps also for straight planes, which can act as a height map, but remember this doesn't actually change the geometry in any way, it's just a lighting illusion. This will look good as long as you look at the plane from above.

Height maps store actual displacements for the geometry (and are thus a simplified form of displacement maps).

/Edit

As an interactive example, check this out with the Unshaded and Unshaded + Normal map modes (assuming the viewer works for you).
The geometry is exactly the same, but the fake normals make the second mode look much higher poly. If you look carefully at the edges of the model, however, you can easily see that it's still low poly.

 

[Manoo]

Yea i know, i know how do u think I created bump on my engine . I'm just asking the difference between normal map and bump map, because clearly they are not the same.

 

[GhostWolf]

Again, bump mapping is a general term for different techniques that make geometry look as if it has bumps (or depressions). One of the techniques is normal mapping.

 

[Dr Super Good]

Graphics memory is more important than hard drive memory. I don't care about games that were made 12 years ago.
The best of both worlds is to put DXT containers (e.g. DDS) in an archive, of course.

Yeh, my 4 GB of dedicated graphic memory might be depleted by a few un-compressed textures! Oh wait...

Nothing stops you from introducing a rather poor DXT compression adapter to the image resource loading process if you really want all textures compressed. Additionally unless you are going for a AAA type project it is unlikely you will make enough content to even strain the video memory of even the poorest dedicated card. If video memory really is an issue you can also downsize all surfaces by a power of two (load a mip-map level instead of full resolution) which will save more than enough space. I know my graphic card is above average, but it is no where near top range.

But I think, you still need to send unit positions to check and fix desync or not?

This is a fully deterministic game engine approach. Each client has an exact copy of the actual game state (not graphics) that is reproduced in parallel. If one client deviates from this state (failure in determinisim) you have two options.
1. Drop. this is what WC3 and SC2 use and why people get dropped (bots even explain when this happens).
2. Rebuild the entire game state from scratch. This requires you choosing a client with a reliable version of the state and then transfer it to the others. Problem is you need at least 2 clients to be identical to detect such an issue and that modern internet has poor upload capabilities (I only get 20 KB/sec if lucky) so it can take many minutes to resynchronize.

Yea and my goal is to create as much realistic graphics as possible so I guess bump would be the best choice .

You may want to look into physical rendering then. This is what very modern game engines are using. It also makes content production easier as it gives more understandable quantities for surface properties.

What you refereed to as bump mapping might be displacement mapping, which stores displacement vectors that actually change the geometry itself. I never looked closely at this technique, so I can't really comment on it.

He may be referring to tessellation based techniques. This is another way of giving very simple geometry higher levels of detail on modern DX11 compatible cards.

 

[Manoo]

This is a fully deterministic game engine approach. Each client has an exact copy of the actual game state (not graphics) that is reproduced in parallel. If one client deviates from this state (failure in determinisim) you have two options.
1. Drop. this is what WC3 and SC2 use and why people get dropped (bots even explain when this happens).
2. Rebuild the entire game state from scratch. This requires you choosing a client with a reliable version of the state and then transfer it to the others. Problem is you need at least 2 clients to be identical to detect such an issue and that modern internet has poor upload capabilities (I only get 20 KB/sec if lucky) so it can take many minutes to resynchronize.

Damn it makes sense, that's why in Sc2 and Wc3 than one client in game loses connection for some times all other players have to wait, I guess it's to avoid desync. But in my opinion this logic only works for strategy games, imagine this in WoW .

 

[Dr Super Good]

Damn it makes sense, that's why in Sc2 and Wc3 than one client in game loses connection for some times all other players have to wait, I guess it's to avoid desync. But in my opinion this logic only works for strategy games, imagine this in WoW .

Exactly. Also it has the problem I vaguely hinted. Who determines what is the correct state? I mean if 2 clients modify the game (cheat) then the non-cheater would be out of sync with them, it would appear as if they were the one dropping and not the cheaters.

In games like WoW and D3 you cannot trust the clients. They can be sending you hacked results. Thus they have 1 central server that does all the calculations with the results being streamed to clients. The clients then interpolate intermediate results with some form of periodic correction.

 

[Manoo]

Hey I got another problem with networking. Okey for example let's have situation where are two clients and server and In games there are only two heroes (one for client). Client 1 moves his hero (whos address is for example 0x00001 in his client), but in server this hero address is 0x00002, so the problem is how server should understand that moved hero (clients 0x00001) is servers hero (0x00002).

One of the solution, Is to numerate all units and send to server not the address of hero, but his number. However I just started using vectors to store hero and if u destroy one segment of vector all numeration moves, so it will create another problem. In addition to this I would had to create all units in same order to keep numeration synchronize.

Other solution, or maybe server should give the address to hero, for instance if new hero is created, server sends to all users that hero is created and address from server prospective. Futhermore clients saves server gives address of hero to struct where hero is saved. This solutation makes sense to me:>.

Any way maybe anybody knows how it should really work, please share some ideas .

 

[edo494]

you can just give class representing Hero's data a additional integer which is the id of the unit, and do something similar to wc3s handleId, you assign closest free(unassigned) integer to 0 to the unit

As long as the ids are the same, the game is synchronized, when they are not the same, you are already in desync

 

[Dr Super Good]

One of the solution, Is to numerate all units and send to server not the address of hero, but his number. However I just started using vectors to store hero and if u destroy one segment of vector all numeration moves, so it will create another problem. In addition to this I would had to create all units in same order to keep numeration synchronize.

You assign every object a unique identifier in a map (not vector). You can then remove elements by removing the mapping and all other mappings will remain unchanged.

Maps might not be the fastest thing to use. Instead you use a sort of handle table (like WC3) where every element is given a unique handle ID (row in the table). The table itself could be a vector (of structures, which represent a row) where only the end elements are removed, the rest are made "empty rows" (special value) and recycled later to keep indices constant.

 

[Manoo]

Okey I think I'm going to change .x format of models to .fbx. So, I made little test importer with FBX sdk and I can say that library is really confusing and lacks of documentation. Any way I think for the first format is the best choice, it got almost all new features execpt physics solvers, atleast I couldn't find them. Eventhough this format is very heavy, slow and got all kind of usless data.
Do u think is it worth it to change .x to .fbx?

I made one screen shot with fbx format and as u see this model is from SC2 Art tool .

Fbx screen

 

[GhostWolf]

At the end an engine should have its own model format that is optimized for it (or multiple ones, if different types of model assets should be represented in different ways).
As to importing models, the more the merrier.
Since you are using Assimp, if you can convert its model structure to your own (when you make it), then you are already good on the importing front, since it supports many formats.

 

[Manoo]

GhostWolf, I know that every game company creates they are own format and it is the best choice format that fits perfectly for your needs. But creating your own format is extremly hard. Firstly, u need to make perfect structure of format, I would say it is easy part. The hardest part is, to code exporter for my format on different programs like 3ds max/Maya/Autocad and as I suspect they all using different programming langauges.
So that's why for my first game release I'm looking for best format that's allready created. Eventough for some time I was using Assimp, but almost all his formats are lack of smth, that's why I'm thinking to move on FBX SDK.

 

[GhostWolf]

http://assimp.sourceforge.net/main_features_formats.html

They also state FBX support is on their TODO, not sure what time-frame that means.

Either way, what are you missing in the other formats?

 

[Manoo]

Material layers like specular, bump, emissive, Animtion layers (like run, walk, attack), physics objects.

3DS- no comments.
COLLADA - same as fbx only crapier...
3DS- no skin animations, normals saved as smoothing groups.
X - one of the best choices, but lack all new features
MD3, Md2 - I'm not sure but they don't have specular, emissive, normal and I hate that animations are exported in other file

Half of other formats aren't finished, bugged and there's no program that can export them.

 

[GhostWolf]

I actually remember coming to the same conclusion a few years ago, but then I saw FBX is closed with no proper specs, so I dropped the notion of ever getting a decent portable model format.
Like I said in my first post, finding a good model format isn't hard, it's impossible.

On a side note, MD2 and MD3 don't even use skeletal animation, 3DS is a terrible format (and not meant for portability), X as text is a terrible format (and not a good one as binary either).
MDX is possibly the only half-decent format I've used, and it could be much better too.

I wonder how good the Assimp support for *.blend is, because that could be effective (assuming one can use Blender).
Either way, making your own exporter might be the easiest solution, as wrong as that sounds (it isn't hard for Blender, never tried making one for 3ds Max).

 

[Manoo]

X is not so bad as u think, it's very very fast and lighty format. I could say they almost indentical with X and MDX even in feature level, except x got better skining. I even have a feeling that blizzard used x format as a template creating their own, but who knows.

 

[Manoo]

Okey I got another problem. I just have made vsync with maximum 60 frames. But when I send any windows message non stop (like holding keyboard button) all the engine gets very slow, enought fps remains same 60. Any ideas, I have a feeling this because of message takes all frames, but how to fix this?

 

[GhostWolf]

Can't say without seeing code.

 

[Dr Super Good]

But when I send any windows message non stop (like holding keyboard button) all the engine gets very slow, enought fps remains same 60. Any ideas, I have a feeling this because of message takes all frames, but how to fix this?

As a rough guess it is because you are parsing messages in your timing system as a frame? Your grammar is not clear what you mean.

 

[Manoo]

I meant, when I made fps limits (60Hz) with vsync. I got a new problem then I send windows input messages non stop. For instance, holding W keyboard button, all engine gets very very slow, although Fps remains same 60. I could show code but it has lots of class and functions, so I could show some fragments.

For every frame.

if(PeekMessage(&m_message, NULL, 0, 0, PM_REMOVE))
{
TranslateMessage(&m_message);
DispatchMessage(&m_message);
}
if (m_message.message == WM_QUIT) return false;

Frame(); // does all kind of shit, but in the end everything finishes with m_swapChain->Present(1, 0);

m_swapChain->Present(1, 0); //That's the function in frame end to flip buffer and shows new frame
argument 1 stands for vsync


Edit: Maybe i got idea what's the problem, because it's not like all game lags, but mouse starts moving very very slow. So I'm assuming then I hold my keyboard button big part of the message input goes to keyboard and mouse loses frames, so that's why it's looks like lag.

Edit: Yea it solved I made two peekmessage one for mouse other for keyboad and it works quite well.

By the way for unit walking I should make 1/60 position update time yea?

 

[Dr Super Good]

I am pretty sure that is a message queue and not a message slot. You need to iteratively (while loop) remove messages from it per frame until it is empty. What is happening is you remove one message per frame so if you send more messages it fills up and takes several frames to empty.

 

[Luorax]

The teapot's shadow is kinda messed up, most likely because of backface culling. You should:

a) switch to frontface culling for rendering shadows (which would help that teapot, and some also recommend it for dealing with shadow acne, but in reality it's still going to be geometry dependent, so if you all of a sudden decide to import a model which has a hole at its bottom, you'd face the same issue);
b) or disable face culling completely. Yea, that'd effectively double the number of triangles, but it'd at least work properly.

Sorry if this has been brought up before, I did not read through all 3 pages, just came by the game development sub to see what other people are working on and found your thread

It's pretty nice, BTW. You have terrain, physics, and I presume that point light can cast shadow too - my engine is not capable of those things yet (only shadows for spot/directional lights, and they're still very basic). I have everything planned, of course, but maintaining a nice architecture that's easy to work with and implementing everything in a way I know I won't have to thrash in a few days/weeks is pretty challanging :'D

 

[Manoo]

Luorax, ty, fun to see that somebody also trying to create game engines. Jus brace yourself before animations and model format finding it's gona be hardest parts .

 

[Dr Super Good]

Actually the hardest part is multi-threading. I am still unsure how one goes about doing it for games.

 

[Luorax]

Well, yea, I know, I haven't really touched the animations topic, but it should be the next thing once I implement a nice tesselated terrain.

As for multithreading, yea, that's a tough topic. From my experience, you need to protect everything that might be accessed from multiple threads with mutexes, semaphores, and other primitives, but of course it's not gonna be that simple. Carefully choosing tasks for your threads could help immensively, though. But I agree, you can find numerous tutorials about animations and such, whereas multithreading is not that algorithmic, it has to be in perfect sync with your application in order to work properly & efficiently.

 

[Dr Super Good]

From my experience, you need to protect everything that might be accessed from multiple threads with mutexes, semaphores, and other primitives, but of course it's not gonna be that simple.

That just results in a multi threaded process that executes sequentially as if it was single threaded but even slower due to synchronization.

The main approach I have found recommended for game engines is message passing of game state. Where each thread has a complete copy of the game state and then performs changes to that copy with message passing to resolve conflicts and a final synchronization phase which updates the read only copy that each thread has.

 

[edo494]

isnt that quite memory inefficient tho?

 

[Dr Super Good]

isnt that quite memory inefficient tho?

Yeh so? Not like a game uses more than a few 100 MB as a working set anyway.

 

[Kanadaj]

Yeh so? Not like a game uses more than a few 100 MB as a working set anyway.

Well yea, since probably you don't need/want to copy the graphics data itself. Given the type of game, map data might not be the best idea to copy though.

If you can properly separate your game state into modules, duplicating those modules might be more than enough. This way you'll have some modules that run asynchronously, and some that don't.

 

[Dr Super Good]

The graphic data is stored in the GPU. You can keep a copy in main memory but it is not technically part of the active set (only used when the graphics are re-loaded if ever) so has low cost.

If you can properly separate your game state into modules, duplicating those modules might be more than enough. This way you'll have some modules that run asynchronously, and some that don't.

That is the un-recommended approach as it introduces problems such as thread balancing (some modules will need more time to execute than others) and also is poorly scalable (good luck getting more than 2 odd threads out of that).

 

[Kanadaj]

The graphic data is stored in the GPU. You can keep a copy in main memory but it is not technically part of the active set (only used when the graphics are re-loaded if ever) so has low cost.


That is the un-recommended approach as it introduces problems such as thread balancing (some modules will need more time to execute than others) and also is poorly scalable (good luck getting more than 2 odd threads out of that).

When I said modules, I meant completely(mostly) independent ones that have a distinct set of functions which other modules are not reliant on. I know that most of the time this is not possible, but in some cases it works and are recommended. (Take chat for an example - it's basically a separate piece of software inside the game.)

 

[Manoo]

Why do u even need threads (if your talking about <thread> library), I could only use them in game engine for networking, because networking needs as much fps as possible.

 

[GhostWolf]

One usually uses a different thread for networking not for speed, but simply because most of the networking functions block by default.
This behavior can be changed, but it's a lot easier (and CPU friendly) to block.

For other tasks, you would indeed use it for speed.
E.g. for the simplest example, running the rendering code in one thread and the logic code (physics, AI, ...) in another.
It's very, very, very hard to use threads in a smart way that will actually make them efficient, so if I were you, I'd wait with them until I actually needed them.
There are many different complex designs, and every one of them is very hard to understand and implement.

Using the direct approach of locks and mutexes is kind of like using linear code, like DSG mentioned, so it's probably not worth going that way (but always benchmark!).
The more complex paths revolve mainly around not needing said locks to begin with, by designing your code in specific ways.

Regarding the animation - running skeletal animations is actually quite easy if you have basic understanding of vector spaces and how to use matrices. If you seem to have issues, I would be glad to give advice.

 

[Dr Super Good]

It's very, very, very hard to use threads in a smart way that will actually make them efficient, so if I were you, I'd wait with them until I actually needed them.

It is always easier to build a big house from the start than to find the house you built is not big enough and try and add a few extra floors to it. Sure you may be able to add a lot of extra floors but eventually the foundations cannot take the weight and you will end up shorter than you could have gone if you built a tall building from the start.

One usually uses a different thread for networking not for speed, but simply because most of the networking functions block by default.

This is a sort of pseudo interrupt. It indefinitely polls a communication source in a blocking way, being revived whenever something triggers the communication source and then forwarding the results in an engine friendly way (usually a message). Such threads should be designed like interrupts with fast and simple code (do the complex computations in the main threads) and also be given a high scheduling priority (the code needs little scheduled time but needs so in a timely manner to keep latency down). It is as close to an interrupt as you can get in the user level and these threads will probably be the most common thread you make as everything from I/O to network communication uses them.

There are some shortcuts to decrease thread overhead. Some OS provide interfaces that allow you to let a single thread block on multiple communication sources and then get which communication sources triggered the revival of the thread. This sacrifices concurrency for less memory overhead while still keeping the performance benefit of blocking threads. The concurrency is only a problem if most of your execution time is spent dealing with communications (multi gbps servers without TCP offload) since if the communications are processed serially it will make no difference if 99.9% of the time the thread is doing nothing.

 

[GhostWolf]

It is always easier to build a big house from the start than to find the house you built is not big enough and try and add a few extra floors to it. Sure you may be able to add a lot of extra floors but eventually the foundations cannot take the weight and you will end up shorter than you could have gone if you built a tall building from the start

While theoretically true, I never found this to be true for software design, at least not for me.
You can never truly know what you want before you make it, which makes designing it in any good fashion pretty hard.
Once you actually know what features you want, and have at least a basic implementation of them, it will be much easier to think how to design them in a better way.

 

[Dr Super Good]

You can never truly know what you want before you make it, which makes designing it in any good fashion pretty hard.

Yes this is actually what makes the difference between a good software engineer and someone who just graduates with some computer science degree.

Most software is written in this fashion and ends up with a gazillion problems. Look at open source projects like 0AD and Simutrans and you can easily see examples. In 0AD they are having a lot of difficulty adding proper multi-thread support as the engine was never intended to be multi-threaded (although with all honesty it was started before that was a requirement). In Simutrans it is the wonky game engine having problem with multiplayer (was never intended to be multiplayer so has had server client code hacked into it). If both were "over engineered" pieces of software solving both the problems would be easy, but since they are not they will likely never be solved and instead may be considered eventually as part of a complete overhaul (the only way to add such features).

Once you actually know what features you want, and have at least a basic implementation of them, it will be much easier to think how to design them in a better way.

Very true. Version 2 should always be better than version 1 with that regrads as you will learn from your mistakes and continuously improve. However the problem is that this is not cost effective. If you can arrive at version 2 from the start without having to go through version 1 it is worth it. Even if it costs 50% more than making a version 1 it is still a saving over the 100% extra needed to go from version 1 to version 2.

Basically the problem is some things are not simple patches. You cannot patch highly scalable multi-threading into a game engine that was single threaded, especially if it placed sequential coupling in places unnecessarily (forcing an array loop instead of abstracting to a functional array transformation). Sure you might be able to get some tasks multi-threaded, but in the end you will need to rewrite huge amounts of code to get a proper implementation going. It might even be almost the same effort to start from scratch with your previous experience and rewrite the entire thing.

Constant changes to how something is done also introduces problems. A lot of computer science tasks can be done better (XML/HTML is a good example) yet are still used purely because everyone uses them. By migrating from a text based mark-up language that is transmitted to clients to a binary based mark-up language would save many terabytes in data transmission every year worldwide. This comes to saving several tons of CO2 even if you want to be green about it. However people still use and transmit the text formats purely because it is convenient as everyone else does. In fact some of the original developers of the format are annoyed with themselves in retrospect that they did not do it better to start with as even though they tried to introduce a migration path, no one ever used it and kept to the old inefficient stuff.