DarkPlaces engine modification - (a bit) faster shadows

[tomasz.swierczek]

Hello everyone!

I'm new to this forum, so please forgive me if this is not the best way to announce it - I've been looking at DarkPlaces shadowing system, and i've made a patch, that makes use of GPU in Shadow Volumes algorithm - in both zPass and zFail versions I've pushed vertex extrusion to GPU - pretty close to one of TODO's about putting it on the GPU side.

I've also changed some little things in skeletal animation and memory management (MEMPARANOIA or sth, probably known issue on nexuiz forums?), so it runs a bit better.

I've been modifying the latest stable release of DarkPlaces from LordHavoc's site, if someone wishes to know.

Patch is available here:

http://students.mimuw.edu.pl/~ts248384/ ... dows.patch

cheers,

Tom

[FruitieX]

Cool! Too bad the patch didn't work for me with the latest SVN engine though, as it broke all dynamic lights with shadows (no light was rendered)

[Samual]

Great work, but yes FruitieX is right.. It broke real time lighting. A note, you made this on an older SVN distribution. I fixed the patch for latest SVN though (r9146), and it can be found here: http://samual.nexuizninjaz.com/patches/opt_shadows_fromsvn.diff

I do note that my FPS is far higher with shadows than normal, however this could simply be because real time dynamic lighting is disabled..... We can only find out if you fix the lighting problem

Edit: Please develop this on latest SVN -- http://svn.icculus.org/twilight/trunk/darkplaces

[FruitieX]

It seems to break realtime DYNAMIC lights when shadows are enabled, but the realtime world lights work absolutely GREAT! Awesome work!

Edit: apparently shadows are not cast correctly... Try the stormkeep2 map for an example: http://pics.nexuizninjaz.com/images/2u7 ... vmianf.jpg

[LordHavoc]

I'm new to this forum, so please forgive me if this is not the best way to announce it - I've been looking at DarkPlaces shadowing system, and i've made a patch, that makes use of GPU in Shadow Volumes algorithm - in both zPass and zFail versions I've pushed vertex extrusion to GPU - pretty close to one of TODO's about putting it on the GPU side.

While looking through your patch I have a few comments:
You disabled some memset's in the skeletal model animation code - I know these are redundant but in my experience it improves framerate to clear memory before writing it (it forces it to be in CPU cache before it is written).

You disabled MEMPARANOIA defines - I agree with this in principle however divverent has told me that he has fixed the performance problems more recently, so this part is unnecessary now (but in the source you based your patch on, it is probably a good idea).

You are using a texcoord1f array for projection distance which could easily be merged into the vertex3f array to make it a vertex4f array which would read faster on the GPU.

You are still copying vertex data for static meshes when casting shadows - it is possible to preprocess the shadowmesh of the model such that it has a vertex4f array with w=0 and w=1 variants of every vertex, and the CPU only needs to generate a new triangle elements array to produce a shadow volume, leaving the shadowvertex array as a static VBO.

You are not making use of the R_SetupShader_SetPermutation system in gl_rmain.c which manages all shader generation, in particular you should take a look at the function R_SetupGenericShader and make a copy of that which activates your new shader, automatically compiling it on first use (by way of the permutation system).

[tomasz.swierczek]

It seems to break realtime DYNAMIC lights when shadows are enabled, but the realtime world lights work absolutely GREAT! Awesome work!

Edit: apparently shadows are not cast correctly... Try the stormkeep2 map for an example: http://pics.nexuizninjaz.com/images/2u7 ... vmianf.jpg

well... maybe I am wrong (so please correct me) but I can't sew anything bad with this picture - there is that bright lava and health pickup is casting shadows just ok for me - maybe if you could post a comparison with version without a patch?

I had some problems with the range of shadow, though I did not notice errors in direction of shadow.

[tomasz.swierczek]

I'm new to this forum, so please forgive me if this is not the best way to announce it - I've been looking at DarkPlaces shadowing system, and i've made a patch, that makes use of GPU in Shadow Volumes algorithm - in both zPass and zFail versions I've pushed vertex extrusion to GPU - pretty close to one of TODO's about putting it on the GPU side.

While looking through your patch I have a few comments:
You disabled some memset's in the skeletal model animation code - I know these are redundant but in my experience it improves framerate to clear memory before writing it (it forces it to be in CPU cache before it is written).

You disabled MEMPARANOIA defines - I agree with this in principle however divverent has told me that he has fixed the performance problems more recently, so this part is unnecessary now (but in the source you based your patch on, it is probably a good idea).

You are using a texcoord1f array for projection distance which could easily be merged into the vertex3f array to make it a vertex4f array which would read faster on the GPU.

You are still copying vertex data for static meshes when casting shadows - it is possible to preprocess the shadowmesh of the model such that it has a vertex4f array with w=0 and w=1 variants of every vertex, and the CPU only needs to generate a new triangle elements array to produce a shadow volume, leaving the shadowvertex array as a static VBO.

You are not making use of the R_SetupShader_SetPermutation system in gl_rmain.c which manages all shader generation, in particular you should take a look at the function R_SetupGenericShader and make a copy of that which activates your new shader, automatically compiling it on first use (by way of the permutation system).

As far as I know, static meshes cast shadows in a precompiled way - the shadow mesh is being build once when a map is loaded (compiling real time light or sth?), co there is no need to exttrude any vertex on the GPU, as they are only extruded once? Correct me if I'm wrong.

I realize that the shaders - to match engine's conventions - should be done with permutations but... Yes, I must say I do not appreciate this kind of shader code - it does not make things clear in reading and - what is even more important - for profiling/optimizing.

Also, as this is my first time posting a patch here - and there are some difficulties with how it is working (though I didn't see any on my laptop) - I will leave it as it is now, maybe trying to fix light errors if I got to know where they are (previous post, about image)

When everything will be fine, surely this should be done in a permutation-way, but again - only to be compatible with nexuis conventions.

As about performance - it matters on low-end CPUs, even if you don't notice the difference, with more dynamic lights and more triangles it will become more visible - every CPU cicle is worth optimizing, isn't it?

[LordHavoc]

As far as I know, static meshes cast shadows in a precompiled way - the shadow mesh is being build once when a map is loaded (compiling real time light or sth?), co there is no need to exttrude any vertex on the GPU, as they are only extruded once? Correct me if I'm wrong.

The world is statically cast, yes, but most models (pickups for example) are not animated, this allows use of static geometry in more optimized ways than is possible with dynamic geometry, the double vertex float[4] array I mentioned is the usual technique.

I realize that the shaders - to match engine's conventions - should be done with permutations but... Yes, I must say I do not appreciate this kind of shader code - it does not make things clear in reading and - what is even more important - for profiling/optimizing.

Note that I'm not complaining about this, only saying that it would have been easier for you, and ultimately it will have to be adapted to this approach for engine maintenance reasons.

Also, as this is my first time posting a patch here - and there are some difficulties with how it is working (though I didn't see any on my laptop) - I will leave it as it is now, maybe trying to fix light errors if I got to know where they are (previous post, about image)

It's a good idea, although I think it needs more optimizations before it will be a consistent speed boost, some people say it's actually slower.

When everything will be fine, surely this should be done in a permutation-way, but again - only to be compatible with nexuis conventions.

As about performance - it matters on low-end CPUs, even if you don't notice the difference, with more dynamic lights and more triangles it will become more visible - every CPU cicle is worth optimizing, isn't it?

Low-end GPUs are usually paired with low-end CPUs, and the low-end GPUs are much slower than the low-end CPUs at this particular task, it's a matter of baseline performance, the method that works best on the worst computers is used.

Note that shadow optimization can only increase fps by the same amount that turning off shadows does, if this is a large percentage of render time then optimizations are warranted, but I have never seen it be a large percentage in practice.

[tomasz.swierczek]

Low-end GPUs are usually paired with low-end CPUs, and the low-end GPUs are much slower than the low-end CPUs at this particular task, it's a matter of baseline performance, the method that works best on the worst computers is used.

Note that shadow optimization can only increase fps by the same amount that turning off shadows does, if this is a large percentage of render time then optimizations are warranted, but I have never seen it be a large percentage in practice.

You are absolutely right about this - it really is quite a little performance gain, no doubt. Even though, putting those calculations on the GPU makes CPU free to do i.e. more vertex-animation or so - with gprof under my Ubuntu 9.04 I've seen that time spent on shadow generation is about half the time it was. If people are experiencing it being slower - well, you can't fit everyone's combination of CPU and GPU and driver version. Also, writing code for TNT's or even GF 5 today is a bit... well, you name it.

I am currently working on a shooter game on my studies, Its kind a big - one - year project with couple of friends, and I am working on the 3D engine side (for a bit longer, as it is also my area of interest) - you can see sample screenshots here:

https://students.mimuw.edu.pl/~ts248384 ... enSlotter/

(that's University's server address, content from OpenArena)

Doing this kind of graphics and keeping "up to the old hardware" today seems not good road map for me, as at some point people will have to understand that without at least GF 9600 you aren't able to play games using modern technologies, or they'll have to stick to old Quake III Arena.

I will take a look at my shadow code once again this weekend, surely will try to do everything with GPU extrusion using vec4 rather than vec3 with texcoord1f.

I will try to do my best to make this modification useful.

Probably around Monday evening I will post something here (my local time, I'm in Poland)

Thanks for comments

[FruitieX]

Do it from SVN trunk the next time, so it'll be easier to apply and up to date.
svn://svn.icculus.org/twilight/trunk/darkplaces

Also as for the screenshot: http://pics.nexuizninjaz.com/images/o9j ... 2emk4b.jpg

Hopefully makes sense.