Hint brushes are a tool to help q3map2 calculate visibility of maps in an efficient way. They cause more splits in the map, meaning more areas for visibility testing. Why are they needed? Why can't q3map2 do it on its own?
Let me explain it in 2D on the following simple map:
We have two big "rooms", separated by two walls that form a hallway that connects the rooms. When we compile this map, q3map2's tries to separate the maps into "clusters" for visibility calculation purposes. For each cluster, it stores which other clusters it can see. How the clusters are separated, usually comes from extending existing structural (not detail) planes on the map:
Here, the red cluster can see the yellow one, as the right part of the red cluster has a line of sight to the left part of the blue one. The renderer uses the list to know what to render - so when the player and the object stand in the indicated places, the object will get drawn even if it can theoretically not be seen by the player! In other words, q3map2's approach to do visibility calculation isn't perfect and needs help from you - the mapper.
What you now do is create a hint plane (that is, a brush that has common/hint on one face and common/skip on all others). The hint plane will be set so it goes exactly through the corners of the walls. In actual mapping, you'd set the plane to very slightly cut into these corners. I'll just show the actual hint face on my example:
When we now run q3map2's BSP and VIS stage again, it will divide the map into parts like this:
Now the red area no longer has a line-of-sight connection to the yellow area, and the object will be skipped from the player's view point! Even when the player is in the blue area, the object won't get rendered. Looks like we've won... but wait, one thing got worse. Let's move the player and the object a little...
From the cyan area, the object will now get rendered as cyan's top left can see blue's top left. This was not possible before:
so the map got WORSE! How to fix it... you guess it: MORE hint planes:
It will now get divided like this:
which, thanks to the added areas, is a better configuration than you got previously. But actually, q3map2 may have already added the portals I manually hinted here. In the engine, r_drawportals can show you how the portals ACTUALLY are generated.
Conclusion:
- a hint brush contains one common/hint plane. All other faces shall be common/skip.
- hint brushes force a subdivision in the BSP tree
- typically, you want to place hint brushes so they slightly cut two corners, and extend to the walls behind the corners; but it usually won't help if you make a hint brush that's just an extension of an existing structural face, as these ALREADY serve the same way
- you may need to also add extra hint brushes to compensate for performance losses when using hint brushes
- how exactly you need to put hint brushes on your map is not always easy to decide. On capturecity for example, I got a huge optimization by putting hint brush pyramids on the rooftops (so what's on the roof isn't rendered when looking from below) and around the car models (so they don't get rendered unless they are REALLY in view). This case here was just a very simple example. I don't think there are existing algorithms to generate a good hint brush configuration, so you have to experiment with it and do it by hand.
- use r_drawportals 1 to actually view how the map is subdivided. Use r_useportalculling 0; r_showtris 1; r_showdisabledepthtest 1 to see what's actually rendered in game, so you see what you can optimize.
- for testing performance, record a demo of jumping a bit around on the map. Play the demo as timedemo with different compiles if your map to see how performance changes when using hint brushes
