I managed to get my voxel renderer working!
Brickmaps were failing me for speed and memory size, so I switched to tree's.
Specifically Sparse Voxel tetrahexacontrees (64tree or contree for short). Contree's are like octrees except they subdivide into 4 per axis instead of 2, or 64 instead of 8.

They are faster and more efficient for games, as there is less up and down tree traversal.
Anyway, here is the current state of the engine: