Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - olive

Pages: 1 [2] 3 4 ... 10
16
Programming with Blackvoxel / Re: How hard could smoothing be?
« on: November 24, 2014, 01:24:11 am »
First, news on some developpement :

We have implemented the memory pooling for voxel extensions (published some days ago on Github). We implemented it in a way you won't have to change your code.

Also, we'll add your good optimisation for water voxels.

Did some research on implementing glCallLists in my stuff; and learned they have been depricated... in favor of vertex buffer objects... which kind of makes sense, since you can create vertex buffers in threads asynchronous to the display... where glCallLists have to be built on the GL thread context...

Keep in mind that these API where designed mainly for vector games made mostly of big mesh objects.

So, VBO are better with mesh of traditionnal games than with voxels rendering(at least with a simple approach).

And display lists have the advantage of better compatibility. This stuff run nearly everywhere.

Yes, that's deprecated like all old OpenGL stuff. Not a problem in practice.

That said, we think it is possible to do better and faster rendering using some advanced OpenGL technics.

That's simply a matter of time and priority to do it.

Quote
so shaders and display buffer building as another pluggable renderer :)

In Blackvoxel, Shaders wasn't used because the "old computer looking" rendering style we wanted simply didn't needed shaders.

But as we did the textures for the game, we also realized that it could be nice to have some effects like bump mapping.

And other kinds of light effects could make some interesting and original things.

So, that's an open way for the future.

Quote
I find that the resonable relation would be building buffers per shader per image source... or per block type; with shaders would think the 'default' blocks could be computation shaders based on a distance from poly edge and a color scalar...

and those built per voxel sector ....

A "simple" VBO approach with voxel have a big drawback : too much draw calls.

The challenge for really gaining performances with VBO is to reduce the number of draw calls.

VBO were obviously designed to render big static meshes and weren't designed with voxel in mind.

But some OpenGL technics could be used to reduce the number of draw calls. The problem will be to ensure that the implementation would work with most machines.

So there is ways to explore.

We must keep in mind that there is interest to make a new renderer only if there is real gain.

Quote
there's also no multi-indexed buffers... like normals in this world I'd only need a buffer of 6 points, if I could have a index buffer for texture, index vertex and index normal independantly... but indirect uses the same index map for vertex/texture and normal so they all have to be the same length...

http://stackoverflow.com/questions/11148567/rendering-meshes-with-multiple-indices for instance indicates it can be done, but it's more work than benefit.

Yes, as voxel are really particular cases, that's never simple to figure out what could be good or bad without testing.

That can become "funny" when you find that some technics work well with some GPU and less with others. And it could also depend on the operating system.

As an example, the actual renderer is faster with comparable "GPU power" with nVidia and Intel than with ATI/AMD.

The Blackvoxel Team

17
Programming with Blackvoxel / Re: Custom VoxelTypes
« on: November 20, 2014, 03:37:58 am »


Hum, I wouldn’t recommend doing this.

The way the storage was made is for memory bandwidth efficiency.

Packing these data will lead to waste an important amount of memory bandwidth.

This is a weird side effect of how modern processors access memory with prefetch mechanisms and important bus width : processors can't fetch only a byte or short.

Sectors are 16384 indexes... voxel types are 2 bytes... so 32k or 8 4k pages.
TempInfos is another short for block temperature; this should probably be in an infos since in general blocks don't change behavior based on tempurature....
OtherInfos is 4/8 bytes to reference voxel instance data; VoxelExtension*...

all together 341 + 4 bytes to 512 per page..........

total is 32k + 32k + 64k(32 bit)/128k(64 bit)  (I've been running 64 bit build, because I have more sectors of smaller voxels usually)

128k/192k ..  32 or 48 pages (4096 bytes) for a sector...

-------------------
I made a branch at that point... I wasn't sure how much of a performance hit it would be... I'm aware of cache... *see below*

I guess given the defined scope of the project
for the renderer which only needs voxel type, then it needs to only fill 8 pages instead of 48... given no custom render for custom voxels... (color shading)... although maybe renderer uses TempInfos to determine texture... so 16 pages; even if blocks are rarely shaded, so temp isn't checked always... could be less...

voxel reactor uses (tempurature?  does temp get modified except by time and water?  Does it dissapate?  Each voxelType have a thermal coefficient thing? :)  ) and OtherInfos... unless a highly static level is used with non extended voxels... with an active environment, OtherInfo will definatly be cached... so no 8+32.. 40 pages loaded without tempurature... so there's no real savings at this level...

Feature: at a voxel-is-center-of-universe view, if I'm given an offset to my data, to get near voxel's (extended)data... I dunno I guess I still need the sector reference...

so ya... a highly static map benefits from only needing 32k loaded for working with a sector (plus another page for the sector, and another for the voxel type manager) ... but then again in a 1M cache 32 sectors can potentially be loaded... so that's only a few more than the block of 9 around the player at any point... so it still has to scroll through the memory...

....
in a sequence of processing blocks, only 4-6 pages are used ... (the center, left/right) (above and below) (forward and backward) , especially on boundaries of sectors... I dunno; the working set is still the same.... this is tripled with the current scheme... because mirror otherinfo, tempinfo pages may be used... 12-18 working set pages...

it resulted in more efficient assembly because there was only one pointer and other offsets computed from that...in the other place I was looking at I do remember noting that...

It may be arguably better if they were allocated all together... and put all voxeltypes and all tempinfos and all otherinfos together instead of interlaced, but with a single pointer...

What is lost is gained in other ways.... and I don't notice a particular performance hit from this change...

--------------


The problem appears more clearly when looking at the data needed by each part of Blackvoxel involved in massive processing :

- The renderer doesn't use anything else than VoxelType and FaceCulling info. It will be reduced in the future.

- The MVI mostly use VoxelType data. That may sound surprising, but in fact, most scanned voxels aren't active. Even on active voxels, only few are using Extensions. (Even if it may change depending on voxel nature).

- On the sector loader, all data are needed. But each are processed independently. So here again, individual fields would be betters.

- Some temperature processing is planned to be implemented in the future. Thats why the TempInfo table is here. It will probably use Tempinfo and VoxelType. The cycle would be slower than MVI. Not sure when we will do it.

- Some pressure informations could be added in the future when most processors will have at least 8 true cores and according memory bandwidth.

The idea of accessing different data with an unique pointer could be interesting. And we can certainly implement it cleanly with limited changes. But I could not tell you what would be the order of magnitude of the gains(because of the caches). And not yet thinked how we could keep Valgrind detecting MVI memory error.

Quote
and I don't notice a particular performance hit from this change...

This does not surprise me at all. :)

Some Core i7 have more than 30Gb/s of memory bandwidth.

The core 2 quad is much around 10Gb/s.

Core i3 and recent AMD are in between.

And and old intel atom (Single channel DDR2) in a netbook is around 4Gb/s. (Yes, Blackvoxel can run on it with some setting adjustments).

(Note these numbers can vary slightly depending on sources and processor generations).

So, the idea is to keep memory bandwidth in order to run on modest machines, but also for keeping room for future stuff : I'm affraid moore law is now dead.

What is weird with Memory bandwidth is it's shared nature. All cores share the memory controller. So using too much memory bandwidth on one core can reduce efficiency on other cores.

We often realized during Blackvoxel tuning that some changes could react differently depending on the hardware.

Not easy to minimize these side effects. That' why we ran a lot of test and had to correct a lot of things.


Quote
Once upon a time I got to play with/use a logic analyzer that decoded the pins interfacing from 386 to motherboard... got to see read/write memory to fetch instruction blocks... so there's the clearing of cache on calls and jumps backward... jumps forward are better handled by setting a ignore flag and processing the prefetch cache anyway... kinda like how arm handles short jumps.... but anyway... it's just much wider now... and burst mode...

That's recall me the good old time when we worked with 68000, wire wrapping boards, TTL's circuit logic, Eproms, PIA, ACIA and all these amazing stuff. We did our own computer.

I think you are a good programmer as you have learned how the computer's hardware is working under the hood. That's why you can understand and do optimisations. :)

The Blackvoxel Team

18
Programming with Blackvoxel / Re: Custom VoxelTypes
« on: November 18, 2014, 02:31:29 am »
...
also; in consideration of making a container for voxels, needed to know what a voxel was, so I reimplemented Sector->Data,OtherInfos,TempInfos as VoxelData *Data;

https://github.com/d3x0r/Blackvoxel/blob/group_voxel_datas/src/ZVoxelSector.h#L203

Hum, I wouldn’t recommend doing this.

The way the storage was made is for memory bandwidth efficiency.

Packing these data will lead to waste an important amount of memory bandwidth.

This is a weird side effect of how modern processors access memory with prefetch mechanisms and important bus width : processors can't fetch only a byte or short.

In practice, in Blackvoxel, the different informations about a voxel are rarely used at the same time. Most massive processing operations are mostly done on one unique field.

With packed informations, the processor will fetch also some adjacent data of other fields. These data won't be used.

So, for this case,  it's better to keep each field in it's own array.  ;)

The Blackvoxel Team

19
Programming with Blackvoxel / Re: Custom VoxelTypes
« on: November 18, 2014, 02:25:06 am »
A large performance increase can be made in the water processor...

Code: [Select]
// original
for(i=0,j=4,vCount=0,WaveCount=0;i<4;i++,j++)
                              {
                                cx = x+bft[i].x ; cy = y+bft[i].y ; cz = z+bft[i].z ; SecondaryOffset[i] = If_x[cx]+If_y[cy]+If_z[cz];St[i] = SectorTable[ Of_x[cx] + Of_y[cy] + Of_z[cz] ]; Vp[i] = &St[i]->Data[ SecondaryOffset[i] ].Data;
                                cx = x+bft[j].x ; cy = y+bft[j].y ; cz = z+bft[j].z ; SecondaryOffset[j] = If_x[cx]+If_y[cy]+If_z[cz];St[j] = SectorTable[ Of_x[cx] + Of_y[cy] + Of_z[cz] ]; Vp[j] = &St[j]->Data[ SecondaryOffset[j] ].Data;
                                if (VoxelTypeManager->VoxelTable[*Vp[i]]->Is_CanBeReplacedBy_Water && VoxelTypeManager->VoxelTable[*Vp[j]]->Is_CanBeReplacedBy_Water) {vCount++; DirEn[i]=true;}
                                else DirEn[i]=false;
                                if (VoxelTypeManager->VoxelTable[*Vp[i]]->Is_CanBeReplacedBy_Water) {WaveCount++;WaveDirEn[i] = true;}
else {WaveDirEn[i] = false;}
                              }

// modified; move the test for (*Vp ) as a common point, then Vp[j] doesn't have to be computed, or tested, because it's an and condition with Vp
Code: [Select]

for(i=0,j=4,vCount=0,WaveCount=0;i<4;i++,j++)
                              {
                                cx = x+bft[i].x ; cy = y+bft[i].y ; cz = z+bft[i].z ; SecondaryOffset[i] = If_x[cx]+If_y[cy]+If_z[cz];St[i] = SectorTable[ Of_x[cx] + Of_y[cy] + Of_z[cz] ]; Vp[i] = &St[i]->Data[ SecondaryOffset[i] ].Data;
                                if (VoxelTypeManager->VoxelTable[*Vp[i]]->Is_CanBeReplacedBy_Water )
                                {
                                  cx = x+bft[j].x ; cy = y+bft[j].y ; cz = z+bft[j].z ; SecondaryOffset[j] = If_x[cx]+If_y[cy]+If_z[cz];St[j] = SectorTable[ Of_x[cx] + Of_y[cy] + Of_z[cz] ]; Vp[j] = &St[j]->Data[ SecondaryOffset[j] ].Data;
                                  if( VoxelTypeManager->VoxelTable[*Vp[j]]->Is_CanBeReplacedBy_Water) {vCount++; DirEn[i]=true;}
                                  else DirEn[i]=false;
                                  {WaveCount++;WaveDirEn[i] = true;}  // bad braces
                                }
                                else  {WaveDirEn[i] = false; DirEn[i]=false;}
                              }

50% processing time reduction

Thanks for that!

It seems a very good improvement. :)

The Blackvoxel Team

20
Programming with Blackvoxel / Re: Custom VoxelTypes
« on: November 15, 2014, 02:56:36 am »
Voxel extensions don't come from pooled memory.

Yes, this must be improved.

At origin, Extensions where used only for some non massively used machines.

All "massive" active stuff uses no extension or only 32 bits of the otherinfo as a direct storage (this is possible).

But semi-massive uses may need extensions. So, the actual system must be improved.

Quote
Moving a voxel from one place to another that has extension causes new and delete operators; and the new voxel is not the same as the prior... (maybe there's a copy operator)...

but also there is no 'move voxel to' or 'swap voxel with' which could move the reference to the original voxelExtension.

So motion is (set new voxel, create extension ) (clear old voxel, delete old extension)

These functions exists in Blackvoxel.

That's the ZWorld::MoveVoxel_Sm() and ZWorld::ExchangeVoxels()

With these functions, extensions are simply moved.

But I'm doing a synthetic simple API for easier active voxels creation. It's now published it on Github(but still unfinished).  :)

The Blackvoxel Team

21
Programming with Blackvoxel / Re: How hard could smoothing be?
« on: November 12, 2014, 11:54:37 pm »
This is a code snippet that's mostly standalone; use of libpng to read/write images.  1/2 is read 1/2 is write.
basically my image stucture is
struct image {
  int x, y;
  unsigned int w, h;
  byte[4] *image; // color data
}

https://code.google.com/p/c-system-abstraction-component-gui/source/browse/src/imglib/pngimage.c

several editors on windows support png and the alpha transparent layer... none (few); probably you use gimp?  support 32 bit alpha saving.  I Have a command line utility that converts 24 to 32; but makes the alpha channel opaque.

png is like bmp that it is lossless... but it is a zip compression per channel...  works good for RLE encodable images basically... or mostly constant images... but does have 1 byte alpha channel support.

can google 'sample libpng read' which is what my code was based on... which is mostly copy and pastable....

----------
On image loading; I open the file memory mapped and pass the pointer to that memory to a routine that trys passing it to several loader routines, which then result with an image... basically if( !ZBitmapImage.PNGLoad(file) ) if( !ZBitmapImage.BMPLoad( file ) ) ... if (!... JPGLoad() ) ...

can just pass the open file and rewind it between....
SDL has an image loading library...
FreeImage is a LARGE library that loads just about everything... but it's like 2M .. png is a few hundred K
-----------

Gimp is what we recommend for texture working with Blackvoxel. That's a very powerfull program.

But any other major image editor will do the job (and provide support BMP32.)

You are right to say uncompressed BMP take more space on the disk. But is it really a problem ?

At this time, the place taken by blackvoxel on hard disk stay low compared to many games. As images are mostly textures, we'll run out of GPU RAM well before hard disk space become a problem.

And for web distribution, packages are compressed anyway.

After all, image loading isn't a central functionnality in a game. That's common to have limited support of texture formats.

There is pro and cons for compressed formats. In one hand, we'll gain space, on other, we'll add a little delay to startup time.

But more complex format mean also more compatibility problems.

And image library are typicaly the pieces of code that need evolution and maintenance over time.

Some months ago, Gimp changed to a new BMP format revision. And we had to update the image loader.

But at last, as we written it, we were able to fix the problem very quickly.

So, as there is pro and cons in the story, we'll have to think about that idea.

The Blackvoxel Team

22
Programming with Blackvoxel / Re: Custom VoxelTypes
« on: November 12, 2014, 11:50:14 pm »

Interesting ideas. We dreamed to make some agriculture and food in blackvoxel since a long time.  But, be careful of animals that could be present in some other voxel games.
Maybe you could make it as some extra terrestrial animals. If you keep reference to some known animals, they could be very different of what we know. In blackvoxel, the idea is that one should be surprised from what he is expecting (when possible, of course).
I agree; was going to make some cyber-space names for them.. but fell back to my idea of foxes and rabbits .... predator/prey and make the foliage be like data nodes... but the concept of growth didn't apply so much... am open to ideas ... maybe 'bit scav'/'virus' and 'antivirus'... so a virus consumes data and anti-virus consumes viruses...

general properties of an 'animal'
health/data size
damage/data consumption
level/entities consumed  (improve damage application effectiveness)
propagation_threshold - if health > threshold spawn new X
age maybe - builtin obsolescence
scent?

----------
Something I've been considering; making transparent voxels have a base voxel extension that can store other voxels... fish in water; animals in plants, animals sharing spots... 1) animals could attack horizontally and just require nearness... 2) animals could be 'on top of' plants... 3) could be a meta voxel type 'fish in water' that behaves like water... but...
Something I'd like to implement is 'smell' that is empty (air) voxels will be like a textureless voxel that's not zero; and contain information about aromas in the air...

We encourage you in your searching.  :)

Quote
normally the player isn't a voxel; so voxels that are player-like should also respect 'can player pass through' ... but then they would be in the voxel.... and since there's only a current reprensentation of a single voxel type in any voxel, it could be an interface implemented on transparent voxels that allow being 'in' the voxel.  Can do a multi-layered render of the voxel at some point to represent the contained voxels... thoughts?  I will assert that from previous experience, I think you'd prefer to keep to a purist approach, and animals will just hop from ground to the top of foliage, and animals can attack animals nearby disregarding height differences.. and then I can't really populate air ... because eventually aroma voxels will exist above fertile land and prevent growth....

Yep, there is the paradigm to keep the rendering of all voxels the same and strictly cubic. That's a choice that will make sense in the future.
That's not something we consider we couldn’t change at all. There is pro and cons in the two ways.
But if we found there is better advantages to change, we'll do it.

In some ways, the "smells" are gas, so like the yellow gas.
Voxel Animals should'nt use the flag 'can player pass through'. When moving, they could either discard "gas" voxels or exchange their positions.
The water is doing that : When moving, a block of water exchange it's position with the gas located in the destination location.
For vegetation, they could simply delete gas voxels in growing over.

Quote
depends on your allocer; most malloc will return the same block as was last freed... so allocating short term storage doesn't fragment.. yes returning into a caller-owned struct solves it.... was more concerned with keeping the function signature the same for quickness; and when not deleted was what was causing my memory error :)  ended up removing that.

well... memory deallocated is defragmented and allowed for reuse; so the things you throw away are collected, so there is garbage collection; just not automagic object reference increment/decrements... and garbage on the stack is collected...

And even so; you still have to intentionally throw away things like Datatables because of the circular references they won't auto implode.... so you should never count on things just going away.

free works kinda like ZMonoSizeMemoryPool::Free such that when released, it's immediately reused for new mallocs...
I find it's better to do a slight more work, and do 2 comparisons in the free list to order free blocks from least memory address to most; further reducing fragmentation... also if the allocer has a minimum memory block size there's less chance of having blocks that are not big enough laying around; my MUD terminal (dekware) improved greatly by setting minimum alloc at 60 bytes... at not that much increase in memory usage.
Also block types that are tiny like binary tree nodes come from a pool of structs so there's only a few allocs from physical memory.

so ya; I saw the techniques and recognized their purpose :)

In fact, the problem is like most optimisation problems : there is a limit to what a general approach can do.

As clever as memory manager could become on some systems, these algorithms remains tuned for general case.

That's why the best system memory manager would remain not optimal for all particular cases and particularly for some cases in video games.

There is also intrinsic limitation with C++ : a system memory manager can't defragment two blocs if you keep one in between. Because there is no reference tracking in C++, there is no way to move that memory.

Also, any defragmenting operation means system will take some CPU time to do it.

In fact, most system memory manager use complicated strategy. On the source code of the glibc, we can read that the strategy depends on the block size. The algorithm is explained in the source code. It doesn't fit well all the cases.

That's also mean something we want to avoid : heuristic approach in these algorithms could lead to unpredictable variability between systems and different memory manager.

Garbage collectors do more than simple memory management, they are tracking unused blocks in order to avoid the obligation of explicit memory freeing.

In Blackvoxel, we doesn't ban system memory manager : we use it for all is general, long term and non intensive memory use.

There is two custom allocators in Blackvoxel and there are used depending on the needs. There is one for specific size memory management and one other for more variable size.

Hmm... does the mem pool really help?  Is new and delete not really based on malloc/free?  Glibc usually collects garbage pretty well... I had my own memory allocator that was in the core of my library; still do, just hard to initialize a dynamic program without having initialized memory; and sometimes core debugging becomes impossible....

In my library I made a set allocator which is a linked list of nodes that have a bitmap mask of used mono-sized nodes... also keep a counter of used nodes so can just check if all used, otherwise scan the bits 32 at a time until it's not ~0 ... otherwise allocates a new block of nodes... so all nodes are continuous in memory ...

even if you're recycling previously allocated things, they're still jumbled all over memory....

Yep, the custom allocators in Blackvoxel helps a lot. For both fragmentation and speed.

These are really blazing fast and doesn't cause any fragmentation growing, even with the most intensive use.

As you guessed, it uses underlying system allocation. But the subtlety is that the memory is never returned to the system, but always recycled. So after a little startup time, no system allocation calls are made.

Of course, this approach have also drawbacks. It's not suited for everything, not even in Blackvoxel.

So, memory management is another case of the optimisation problem leading to the best compromise for each particular needs.

Quote
even if you're recycling previously allocated things, they're still jumbled all over memory....

Our goal with fragmentation isn't to avoid it completely or to make things perfect, but rather to avoid weird degenerative problems after some time of gaming.

The other goals are to get it the fastest as it is possible and limit variations between systems.

With our custom memory managers, even with billions of calls, the fragmentation doesn't grow over time and recycling is very fast. That's all what we need... :)

The Blackvoxel Team

23
Programming with Blackvoxel / Re: MSVC or non-GCC Porting
« on: November 10, 2014, 09:18:50 pm »
That's very interesting informations. And it will be useful for the mac os port.

We never had a mac. But as a BSD derivative, we suppose it shouldn’t be too far from linux for system base application calls.

The Blackvoxel Team

24
Programming with Blackvoxel / Re: Custom VoxelTypes
« on: November 09, 2014, 11:19:41 pm »
Success!
I have a block called 'fertile ground' that in a second or 2 from creation grows 'low foliage' aka food.
food will eventually age and become medium and tall; which will be unusable by the first animals... a block called rabbit and a block called fox....

Interesting ideas. We dreamed to make some agriculture and food in blackvoxel since a long time.  But, be careful of animals that could be present in some other voxel games.
Maybe you could make it as some extra terrestrial animals. If you keep reference to some known animals, they could be very different of what we know. In blackvoxel, the idea is that one should be surprised from what he is expecting (when possible, of course).

Quote
------------
I implemented a type called ZVoxelRef
Code: [Select]
#ifndef ZVOXEL_REF_DEFINED
#define ZVOXEL_REF_DEFINED

#include "ZVoxelSector.h"
#include "../ZVoxelExtension.h"
class ZVoxelWorld;
class ZVoxelRef
{
public:
    ZVoxelSector * Sector;
    int Offset;
    int x, y, z;
    UShort VoxelType;
    ZVoxelWorld *World;
    ZVoxelTypeManager *VoxelTypeManager;
    ZVoxelExtension *VoxelExtension;
    ZVoxelRef( ZVoxelWorld *world, ZVoxelTypeManager *vtm, long x = 0, long y = 0, long z = 0, ZVoxelSector *Sector=NULL, UShort VoxelType = 0, int offset = 0 )
    {
        this->x = x;
        this->y = y;
        this->z = z;
        this->Sector = Sector;
        this->Offset = offset;
        this->World = world;
        this->VoxelType = VoxelType;
        VoxelTypeManager = vtm;
    }
    static int ForEachVoxel( ZVoxelWorld * World, ZVoxelRef *v1, ZVoxelRef *v2, int (*f)(ZVoxelRef *v) );

};

#endif

that I for implemented for the result of raycast at maxiter... before I changed method to render selection cube in space in front of player...


// modified GetVoxel which just returned the UShort from a world coordinate
//  to return the World, Sector, relative x,y,z within the sector, and the UShort type
// the sector has it's x,y,z scaled world coordinate... so the absolute coord is knowable from the reference.
... reviewing this a little... the ref could be used for all the temp variables themselves... this actually returns a copy of the world coordinate, React uses it as x,y,z within the sector...
Code: [Select]
inline ZVoxelRef *ZVoxelWorld::GetVoxelRef(Long x, Long y, Long z)
{
  ZVoxelSector * Sector;
  Long Offset;

  Sector = FindSector( x>>ZVOXELBLOCSHIFT_X , y>>ZVOXELBLOCSHIFT_Y , z>>ZVOXELBLOCSHIFT_Z );

  if (!Sector) return NULL;

  Offset =  (y & ZVOXELBLOCMASK_Y)
         + ((x & ZVOXELBLOCMASK_X) <<  ZVOXELBLOCSHIFT_Y )
         + ((z & ZVOXELBLOCMASK_Z) << (ZVOXELBLOCSHIFT_Y + ZVOXELBLOCSHIFT_X));

  return new ZVoxelRef( this, VoxelTypeManager, x, y, z, Sector, Sector->Data[Offset], Offset );
}

inline UShort ZVoxelWorld::GetVoxel(Long x, Long y, Long z)
{
  ZVoxelSector * Sector;
  Long Offset;

  Sector = FindSector( x>>ZVOXELBLOCSHIFT_X , y>>ZVOXELBLOCSHIFT_Y , z>>ZVOXELBLOCSHIFT_Z );

  if (!Sector) return(-1);

  Offset =  (y & ZVOXELBLOCMASK_Y)
         + ((x & ZVOXELBLOCMASK_X) <<  ZVOXELBLOCSHIFT_Y )
         + ((z & ZVOXELBLOCMASK_Z) << (ZVOXELBLOCSHIFT_Y + ZVOXELBLOCSHIFT_X));

  return(Sector->Data[Offset]);
}


There is a function like this in ZVoxelWorld::GetVoxelLocation() and the VoxelLocation structure returned provide most of the informations about a Voxel. I'll explain how to use it.

There is a particularity in C++ games : the  operator "new" should be avoided in "game loops" whenever possible because use of the system heap memory manager is always slow (and could lead to massive use causing memory fragmentation problems).
There is no garbage collector like in C# or Java, so that's very different in programming style.

We used some ways to avoid problem:

In Blackvoxel, we use a custom "recycling" memory manager for some classes which absolutely needed "on the fly" memory allocation.

But the simpler and best way in most case is to declare a "stack variable" structure in the caller and pass it to the a function that will fill it. That's what GetVoxelLocation() is doing.

Quote
-------------
Used this in VoxelReactor to call React virtual method in voxel table...
Code: [Select]
      Long RSx = Sector->Pos_x << ZVOXELBLOCSHIFT_X;
      Long RSy = Sector->Pos_y << ZVOXELBLOCSHIFT_Y;
      Long RSz = Sector->Pos_z << ZVOXELBLOCSHIFT_Z;
      ZVoxelRef ref(World,VoxelTypeManager, 0,0,0,Sector );
    // z, x, y should just be ref.x, ref.y, ref.z, ... redundant copy
      for (z = 0; z < ZVOXELBLOCSIZE_Z; z++)
        for (x = 0; x < ZVOXELBLOCSIZE_X; x++)
          for (y = 0; y < ZVOXELBLOCSIZE_Y; y++)
          {
            VoxelType = *(VoxelP);
            if (ActiveTable->Get(VoxelType))
            {
              if (!Sector->ModifTracker.Get( MainOffset ) ) // If voxel is already processed, don't process it once more in the same cycle.
              {
                switch(VoxelType)
                {
                default:
                      ref.x = x; ref.y = y; ref.z = z;
                      ref.Offset = VoxelP - DisplayData;
                      ref.VoxelType = VoxelType;

                    IsActiveVoxels = true;
                    ref.VoxelExtension = (ZVoxelExtension*)Sector->OtherInfos[MainOffset];
                    // not sure what St is ...
 //St[i]->ModifTracker.Set(SecondaryOffset[i]);
 VoxelTypeManager->VoxelTable[VoxelType]->React( ref, LastLoopTime);
                    break;

As we said, an extension will be added to support this kind of stuff. Be patient as we'll finish it in the next few days (we understand you need it).

We need to think it well and not precipitate because there is some important considerations to make it useable easily.

Quote
// not sure what St is ...

St is a table holding sector pointers for the neighbor points. The problem with MVI is accessing quickly neighbor voxels that could be in another sectors than the one where the voxel is located.
The modif tracker is a system recalling "processed" voxels in order to avoid moving these again in the same MVI cycle.

The Blackvoxel Team

25
Programming with Blackvoxel / Re: MSVC or non-GCC Porting
« on: November 09, 2014, 10:56:47 pm »
...
Code: [Select]
// in src\z\ZMemPool_Optimized.cpp
#ifdef __GCC__
      if (__sync_bool_compare_and_swap(&MemTable[BitPosition],NewBlock,NewBlock->Next))
#else
        if( (MemTable[BitPosition] == NewBlock) ? (MemTable[BitPosition]=NewBlock->Next),1:0)
#endif

which does the same job, but without lock.....

During development, we forgotten the atomic instructions here. It seemed to work. But when trying to track weird and inexplicable crash, we finally found it was this.

So, it won't work correctly without the atomic instruction at this particular place.

Because the comparison and the assignation are not atomic, something can happens between. The tread can be interrupted or another thread can execute the same code at the same time.

As unlikely it might sound statistically, it happens more often one could believe.

The weird about this kind of issue is that the occurrence may vary between machines from rare to frequent depending on a lot of parameters involving scheduler algorithm, number of cores.

Stabilizing the Blackvoxel core took a long time and involved many tests on a lot of machines, graphic cards (some remains to do...). Even with the powerful tools like valgrind we have on linux, some issues was long to track.

Sorry, should be Pointer not 32/64 bit switch...
needed to reverse b and c params and add test if result is the assigned value... not exactly the intrinsic.
Code: [Select]
#ifndef __GNUC__
#    define __sync_bool_compare_and_swap(a,b,c) (InterlockedCompareExchangePointer((void*volatile*)a,c,b), (*a)==(c))
#endif

We suggest you to change the test on define because testing for gcc could trigger on some wrong case, as an example on a future MacOS port with LLVM toolchain.

So we recommend applying compiler/platform specific patch with testing for compiler and (when pertinent) to platform.

But our goal is primarily to support well Blackvoxel on one "official" compiler.

As we are only two in the team to do everything, we prefer to focus on doing one thing very well rather than several poorly.  :)

The Blackvoxel Team

26
Programming with Blackvoxel / Re: How hard could smoothing be?
« on: November 05, 2014, 03:22:27 am »
The other example of 'poor relationships' I ran into is C# has a type called DataTable, which is a representation of a SQL table, with dynamic columns with types and rows, etc.  But datatables contain columns and rows but all columns know the datatable they are in, and all rows know the datatable, also datatables can be contained in datasets which is a group of datatables and adds the foriegn key relationships between tables.  So from any row you can get to any other row in any other table that row is remotely related to... so there's sometimes merits of having... say worlds know all sectors, but sectors know their world, and hence their renderer... or something.

True. It's a good idea.

compare_swap_and_xchg and InterlockedExchange are similar; other than the function results...
I see; that is one thing I had an issue with when porting; think I made it thread unsafe...

Yes, thread unsafe code can easily create weird instabilities. The problem with a performance oriented program is that we are trying to avoid any form of lock whenever possible. The difficulty is to avoid missing the cases where it won't work.

Quote
Here's something I was playing with - surround-o-vision

http://youtu.be/R2izTRpP2kM

each display is actually independant, and should build its display list appropriate for its view direction, but right now, everything is added to every display list, and 6 windows shown each rendering pass... at very low frame rates there is frame tear between the displays....

It's fun. We see that it can do many things.  :)

Quote
More optimal would be to target 3 monitor, and just show forward left and right.. a single display stretched across 3 displays is not right, and perspective gets distorted badly.

For doing it right, it would need entering some parameters for each screen. As far as I know, only some flight simulators support some kind of advanced multi screen.

Quote
So ya I'll turn my attention more to voxel based operations... I dunno just feel I'm missing something... like I saw the voxel engine

Yep, the most interesting part of Blackvoxel is on the MVI side...

That's the core of the game interest.

The rendering engine is designed mainly to be fast and efficient in order to serve MVI.  :)

The Blackvoxel team

27
Programming with Blackvoxel / Re: How hard could smoothing be?
« on: November 03, 2014, 02:26:44 am »
True enough.... but it's at least 16 calls that function makes itself... to emit the 4 points with 4 texture coords, setup primitive etc... and that's 1 face of 1 voxel... so figure perfect flatness... 256*16 ... so 1 in 4096 increase in function calls is not significant  :)  Edit: I got that a little wrong... If something NEEDs to be inlined... there's always Define's   :)  Conveted some of the emitting to defines so I could have consistant texture coord references ... like point 0, face 0,1,2 from it has a texture coord defined for it....

Ther is 14 function call per face, so 84 calls for all faces. So, roughly 1/100 of function calls. But lot of parameters, so much like 1/50 of the calls. Sure, it's not huge.
The idea is more to code critical pieces of code in the spirit of doing the maximum optimisations.

Defined code are often used for code snippet. But not very handy for large portions of code. Have also some drawbacks.

We also studied the idea of ​​meta programming. But there are also disadvantages.

Quote
If a compiler can't do the job 'right' then don't use it... LCC for instance; wicked fast, but linker cripples real usage.  Wish I could play with icc (intel) but they're so closed for some reason.

Doing the job "right" for a compiler could also mean to take into account a lot of contradictory considerations because modern CPU are very complex.

As an example, a compiler must avoid over-inlining big pieces of code in every portion of non critical code because increasing all the code too much could lead to bad cache usage and worse performances.

But an heavily used little piece of code is a very special case that could suffer of this general behaviour.

The problem is that even the best compiler have no real mean to know if a code portion is critical. It's decisions are based on "general considerations". That's optimal for general code, but might not be the best for special cases. That's why the need to help compiler sometime.

Of course, there is a lot of other mean to help compiler in such cases. Some compilers have a lot of flags for tuning. There is special non standard instructions, like the "always_inline" of gcc.

Profiling is also used to help compiler to know what pieces of code are heavily used and need to change it's rules.

But all these mean have big drawbacks. Profiling add a complicated phase, special compiler flags and instructions aren't working across compilers.

Here again, whatever way mean a price to pay.

Quote
Well Again, quaternions shouldn't be 'used' ... since to be applied for transformation it needs to convert to a 3x3 matrix... might as well just keep the matrix.  linear interpolation only counts for follow cams from arbitrary points to other arbitrary points... but mostly a follow cam will be a relative translation of an existing view camera and not really require quaternion representation either... and the 4 coordinates don't translate into something comprehendable... so expressing any constant quat-vec4's is not really possible... just to retrieve from an existing rotation 3x3 state.... well i,j,k,l vectors work; but once they get combined multiple coordinates change at once for a simple rotation.

I think it's a good advice. We'll certainly follow it.

Quote
But; I guess I'm really looking at the wrong scale of things...
I know you mentioned some things already... but how do I really add a new block behavior?  Maybe creation of a block should create a voxel body  :)  A voxel brontasaurus or something... so a simple block spawns all of the others when created... and generate motion in voxel units... was considering a ground displacement for footprints ... so many thoughts.

Yep, that's exactly the principles of how things are working with MVI.

In Blackvoxel, a voxel can have a program defining it's behavior, that could be called a voxel-shader.

In this program, you can manipulate other voxels around : make animations, chemical reactions, transformations.... nearly everything is possible.

Yes there is amazing things to do with this.

Making a "block behavior" means simply add a the behavior code in ZVoxelReactor.cpp switch(case is the voxeltype). Then add the BvProp_Active=1 statement in the voxelinfo corresponding file.

Like I said in a post some days ago, there is also some other classes to add if your voxel need to have it's own data storage for internal state, inventory or whatever you want.

We think we'll make this stuff simpler in the future by adding an "On_Execution" kind of function in ZVoxelType. This is less efficient, but only massively used voxels needs high performances.

Quote
The bomb in black forest uses world distance for detection and transformation into red liquid distance.... the x64 factor for reduced world size caused much too many red blocks to be created  :)

Also attempted to simply add more threads to process things; wasn't as simply extendable as I would have hoped  :)
InterlockedExchange()/*windows, not sure what linux equiv is... gcc __asm...*/... "lock xchg ax, [mem] " is a cheap multiprocessor safe lock.  for things like queues and lists to add/remove items to process, a simple spin-on-lock with a Relinquish() ... sched_yeild() /*linux*/ or Sleep(0); /* windows */ is sufficient and doesn't require things like scheduling.. like if( locked ) sleep( until_lock_is_unlocked ); ..

static size_t lock;  /* register sized variable */
while( locked_exchange( &lock, 1 ) ) { Relinquish(); };
/* do stuff */
lock = 0;

Blackvoxel use such concepts in message files used for inter thread communications. We used slightly different instructions as we are doing "lockless" working(while ensuring multithread correctness).
In Gcc there is "compiler intrinsics" that are special functions converted directly to assembly. These intrinsics can be translated on the right instructions with different processors.

So, we used the  __sync_bool_compare_and_swap() intrinsic for our stuff.

As what I could read on the web, the InterlockedExchange() equivalent should be the _sync_lock_test_and_set() intrinsic on Gcc. But it must be verified.

Be careful to add the -march=i686 flag on windows gcc, otherwise, the intrinsics won't compile.

As an information, Blackvoxel was nearly entirely developed on Linux. Even the Windows package is compiled on Linux :).

The Blackvoxel Team

28
Programming with Blackvoxel / Re: How hard could smoothing be?
« on: November 01, 2014, 01:20:42 am »
Thanx for your long, detailed response.
Speaking of convering to enum...
It is better to use enums for defines... like the set of FACEDRAW_ values can all be assoicated by putting them in an enum; I dunno; maybe it's more useful in C#... then you reference the enum type and all the values you can pick are available in a drop list.  I guess the other reason I started converting defines to enums was for Doc-o-matic code documentation tool; then all the values would be groups in the same documentation page.  To the compiler enums are constants, so there's no performance hit.

We used enum a lot in Blackvoxel, but also some define  ;).

Our IDE doesn't work well with "auto completion" on enum types parameters (but it work well on most things).

We had some issues while debuging with enums that doesn't make them comfortable because the debugger only show enum name and not the value. So, in some cases, we used define.

Another problem with enum is the inability to specify underlying integer type other than the default int. That's why we rarely use "enum type" with enums. In some case that's because we want to use different storage size. But also for fixing integer size to avoid 32/64 bits issues. C++11 enables to specify enum storage type, but we don't want to switch to this yet.

About documentation, we hope we'll have time to write it one day  ;).

Quote
I do appreciate the care for optimization during development...it's apparent to me as I was learning where things were.
My only criticism is ... some lines are too long  :)  When doing line breaks on long lines, operators should be prefixed on the continuation line and not left on the tail of the prior line... makes reading the expression easier becuase you can see what the op is, inc ase the line is just a little too long; also makes it clear whether a line is a new line (no operator prefix) or a continuation....  This is a habit noone uses or teaches; but I've found of great benefit since early 2000's...

We agree with you on the interest of putting the operators before on line break cases.

About "long lines", we didn't like "over spreaded" code : we found it to be deconcentrating.

Rather than using a single dimension, we write code on 2 dimensions and get it more compact. So can view more of it at the same time.

Prioritarily, the most important code spreads verticaly to get more readable and the less important spreads horizontaly for avoiding bloating as much possible.

But we know every programmer would have have their own personnal opinion on coding style and presentation. Whatever arguments, that's always a matter of personnal taste and programming style.

Quote
Re: multiple rendering paths... yes, I saw the multiple paths, but the difference is how the voxels are iterated, one uses a foreach(x,y,z) the other uses foreach( in sorted list) and then did the same drawing... so I broke out the triangle emitter into a separate function that both use.
Code: [Select]
void ZRender_Smooth::EmitFaces( ZVoxelType ** VoxelTypeTable, UShort &VoxelType, UShort &prevVoxelType, ULong info
                              , Long x, Long y, Long z
                              , Long Sector_Display_x, Long Sector_Display_y, Long Sector_Display_z )
https://github.com/d3x0r/Blackvoxel/blob/master/src/ZRender_Smooth.cpp#L670

Guess I ended up reverting that in ZRender_Basic; so it's back to mostly original code (added culler init)

The render code is considered as a critic portion. So adding a function call in a loop should be avoided.

There is 16x16x64 voxels on a sector, so it's making 16384 functions calls.

If the engine needs to render 10 sectors per frame and 60 frames per second, that would do :

16384 * 10 * 60 = 9830400 function calls per second.

In fact the relative overhead must be relativized because the called function is very long and have a lot of stuff in it. But that's always an overhead that could be avoided.

You could always inline the code to remove this overhead. But some compilers may also choose to ignore this on their own criteria. Inlining is never guaranteed.

So, in parts where you want to be sure the things will be compiled like you want to in a fully predicable way, the best is to write them like they must be compiled without making any suppositions on what optimizations compiler could do.

In doint so, you'll also eliminate possible issues on performance depending on inlining heuristic oddities on a particular compiler.

Of course, I would recommand to do such methods only on critical code sections that realy need maximum speed because reducing code factorization also have bad counterparts.

Quote
---------
My issue is... I realize that voxelSector should be a low level class, and not know things like World.. and now after each new ZVoxelSector    another call to InitFaceCullData has to be made... makes it less clean to just get a sector... need like a sector factory in render or world ... one in render could could use one in world.... But then; Render right now is a game(?) a world(?) ... If it was a factory from render, than smooth sectors could be requested or basic...
------

Video games are very particular programs. That's the kind of stuff that would rarely end up to be coded in a perfect academic ways.

In a game, there is a lot of inner interactions, a lot of technical complexity, a lot of technical constraints, permanent evolutions caused by new ideas.

Doing anything require tough decisions to balance between a lot of conflicting considerations.

In most cases, even the best decision will only lead to the best compromise.

Whatever you do, that will always let you with an impression of imperfection.

Quote
I was having issues getting yaw/pitch/roll reverse calculation working, so I ended up making zmatrix.cpp for those so I could just recompile the one source, since basic types get included by like everything, a change was having to
rebuild everything; need to get rid of .cpp and make the yaw/pitch/roll inlines.  *done*  still some fixups to do in planePhysics I think...
-----
Oh, was playing blackice, and like most FPS they have a pitch lock at +90 degrees... was playin in modified blackvoxel for so long, expected the camera to auto rotate as the angle went over my head. I'm not 100% sure how this works... and I can't seem to get to exactly 90 degrees up or down, and I'm not sure what would happen if you did get there... since I'm always just a hair to the left or right of exactly down, it results in a roll that undoing results in turning you around in yaw... it's really kinda natural... anyway the snap to 'up' annoyed me yet again... it's not like that actually saves you anything, since you later have to compute sin/cos values to apply motion instead of just having the axis array (matrix) available to use....
I thought quaternions would be way complex; but their usage is really to and from storage, and is a way to smoothly translate one rotation matrix to another through a simple linear interpolation of the quaternaion (which is just a vec4) that can be read/written to the 3x3 rotation matrix.  (matrix is 4x4 cause they have a translation/origin row)... would be nice if d3d and opengl used sane matrixes... layed out so the X/y/z axis normal-vectors are lines up as an array of 3 values so you get the vectors immediately without applying a row-pitch offset thing...
------------
but... marching cubes is the 8 bits... which is the 8 corners...  I guess
and I don't need what's diagonal...so the 8 corner cubes from the center... because those don't affect the shape of this or any of the near cubes... so only 18 (6 original, 12 in the 6 ordinals from each of the original 6 ( accessable by 2 names, FRONT_HAS_ABOVE, and ABOVE_HAS_FRONT is the same cube..... but since I don't use the diagonals in the center, then I don't need diagonals from the first mates either... but the I do use 18 bits instead of 8

It's possible we'll use quaternions in the future, at last for some kind of airplane or rocket that needs to avoid gimball lock. As you said, it could have pro and cons. A lot of stuff could be made without.
For the airplane, fine tuning and balancing it's behavior is a long job. Tuning took in fact longuer than implementing the aircraft itself.

----------------------------

Quote
Glad to meet you  :)  don't actually have many peers  :)
can I add you on facebook maybe?  I'd share my name, but it's not like John smith... it would immediately indicate a individual so it's like posting my SSN# here :)

We are also glad to meet you  :)

Yes, you can add us on Facebook. We use the Blackvoxel account as we don't have personnal Facebook accounts. (But our name and the company info are here : http://www.blackvoxel.com/view.php?node=14)

Quote
Uhmm... I had a TI-99/4a with speech synthesizer (why don't I have a simple speech synthesizer today?)  then several years later a Tandy 1000 (no letters after, original - PC jr clone) and turbo pascal

I remember TI-99, but it wasn't very common in our country. Here, most computers were C64, Atari, Amstrad and the local manufacturer, Thomson (These integrated light pen).
At that time, We were programming in Basic and assembly language. Also doing some electronic.
C langage come few years later with Commodore Amiga.

The Blackvoxel Team

29
Programming with Blackvoxel / Re: How hard could smoothing be?
« on: October 30, 2014, 03:15:22 am »
(No worries  about deleted post  :) )

I dunno, from an outsider perspective, having a lot of the system developed, like sector paging, allows focusing on detailed things like smoothing.

A brain as a tiny voxel world would only be a few sectors really... and really only rendered in a certain mode...
The brainboard itself has worked for a long time, even had a virtual world ages ago; but lost a lot of code around 2000.  The idea is certainly sound... right now I have just 2d gui's for laying them out... which is even sufficient... I have free-vector renderer even, so can dynaimcally script creating shapes and give them brains.... but; like you mentioned; doesn't do much good if there's no way for people to learn how it works... need tutorial levels for instance... in black voxel; how to make a sequencing conveyor system for instance... saw a conveyor system setup to sequence carbon and metal into furnaces...

This seems a very interesting project. When I was younger, artificial intelligence fascinated me.

Yes, there is a sequencer : http://www.blackvoxel.com/view.php?node=1457

I guess, maybe you need to program your own voxels ?

So :

The "active voxels" behavior is programmed in ZVoxelReactor.cpp.

At line 1971 is the sequencer code. But simpler voxels like water and falling rocks are easier to understand.

Don't worry about the voxel fetching code in these actual examples which is complicated to read.
You can also use the world's GetVoxel(), GetVoxelExt() and SetVoxel_WithCullingUpdate() to fetch and set Voxels.

If you want to store some data for your voxels, you need to create extensions. These are classes derived from ZVoxelExtension (See ZVoxelExtension_Sequencer.h) . Pointers to extensions are stored in the OtherInfo[] table of the sectors.

Active voxels needs also have a voxeltype class derived from ZVoxelType : See ZVoxelType_Sequencer.h

Then, your class should be instanced in ZVoxelTypeManager.cpp in function ZVoxelTypeManager::LoadVoxelTypes()

The last thing to do, is creating a voxelinfo_xxx.txt file in the voxelinfo directory with BvProp_Active=1.

I dont know if my explanations are sufficient. I think a tutorial should be made to create new voxels.

Quote
I dunno... I have a voxel project I started several weeks ago, but realized there was a lot of managment system around that would have to be implemented...

Yep, that's exactly the problem. That's all what is around that is the true huge work.

Quote
Me, I'm a software developer from '84 when I was 12.

Me too... :) , it seems that we have the same age and started at the same time!

I had an Atari 600xl for my first computer.

That was a fascinating machine designed by Jay Miner, who was also the father of the Commodore Amiga.

So, we know your are understanding what we mean when speaking of game history  ;) .

Quote
I've seen a lot of 'this should be a simple idea' turn into 'that can't be implemented in a reasonable amount of time'  while I've also seen things that seemed really complex turn into less than 2 line code change.. But I'm more in the practice of implementing before fully fleshing out; if the skeleton doesn't work at all, not much use in fleshing it out.

Yep, you are right on saying that sometime it's possible to find clever idea to make complex things in simple way.

For making multi grid, the idea would involve several systems. Not only rendering, but collision and game interaction. It's very unlikely to found fast ways for all the stuf, so there is an important incompressible work.

That said, that's always interesting to think and speak about things. That's giving us ideas on what we could plan in the future. And some of your propositions will make their way.

I've said that making some rework on rendering engine is on the roadmap. And we remembered that some math stuff need some rework (because of some Z-buffer problems when going very far).

Adding some features on this occasion won't add much work. And at first glance, it won't harm performance to add world level Matrix. And maybe we could add also some sector Matrix.

So, even if Blackvoxel won't use it at short term, that will be another capability for some future ideas.

Hoping some great ideas will come with time.

Of course, that wont be enough to make some multi grid effective in the game. There will remain some missing parts.

Quote
And I remember when CPUs were all there was, and they weren't all that fast, and had 3d rendering code back then that was decent... then 3DFX game out with Glide, and things looked better, then nVidia bought them and killed Glide.  Which made me hate nVidia for a long time... I did notice that these opengl display lists are amazingly fast.  Probably going to take that and port it to my opengl 1 display layer... for opengl2 I was building vertex buffer object things, but with display lists it would record the change in textures too... had to build vertex buffers for each source texture (or texture combination)...

Yep, display lists have their advantages when speaking of voxel rendering. At beginning we made several experiments with different approaches including VBO and Geometry Instancing. Display List appeared to be the best compromise for what Blackvoxel needed.  At last it was what we found at the time we made the engine.
But we have a lot of ideas of better things for the future.

Quote
But; I've even hand coded assembly for optimization... although even the slowest processors today make it unnessecary for presentation/interaction purposes that I've let most of that code fall by the wayside about the time 64 bit processors came out... most compilers do clever optimizations.... but I'm well away of the difference between accessing a pointer/reference and just having a thing there... and the additional lookups required for what might on the surface seem like a simple operation... Every line is often written with 'is there a way to make this cheaper?' in mind... Voxel engines definatly benefit from having memory resources, and doesn't require as much optimization as once upon a time (64k was tough to do much, 640k seemed like infinite memory... until it wasn't... )... but anyway; I'm certainly not a noob when it comes to development...

Yep, like old developers, you had the chance to do assembly. That's a chance we had at these old period of time.

Modern processors have incredible power compared to what we had in early time.

In many program, even game, there is no need to make great attention to optimization. In most cases, a programmer make it's program, almost never think about optimizing... and most time it will work fast enough.

When developing Blackvoxel, we quickly understood that what we wanted to do would be a very special case because of the MVI problem. It quickly appeared that it wouldn’t run fast enough at all. Without optimizing, things were far too slow. It appeared that we had to optimize strongly to achieve the goal.

This is the reason for the mania about performance in Blackvoxel. This is also why we have adopted in this project a specific coding style much "close to the machine" . Although we know that compilers can optimize many things, staying close to the machine is helping to avoid writing too heavy things without realizing it.

We know however that there is a downside to this. Like said, whatever way a programmer will choose, there will always be a price to pay...

Quote
-----
Okay So I've associated a ZVoxelCuller_Basic:ZVoxel_Culler with ZRender_Basic and a ZVoxelCuller_Smooth:ZVoxel_Culler with ZRender_Smooth...

Code: [Select]
class ZVoxelCuller
{
protected:
   ZVoxelWorld *world;
public:
    ZVoxelCuller( ZVoxelWorld *world )
    {
        this->world = world;
    }

    void SetWorld( ZVoxelWorld *world )
    {
        this->world = world;
    }

    virtual void InitFaceCullData( ZVoxelSector *sector ) = 0;

    // if not internal, then is meant to cull the outside edges of the sector
    virtual void CullSector( ZVoxelSector *sector, bool internal ) = 0;

    virtual void CullSingleVoxel( ZVoxelSector *sector, int x, int y, int z ) = 0;

    // get a single voxel's culling (copy sector); ULong value used even though some culling only uses 6 bits
    virtual ULong getFaceCulling( ZVoxelSector *Sector, int offset ) = 0;
    // set a single voxel's cullling (copy sector)
    virtual void setFaceCulling( ZVoxelSector *Sector, int offset, ULong value ) = 0;
    // Load culling from a stream
    virtual bool Decompress_RLE(ZVoxelSector *Sector, void * Stream) = 0;
    // save culling to a stream
    virtual void Compress_RLE(ZVoxelSector *Sector, void * Stream) = 0;


};
which the moves code from ZVoxelWorld to an implementation of the virtual interface, then set the interface per sector.... added a ZVoxelCuller *culler; next to void *FaceCulling;

----------------
Nuts and bolts out of the way...

was thinking about how to implement reasonable alogrithms for culling... since corners will now need diagonal sectors to get information.

so... culling internal works
culling external should be broken into 3 parts... the face... if PartialCulling is set, process the inset face, without edes and corners.
Edge cubes should get updated when any of the other near three sectors get added... I guess only a subset of bits have to be set;
And corner cubes...

Yep, that's a more complex algorithm and it needs more sectors. That's a great work.

Full Culling for actual unsmoothed voxels are working with 7 sectors (1 + 6 adjacent).

With need of diagonal voxel data, it will need 27 sectors (1 + 26 adjacent).

Quote
if I portion it out that way should simplify some iterations...

The basic culler should use a staggered cube algorithm... then only 50% of the cubes have to be checked...

Yep, as you said it can do a significant difference, it sounds interesting.

Quote
Other thoughts I pursued a while ago was to make like a surface interface between cubes and non-cubes... so solid things wouldn't have to be considered... kinda similar to what sorted render does... that a list of 'interesting' voxels is produced.... but similar to the face conglomerating idea; it ends up causing more work than benefit in sparsely populated voxel sectors... although a normal landscape would basically boil down to 256 interesting points per sector... for a normal terrain mesh.. a few extra points for steep hills...
just musing

I take this opportunity to mention that there are currently two different versions of the rendering system. A flag in the sector (Flag_NeedSortedRendering) may activate one or another. The sector generator decides which render type must be used on a particular sector.
The second version adds sorting by voxel type. Depending on zones, sorted rendering is much faster... or reduce performances. The sorting algorithm is supposed to be activated in the areas with exposition to view of many vertical voxeltype alternances.
I'll modify Blackvoxel by changing this flag to an integer enum type. So, Blackvoxel will support the possibility of having many rendering types for the same world.

The Blackvoxel Team

30
Programming with Blackvoxel / Re: How hard could smoothing be?
« on: October 28, 2014, 12:05:07 am »
Sorry d3xOr, we have deleted one of your posts by mistake  :(.

Unfortunately the database backup was made before this post.

It's remaining in the quotes of our last reply.

The Blackvoxel Team

Pages: 1 [2] 3 4 ... 10