Author Topic: How hard could smoothing be?  (Read 35538 times)

d3x0r

  • Jr. Member
  • **
  • Posts: 75
    • View Profile
How hard could smoothing be?
« on: October 25, 2014, 02:20:50 pm »
Well... it was a lot harder than I thought...

http://youtu.be/F4AUEHm7BAk

work in progress; maybe

inner corners are unsmoothed... just smooths the outer edges.

Have many corner cases that have multiple solution paths... light right now single blocks and single rows of blocks don't smooth either....

and I have a broken corner that could be better; not to mention all the directions that are yet missing.

was going to map textures on diagonals such that if it's a top & left visible, to show the top and left side of the texture, so the diagonals would get an extra line; right now all left & anything is left and any right & anthing is right and any forward & any is forward and same for back; sorta, top&((right/left)&(back/front)) is top....


olive

  • Administrator
  • Full Member
  • *****
  • Posts: 149
    • View Profile
Re: How hard could smoothing be?
« Reply #1 on: October 25, 2014, 11:08:10 pm »
Well... it was a lot harder than I thought...

http://youtu.be/F4AUEHm7BAk

work in progress; maybe

inner corners are unsmoothed... just smooths the outer edges.

Have many corner cases that have multiple solution paths... light right now single blocks and single rows of blocks don't smooth either....

and I have a broken corner that could be better; not to mention all the directions that are yet missing.

was going to map textures on diagonals such that if it's a top & left visible, to show the top and left side of the texture, so the diagonals would get an extra line; right now all left & anything is left and any right & anthing is right and any forward & any is forward and same for back; sorta, top&((right/left)&(back/front)) is top....

Amazing mod...  :)

Nice marching cube smoothing.

It looks very good in the video.

We have seen in the video that you are using Visual Studio. We assume you are using Gcc/Gdb with it?

The Blackvoxel Team

d3x0r

  • Jr. Member
  • **
  • Posts: 75
    • View Profile
Re: How hard could smoothing be?
« Reply #2 on: October 26, 2014, 12:45:01 am »

We have seen in the video that you are using Visual Studio. We assume you are using Gcc/Gdb with it?

The Blackvoxel Team
Nah, using cmake, which spits out makes appropraite for visual studio, I also use open watcom on the side... watcom, gcc and msvc all catch different errors; but cmake allows one make system to support them all...
SDL also uses cmake scripts...
It took a long while; but cmake 2.4 was actually usable.
--------
It's actually not marching cubes...
Was reverting it so I could sway between Render_Basic and Render_Smooth by making most of renderBasic be REnder_Interface, and a virtual 'render' function... but it's just based on existing culling flags
Code: [Select]
if ( ( culling & (FACEDRAW_TOP|FACEDRAW_LEFT) ) == (FACEDRAW_TOP|FACEDRAW_LEFT) )
{
     // example excpetion...
     // if( ( culling & ( FACEDRAW_TOP_HAS_FORWARD| FACEDRAW_LEFT_HAS_FORWARD ) )
     //      == ( FACEDRAW_TOP_HAS_FORWARD| FACEDRAW_LEFT_HAS_FORWARD ) )
     //     // render more complex interpretations
     //  else
     //      //basic, draw  diagonal slope.
}
unfortunately because near sectors aren't always present, and i haven't fixed up the mating clipping updates, on sector boundaries I get false lacking cubes...

and I broke saves because I expended the clipping data from a byte to a long; needed 12 more bits in addition to the original 6 facedraw bits...
and I think I might need all 8 more for the corners ... 26 total... representing the cube of voxels around a center voxel

----------
re the floating land issue; it started showing up when I shrank the block size, so I figured it was just able to see so many more sectors above it that it actually ended up just treating every sector as 'need to fill' ... but it may be a cube that failed to be ejected properly... because again the scale of the world and that doesn't match what it was designed to be..
was just curious if it was something you had seen or something I had messed up... I'll revert some at some point and see what I did.
« Last Edit: October 26, 2014, 01:01:42 am by d3x0r »

olive

  • Administrator
  • Full Member
  • *****
  • Posts: 149
    • View Profile
Re: How hard could smoothing be?
« Reply #3 on: October 26, 2014, 07:07:02 am »
...
re the floating land issue; it started showing up when I shrank the block size, so I figured it was just able to see so many more sectors above it that it actually ended up just treating every sector as 'need to fill' ... but it may be a cube that failed to be ejected properly... because again the scale of the world and that doesn't match what it was designed to be..
was just curious if it was something you had seen or something I had messed up... I'll revert some at some point and see what I did.

Hi,

Maybe something could help you  :)

We just published on github some changes allowing to modify voxel size.

But we have taken a particular way to do it.

That's possible because in Blackvoxel, the voxel size is considered to be some kind of base unit.

The player coordinates are considered subdivision of the base voxel unit, something like centimetres. 256 was chosen because it's a power of two.

So, even if the voxel size are changing, the conversion between "voxel units" and "player units" can stay the same.

Look at this line ( in ZActor_Player.cpp), where is defined observer eye position relative to the voxel size.

EyesPosition.y = 256.0 * 1.75;

We changed it to :

EyesPosition.y = 256.0 * 1.75 / COMPILEOPTION_VOXELSIZEFACTOR;

There's also some other locations where some values had to be changed, we did it for selector distance. This remain to be done for player speed.

Of course, player collision should be fixed also with this. But it's "working" enough for testing.

You can change voxel size in ACompileSettings.h with the parameter COMPILEOPTION_VOXELSIZEFACTOR

Changing it from 1.0 to 0.5 will make the voxels smaller.

We hope this could help.

The Blackvoxel Team

d3x0r

  • Jr. Member
  • **
  • Posts: 75
    • View Profile
Re: How hard could smoothing be?
« Reply #4 on: October 26, 2014, 10:12:16 am »
Ya but you're shrinking the character too; the point was to add more detail to the world... not just shrink the virtual render size :)  increases the sector count visible though I suppose; but that's just extending the sphere selector thing

It's basically the same mod I did... as a const int variable it's gonna compile away anyway... but as a variable it's not been a huge performance hit.

eye position should be 256

---------
So the inner-sector culling routine is working.... I'm gonna have problems with the inter-sector culling though... the inner-sector had a nice 3x9 block of sectors it was working with... I'm thinking of redoing the seam culling with a 2x3x3 row of voxel references.... need to recover blocks that are near the blocks that are near also...

pretty sure I can do this without the 8 corner cubes in the 3x3x3 set around a point.

was also considering how to implement that culling data is 8 or 32 bits (save version?) but would like to retain so renderer could be applied per sector... but keep the optimal 6 bit culling for render basic...
--------
not sure that a reference looking for Xoffset Yoffset etc is any faster than doing the shifts... since the coorindate has to be shifted to index the array anyway...
« Last Edit: October 26, 2014, 10:14:44 am by d3x0r »

d3x0r

  • Jr. Member
  • **
  • Posts: 75
    • View Profile
Re: How hard could smoothing be?
« Reply #5 on: October 26, 2014, 11:52:54 am »
I see what you did...
just moving the eye position can shrink the world... but then the player body doesn't match the eye .. and if I have a 2 block hole in a wall I can go through the wall even though my eyes are staring at blocks.

Detection distance.... should stay the same I guess... just has 64x the blocks in the same space...

like I should still be able to dig the same distance as 6 blocks from before... but now it's 24 blocks deep instead

d3x0r

  • Jr. Member
  • **
  • Posts: 75
    • View Profile
Re: How hard could smoothing be?
« Reply #6 on: October 26, 2014, 03:29:13 pm »
Is sector based PartialCulling really worth even having?
How often is an entire side of a thing opaque?
Culling needs to not be a world property; but really more something associated with the renderer; because only the renderer (and save file) is a consumer of cull data.  It certainly makes sense how it currently is that nothing(except game container) knows about the renderer... I started passing the game environment as initializers so classes could get the renderer; and added some culling methods to renderer... but maybe ZVoxelCuller should be a utility class that's associated with/retrieved from the renderer, and then pass that.  ZVoxelCuller would look more like an interface with


ZVoxelCuller( ZWorld &world );

InitFaceCullData( sector );

CullSector( sector, internal/edges);
-or-
CullSectorInternal( sector );
CullSectorEdges( sector );

and CullSingleVoxel( sector, (x,y,z/offset) );

----------------
started ending up with circular references, and things that had static members of thigns like SectorStreamLoader has a SectorCreator, which allocates sectors, and needs the culler (sounds like color)....
----------
I'm enjoying mangling your code :)
I'm going to make some other forks and clean up portions of this, like ZMatrix modification to Camera/PlayerPhysics
---------
also right now; draw_left actually draws the right side; and draw_right draws the right side...
this is a texture labeled according to clipping output...
the letters are oriented up on the sides... and the A for 'above' is up-right if looking from the back to the front.   if you like bypass the 'DRAWFACE_LEFT' there will be a hole for the 'L' side...  either that or forward/backward are reversed... or above/below... if any of these were wrong in the current... render matrix, then the RTFM cube would have backward text on some sides... and an original version does the same thing. (I'm pretty sure.... mine's totally broken right now, going forward with ZVoxelCuller utility class...)
-------------
Disrelated to the thread, but was thinking about other objects... I guess ZVoxelWorld is where I should add a ZMatrix for its current orientation, and give it a small sector size... was going to make ZVoxelSector instances that were standalone for large object models... (an airplane instead of a box with an airplane picture for instance), but maybe it makes more sense to instance worlds
« Last Edit: October 26, 2014, 03:38:06 pm by d3x0r »

d3x0r

  • Jr. Member
  • **
  • Posts: 75
    • View Profile
Re: How hard could smoothing be?
« Reply #7 on: October 26, 2014, 06:57:56 pm »
Neither of these routines use FaceCulling or 'data' variable associated with the face culling faces... so I think the faceculling save/load isn't working...

Compress_FaceCulling_RLE(UByte*data, void*stream)
{
     ... actual = OtherInfos[p];
}
Decompress_FaceCulling_RLE(UByte*data, void*stream)
{
    OtherInfos[p] = read;
}


olive

  • Administrator
  • Full Member
  • *****
  • Posts: 149
    • View Profile
Re: How hard could smoothing be?
« Reply #8 on: October 26, 2014, 11:54:02 pm »
I see what you did...
just moving the eye position can shrink the world... but then the player body doesn't match the eye .. and if I have a 2 block hole in a wall I can go through the wall even though my eyes are staring at blocks.

Detection distance.... should stay the same I guess... just has 64x the blocks in the same space...

like I should still be able to dig the same distance as 6 blocks from before... but now it's 24 blocks deep instead

The problem come from the collision detection system. That's what is making in fact the effectiveness of the body size.

Anyway, whatever the way, the collision detection must be redone if you want to change voxel size.

Increasing the detection distance appeared to be necessary because with some ratio, the player was unable to get voxels just at it's feet.

Quote
...but as a variable it's not been a huge performance hit.

Not only a variable, but also using division/multiplication instead of shifting ... unless you would be limited to power of two ratio.

Sure, it's not huge, but it's always good to avoid when possible.  :)

Yes, it's not always possible to avoid adding overhead when adding features.

Unfortunately, there is actually no way to make all voxel game possible features in one single engine. Some combinations would be technically impossible.

That's why, adding features will always be a matter of choice depending on what feature are wanted for a particular game.

The Blackvoxel Team

olive

  • Administrator
  • Full Member
  • *****
  • Posts: 149
    • View Profile
Re: How hard could smoothing be?
« Reply #9 on: October 27, 2014, 01:02:58 am »
Is sector based PartialCulling really worth even having?
How often is an entire side of a thing opaque?
Culling needs to not be a world property; but really more something associated with the renderer; because only the renderer (and save file) is a consumer of cull data.  It certainly makes sense how it currently is that nothing(except game container) knows about the renderer... I started passing the game environment as initializers so classes could get the renderer; and added some culling methods to renderer... but maybe ZVoxelCuller should be a utility class that's associated with/retrieved from the renderer, and then pass that. ZVoxelCuller would look more like an interface with

ZVoxelCuller( ZWorld &world );

InitFaceCullData( sector );

CullSector( sector, internal/edges);
-or-
CullSectorInternal( sector );
CullSectorEdges( sector );

and CullSingleVoxel( sector, (x,y,z/offset) );

...
-------------

Does culling must be done at world level ? In some parts, sectors need to be processed independently to the existence of a world.  So, your proposition of an independent class is an efficient idea to clean the mess.

But we are tempted to say : don't mangle your head too much with that.

Because what is on our roadmap for future is making face culling completely integrated with rendering phase.

Yes, making that could be a very, very bad choice in some ways because it can add significant overhead to the main thread.

So, why we plan to do it ? Let's explain the actual system and why it could be better.

Keep in mind that Blackvoxel made very specific game choices. It's MVI engine cause massive load on rendering. Also, vehicles like airplane needs very quick refresh of sectors.

That's why avoiding bottleneck in rendering was vital for Blackvoxel. That's wasn't easy to make rendering fluid enough and balance rendering settings for getting fluid and quick.

To alleviate the task of rendering, work had been distributed on other threads. At sector loading/creation, main culling is done. It's completed on the MVI's thread for partial culling. Partial culling is also done on individual voxel refresh.

So you are asking, why partial culling ? Does it is really needed ?

Partial culling is not related to having an entire sector face culled, that's simply because voxels on the sides of a sectors needs to have culling like others. Unless that, it will make visible errors on side... or huge slowdowns if you make sides "all visibles".

The problem is simple : external faces of voxels on side can't be culled without data of the neighboring sectors.

At sector loading and creation, we can't rely on other sectors being in memory, so we made culling for all faces but the outside.

Further, MVI thread check if culling is needed on sectors and look if neighboring sectors are loaded. If so, culling is completed on the relevant side. Making these partial culling make less load on memory system than full culling.

All that stuff is complicated, but it's fast and working well.

So, why wanting to change that ?

In one word : because of MVI. What is optimized in one hand is bad on other.

The system rely on fact the faces are (nearly) always culled. So when MVI needs to move a voxel, it must redo culling on it's faces. Looking at the voxel changing routine will let you understand what we mean : it could be faster without need to keep faces culled.

Since the begining, we optimized rendering a lot with various stuff. It may compensate enough for what will be lost.

It will also be simpler, eliminate a lot of architectural complexity and make it more powerfull for future architectural change we are planing.

The Blackvoxel Team

olive

  • Administrator
  • Full Member
  • *****
  • Posts: 149
    • View Profile
Re: How hard could smoothing be?
« Reply #10 on: October 27, 2014, 03:06:30 am »
Neither of these routines use FaceCulling or 'data' variable associated with the face culling faces... so I think the faceculling save/load isn't working...

Compress_FaceCulling_RLE(UByte*data, void*stream)
{
     ... actual = OtherInfos[p];
}
Decompress_FaceCulling_RLE(UByte*data, void*stream)
{
    OtherInfos[p] = read;
}

Yep, that's buggy as hell. In fact, this feature was never tested. Because we found some conceptual problems, we let enabled culling at sector loading.

Quote
was going to make ZVoxelSector instances that were standalone for large object models... (an airplane instead of a box with an airplane picture for instance), but maybe it makes more sense to instance worlds

We are thinking for a long time about adding multi-grid capability. But not decided yet to do it.

Technically, doing multi grid :

Rendering at screen multiple grid is the easy part. Making collision, physics and game interactions is the true big work. This could also lead to rework architecture and change multiple parts.

There is the CPU limit problem : Blackvoxel is already using a lot with MVI. That's already limit for dual cores machines. Unfortunately, Intel is much too slow to popularize 8 cores machines.

For your questions, making it at the world level could be more ideal on the paper. Doing it with sector might be more pragmatic. There is good and bad on two ways.

Some stuff are already implemented to get that kind of thing with sectors :

Sectors are supporting dynamic size along with 'optimized' fixed size.

Some code can manipulate data between sectors.

Look at the "Copy clipboard" and "Paste Clipboard" labels in ZGame_Events.cpp

The Blackvoxel Team

d3x0r

  • Jr. Member
  • **
  • Posts: 75
    • View Profile
Re: How hard could smoothing be?
« Reply #11 on: October 27, 2014, 03:09:49 am »
The problem come from the collision detection system. That's what is making in fact the effectiveness of the body size.

Anyway, whatever the way, the collision detection must be redone if you want to change voxel size.
Increasing the detection distance appeared to be necessary because with some ratio, the player was unable to get voxels just at it's feet.

Quote
...but as a variable it's not been a huge performance hit.

Not only a variable, but also using division/multiplication instead of shifting ... unless you would be limited to power of two ratio.

Sure, it's not huge, but it's always good to avoid when possible.  :)
again .. I didn't change any operators... defined the number of bits a voxels is...
certainly changing just how it's rendered by moving the eye and changing the body size would be a smaller change.

*Shrug*
but voxel size should be a property of a ZVoxelWorld... (like space engineers, the base world is 'Platform' and is aligned square with the universe, other 'worlds' would be implemented for small/large ships which have a translation/rotation matrix associated with them... and a different rendering size)

just played a demo of another game that's voxel based called 'Masterspace' a starforge clone kinda, but with much better game dynamics than starforge ended up with... land is smoothed, but when you re-place blocks you can put them back out as square units, so buildings built with mined resources are square....
« Last Edit: October 27, 2014, 03:25:22 am by d3x0r »

d3x0r

  • Jr. Member
  • **
  • Posts: 75
    • View Profile
Re: How hard could smoothing be?
« Reply #12 on: October 27, 2014, 03:19:43 am »

Does culling must be done at world level ? In some parts, sectors need to be processed independently to the existence of a world.  So, your proposition of an independent class is an efficient idea to clean the mess.

But we are tempted to say : don't mangle your head too much with that.

Because what is on our roadmap for future is making face culling completely integrated with rendering phase.

Yes, making that could be a very, very bad choice in some ways because it can add significant overhead to the main thread.

So, why we plan to do it ? Let's explain the actual system and why it could be better.

Keep in mind that Blackvoxel made very specific game choices. It's MVI engine cause massive load on rendering. Also, vehicles like airplane needs very quick refresh of sectors.

The Blackvoxel Team
Right; it looks like something that should be part of the render path, with the same dirty flags, so if the display lists are built already, just run those, otherwise do the required updates.

Also, the game is maxed at 17-19%... would be interesting if more threads could be started to process things like water.  looks like there's only 1... and the renderer doing work.

Re the PartialCulling... when it comes time to check the edges... Oh I see... partial culling is cleared as near sectors are found so it doesn't have to do all sides every time a new near sector is added.

olive

  • Administrator
  • Full Member
  • *****
  • Posts: 149
    • View Profile
Re: How hard could smoothing be?
« Reply #13 on: October 27, 2014, 08:24:00 am »
The problem come from the collision detection system. That's what is making in fact the effectiveness of the body size.

Anyway, whatever the way, the collision detection must be redone if you want to change voxel size.
Increasing the detection distance appeared to be necessary because with some ratio, the player was unable to get voxels just at it's feet.

Quote
...but as a variable it's not been a huge performance hit.

Not only a variable, but also using division/multiplication instead of shifting ... unless you would be limited to power of two ratio.

Sure, it's not huge, but it's always good to avoid when possible.  :)
again .. I didn't change any operators... defined the number of bits a voxels is...
certainly changing just how it's rendered by moving the eye and changing the body size would be a smaller change.

*Shrug*
but voxel size should be a property of a ZVoxelWorld... (like space engineers, the base world is 'Platform' and is aligned square with the universe, other 'worlds' would be implemented for small/large ships which have a translation/rotation matrix associated with them... and a different rendering size)

just played a demo of another game that's voxel based called 'Masterspace' a starforge clone kinda, but with much better game dynamics than starforge ended up with... land is smoothed, but when you re-place blocks you can put them back out as square units, so buildings built with mined resources are square....

I understand what you mean.

But Blackvoxel isn't Space Engineer  :)

Every game engine is developed with particular features and choices in mind.

Blackvoxel have it's own features that no other have. And there is a lot of things to dot with these.

True treasures are where nobody is searching, not the shiny things in the other's garden.

The main concern of our team was to make it's own way, to find original concepts and ideas. That's what given it's own personality to Blackvoxel. That's what we'll continue to do.

Believe me, there is more interesting things to do with Blackvoxel actual engine than running after concepts everybody is already exploring. Remember what's happened to all these Minecraft clones.

So, even if we decide for some reasons to switch to multi grid one day, we'll need to do this with an interesting and original way.

Quote
Right; it looks like something that should be part of the render path, with the same dirty flags, so if the display lists are built already, just run those, otherwise do the required updates.

That's what the engine is doing. But it isn't sufficient at all. That's why there is much more optimizations.

OpenGL/DirectX are mainly optimized for vector games and are big bottlenecks for voxel games.

Quote
Also, the game is maxed at 17-19%... would be interesting if more threads could be started to process things like water.  looks like there's only 1... and the renderer doing work.

Yes, there is some room left on high end machines. But a game should be playable also on modest machines.

As we ran Blackvoxel on a lot of machines we know that some modest dual cores machines are on the limit with actual load. And there is places where loading is very strong.

Percentage of use given by operating systems aren't really useful about games. Strong stuttering can occur while processors have time left remaining.

Quote
Re the PartialCulling... when it comes time to check the edges... Oh I see... partial culling is cleared as near sectors are found so it doesn't have to do all sides every time a new near sector is added.

Yep  :)

The Blackvoxel Team

d3x0r

  • Jr. Member
  • **
  • Posts: 75
    • View Profile
Re: How hard could smoothing be?
« Reply #14 on: October 27, 2014, 08:52:31 pm »
Kinda off topic

a game before minecraft; resurrected...

http://gamasutra.com/view/news/228123/Infinifactory_and_the_next_generation_of_the_Minecraft_genre.php

http://www.zachtronics.com/infinifactory/

has conveyor system

I dunno the other thing I'm working towards is a visual pgrogramming language... Alice is close to usable
http://www.alice.org/index.php

someday there will be a game that is one game to rule them all.  that should be my signature :)