My View David's view on anything and everything…

18Jan/14Off

Development 2014-01-18

A few notes on what is currently under development for TerreSculptor.

Dark Theme

The conversion to a dark color theme user interface is underway and going well.
It is actually taking less time to convert than originally estimated.

Included with the dark theme change will be some required rearranging of a few user interface items, and some updated user interface controls.
The interface change will also require higher computer system specs, such as a Shader Model 2.0 but preferably Shader Model 3.0 video card.

I will post some initial screenshots of TerreSculptor in its dark color theme soon, plus additional blog posts during the conversion process.

dotNET

In addition to the dark theme change, TerreSculptor will be updated from dotNET 3.5 to dotNET 4.5.
The main features that the update to dotNET 4.5 will bring are some additional multi-threading capabilities and arrays larger than 2GB.

Mask Editor

The design and layout of the Mask Editor is almost complete.
The Mask Editor is a full-featured Mask, Weightmap, and Splatmap creator and editor, and will also eventually include a full set of grayscale painting and touchup tools.

A weightmap is a special type of mask used in specific terrain texture shaders, and the Mask Editor will take care of all of the fiddly technical bits to make them easier to create and save.
Weightmaps are used by many engines including UDK for its Landscape actor.
Splatmaps are essentially 4 weightmaps packaged into the ARGB planes of a single 32-bit image file, and are used by many engines including Unity.

The current Extractors dialog was only a temporary test dialog and will be replaced by the Mask Editor.
The next steps for the Mask Editor are to draft up the dialog's functionality list and flowchart, and then commence with writing all of the required code classes.
I will post a screenshot of the Mask Editor soon, using a few simulated design elements, in order to provide an early preview look at it.

-eof-

Filed under: TerreSculptor No Comments
6Jan/14Off

2014: The Roadmap from Here

I receive many emails asking if and when certain features will be available in TerreSculptor.

The concept behind TerreSculptor is to create the ultimate heightmap and terrain creation and editing software.
A blend of the relevant parts of Autodesk Max, Pixologic Z-Brush, Adobe PhotoShop, with GIS and other features mixed in.
The creation and editing of heightmaps and digital elevation models and terrain meshes.
Plus the creation and editing of masks and splatmaps.
Plus part 3D game engine to include backdrops, water, and the ability to walk on or drive over the terrain.

Full 3D painting and sculpting with multiple brushes, and full 3D spline and object modifiers.
Immediate Mode editing like PhotoShop: ie. load a file, make a change, save the file.
A Max-like Terrain Stack for 2D and 3D modifiers: ie. add 3D splines such as rivers and roads.
The TerreNetwork: a comprehensive graphical node network of all available devices.

 
Often I can sense the impatience behind the questions, which I completely understand, I too would like to have everything fully functional.

To place the software development time into perspective, some readers may be familiar with Substance Designer from Allegorithmic .
To quote Sebastien Deguy who started the company:

"At the time I thought it would take two guys and a year or two,” says Deguy. “It took more than 20 people and about ten years."

The vision for TerreSculptor is equal or greater, a blending of features from Max, Z-Brush, PhotoShop, GIS, and more.
I am but one programmer working more than full-time on this, one person doing all of the work.  Substance Designer required more than 200 man-years.

It has been difficult to even get 'beta testers' who are willing to test the software for no pay.
My end game is to hire a second programmer using all of the funds received from the sale of the first version when it becomes available.
Until then, please be patient if the feature you are looking for has not been completed yet, this type of software literally takes many years to develop.

Some of the outstanding major features still to be completed include:
- loading and saving of the full project file
- terrain painting
- terrain stack
- terrenetwork
- mask and splatmap editor

-eof-

Filed under: TerreSculptor No Comments
26Dec/13Off

Build 2013-12-25

The TerreSculptor build for Dec 25 2013 is available for download.

This is a highlight of some of the changes and new features.

 

Noisemap Generators

The Noisemap Generator dialogs have undergone some minor layout changes which includes moving the Toolbar and Presets controls out of the Generator group and into the top-level Noise Parameters group.
The dialog functions for Copy, Paste, Reset, and Default, are moved to the Toolbar, along with some newly added functions.

The Toolbar buttons from left to right are:

- Randomize the noisemap parameters.  Note that this function randomizes the parameters through their full range, so it is possible to get full 0% or 100% flat noisemaps.
- Hold the current noisemap parameters in the Hold buffer.
- Swap the current noisemap parameters with the contents in the Hold buffer.
- Open a Noisemap parameters file.
- Save the Noisemap parameters to a file.
- Copy the Noisemap parameters to the Windows clipboard.
- Paste the Windows clipboard to the Noisemap parameters.
- Reset the Noisemap parameters to the values prior to opening the dialog.
- Set the Noisemap parameters to their default values.

The Noisemap Generators are now fully multi-threaded as covered in the previous blog posts, with the exception of Random which is still being updated.

The Tools menu, Options item, System tab, Scaling settings will show the total system logical processors available for multi-threading.

In most cases the Threads value should be left at Auto, which will use the total number of logical processors in the system.
However, in cases where there may be other CPU intensive applications running at the same time, it may be benefitial to their performance if the TerreSculptor Threads value was set to 1 or 2 less than the total logical processors available, in order to allow some CPU threads for the other applications.

The Threads count drop-down list for a 16-core computer system:

The Build Processing dialog will show the current number of logical processors in use when building Noisemaps.
The ability to Cancel the build will be functional soon.

 

Dimensions

The Dimensions Sets management for the entire application has been refactored.
This currently affects the New dialog.  The Resample dialog will also be updated to utilize these changes.

The Tools menu, Options item, Preferences tab, Dimensions settings allow for customizing which dimensions sets are displayed on the New dialog's Size drop-down list.
These settings only affect the Size drop-down list, they do not affect the Width and Length spin numeric controls.

The Range drop-down lists allow you to specify the minimum and maximum heightmap range that is shown.
For example, if you are working on projects that only use a range of dimensions from 1024 through 4096.

Enabling one or more of the dimensions sets will include them in the New dialog's Size drop-down list.
For example, if you are working on projects for CE3 you might choose to enable only the CryEngine checkbox.

These options include:

- Power-of-Two (2^n) eg. 32, 64, 128, 256, etc.
- Power-of-Two's Include Intermediate will add 48, 96, 192, etc.
- Power-of-Two Plus 1 (2^n +1) eg. 33, 65, 129, 257
- Power-of-Two Plus 1's Include Intermediate will add 49, 97, 193, etc.
- CryEngine 3
- UDK Landscape, enabling this will also show the UDK button on the New dialog.

 

Project Open and Save

The Project Open and Save functions are now partially functional.
As of this build, only the Project File Properties and Lights are currently saved to the file.
I will post updates to the blog or create a page on the wiki to track which application objects and sections are saved and opened to the project file over the next few builds while this functionality is being completed.

The Project Open and Save functions will include an option dialog that allows selection of the various application objects and sections that are to be saved or opened.
For example, Cameras, Lights, Grids, Backdrop, Fog, Water, Heightmap, Layout sections, Terrain sections, etc.
This will allow for saving custom template files, or merging selected objects and sections from an existing file with the current project, etc.

 

A note to anyone attempting to register on the forums.
I had to temporarily disable registration due to someone (possibly in Russia) performing continual hacking on the forums.
Personally, I find it sad when sites such as mine provide free information, services, and software, and some hackers just have to try to take it down.

-eof-

 

Filed under: TerreSculptor No Comments
18Dec/13Off

Multi-Threaded Part 2

I recently assembled a 16-core AMD Opteron based workstation for use as a software test system.

The relevant hardware includes:

- AMD Opteron 6376, 16-Core Server Processor, Socket G34, 2.3GHz base, 16MB L3 Cache.
- 8GB Memory in 2×4GB PC3-12800 DDR3-1600 CL9-9-9-24 (note: running dual channel).
- SuperMicro MBD-H8SGL-O ATX Server Motherboard, Socket G34, AMD SR5650 chipset.
- Microsoft Windows 8.1 Enterprise 64-bit.

The architecture of the Opteron 6376 is as follows:

- The 16-core processor is split into two banks of 8 cores.
- Each 8-core bank has its own 6MB L3 cache.
- Each 8-core bank is then split into 4 sets of 2 cores.
- Each 2-core bank is sharing a 2MB L2 cache and a 64KB instruction cache.
- Each core then has its own 16KB L1 cache.

The Opteron 6376 has a variable clock frequency dependent on the number of cores in use:

Core count  -  16
Idle frequency  -  1.4 GHz
Base frequency  -  2.3 GHz
Turbo frequency (more than 8 cores)  -  2.6 GHz
Turbo frequency (8 cores or less)  -  3.2 GHz
System bus speed  -  3200 MHz
HyperTransport  -  6.40 GT/s

 

Multi-Threading Performance

Note that the main point of this test is scalability of the software when adding more cores, and not a comparison to other processor hardware.
The test was ran multiple times and the results averaged.
Note that using quad channel memory will probably improve these scores as this test is heavily memory dependent for the large noisemap, and since memory is a shared resource for all threads, the faster it is, the better.

Heightmap dimensions of 4096×4096
Gradient Noise Preset: Eroded Rivers

1 thread = 38808 ms = 38.808 seconds
175ms variance, 3.2 GHz

4 threads = 11515 ms = 11.515 seconds
36ms variance, 3.2 GHz
3.370 times faster than 1 thread

8 threads = 6849 ms = 6.849 seconds
329 ms variance, 3.2 GHz
5.666 times faster than 1 thread

16 threads = 3702 = 3.702 seconds
135 ms variance, 2.6 GHz
10.483 times faster than 1 thread

For comparison from the previous blog post:

i7-2600K 3.4GHz Quad Core HyperThread
8 threads = 4890 ms = 4.89 seconds

Results graphed:

 

Conclusion

TerreSculptor noisemap multi-threading on this processor when increasing from 1 thread to 16 threads has an average performance improvement from 38.8 seconds to 3.7 seconds, which is more than 10x faster.

While it does not achieve a true 16x performance improvement, due to the lower clock frequency with 16 cores running, the shared memory access and processor cache access delays, etc., the more than 10x performance increase is a significant productivity boost and easily noticeable when using the software.

Future builds of TerreSculptor will also include multi-threading code improvements to the Modifiers and any application sections than can utilize and benefit from more threads.

Note that the next build of TerreSculptor is almost ready for public access.

-eof-

 

Filed under: TerreSculptor No Comments
27Nov/13Off

Multi-Threaded

The next TerreSculptor build release will be available soon.
I had estimated for the end of November, but it looks like it will be sometime around the first or second week in December.  As usual, I have coded significantly more than what was originally planned for the build release.

This next build release will include the following features and more:

1.  The ability to open and save a TerreSculptor project file.
Please note that the initial file version will be limited in what application areas are saved to the file, such as the current Grid and Viewport settings, etc.
Additional application areas will be constantly added to the project file format over the next months of development, including the upcoming Terrain Stack which is where a lot of the application's future capabilities will be implemented.

2.  The ability to open and save Noisemap Generator settings to file.
Have you just created an awesome Noisemap?  Now you can save the settings to a file and open it again whenever you need.
The ability to add to and manage the dropdown list of Noisemap Presets is coming soon.

3.  A new Hold-and-Swap buffer for all Noisemap Generators.
A click on the Hold toolbar button will grab a copy of the current Noisemap parameter settings into the Hold Buffer.
You can then perform a few tweaks and adjustments to the Noisemap without losing the original settings that are in the Hold Buffer.
Then clicking on the Swap toolbar button will allow you to toggle between the current settings and the settings in the Hold Buffer for visual comparison.

4.  All Noisemap Generators are now fully multi-threaded.
The threading code design is not limited to a specific thread count and fully supports unlimited threads.

The imposed thread-count limitations are:
- The number of supported processors in the installed version of Windows.
- The number of threads supported by the installed physical processor(s).
- The number of lines in the Noisemap resolution (the length dimension).

A computer with a quad core processor and a noisemap that is equal or larger than X×4 resolution can use all four threads.
A computer with a quad core hyper-thread processor and a noisemap that is equal or larger than X×8 resolution can use all eight threads.
A computer with two xeon octal core hyper-thread processors and a noisemap that is equal or larger than X×32 resolution can use all thirty-two threads.

The use of more threads when executing algorithms is typically not a linear gain.  In other words, having twice as many threads does not always result in an execution time that is one-half.  The running threads have to share certain system resources such as access to memory.  So gains are typically slightly less than the total number of threads.

Intel processors with Hyper-threading do not supply the same throughput on a hyper-thread as they do on a core.  Typical performance benefits for hyper-threading are gains of 10% to 20%.  So a quad core hyper-threading processor will typically only see between 4.5× to 5× the performance increase with all 8 threads.

There is no benefit to specifying more threads than there are physical cores plus hyper-threads available.  Typically doing this results in negative performance improvements due to thread context and switching overhead.

I have an AMD Opteron 16-core system on the way and will post threading statistics on it in the very near future.  An Intel Xeon dual 8-core Hyper-thread system will be purchased in the near future for additional threading tests up to 32 threads.  These systems have to be workstations since many servers do not support the installation of a gaming-level PCIe x16 video adapter required to run TerreSculptor.

Multi-threading capabilities are also planned for many of the TerreSculptor Modifiers, Extractors and other Devices.

Here are some Noisemap Generator results using one of my systems:

Intel Core i7-2600K @ 3.4GHz 4-Core Hyper-thread (8 thread) Processor
16GB Mushkin Redline DDR3-1600 CL7 RAM
ASUS Sabertooth P67 Motherboard
Windows 7 Professional 64-bit
Heightmap dimensions of 4096×4096
Gradient Noise Preset: Eroded Rivers

1 thread = 26330 ms (26.33 seconds)

2 threads = 13190 ms (13.19 seconds) (Task Manager 25%)

1.996 times faster (199.62%)
Only 50.09% of the time that was required for a single thread execution.

4 threads = 7780 ms (7.78 seconds) (Task Manager 50%)

3.384 times faster (338.43%)
Only 29.54% of the time that was required for a single thread execution.

8 threads = 4890 ms (4.89 seconds) (Task Manager 100%)

Note that this will be 4 cores plus 4 hyper-threads.
5.384 times faster (538.45%)
Only 18.57% of the time that was required for a single thread execution.

-eof-

Filed under: TerreSculptor No Comments

Recent Posts

Categories

Links