As part of my brain dump series on writing creative tools off the back of Substance 3D Modeler and Dreams I want to talk about technical innovation and the importance of your Tech Stack for Virtual Reality productivity and creativity software. So let’s talk about the speed of light and Godot.
Conclusion at the top to save you time
- Speed of Light - What are the physics / hardware constraints
- Design for the possible, performance is a feature
- Owning your tech stack makes innovation possible
- I suggest using Godot as the seed to grow your tech from
- Ownership comes with responsibility, tools are forever.
Speed of Light Thinking
Something that was super common on the Dreams team and also underpins a lot of decisions on the Modeller team was speed of light thinking. Alex Evans was the one who really got me thinking this way. Long before joining Mm, when I was still in Sony networking division he gave a talk at an internal conference laying out the thinking for a graph database which was single threaded motivated by this article about the cost of multithreaded smart code on cloud hardware vs dumb cache friendly code on an old laptop. Spoiler alert: Computers are fast and the laptop, when not burdened by complexity, could do the graph calculations a lot faster.
I look at this chart a lot, Latency Numbers Every Programmer should know.
This thinking was in a lot of PlayStation which led to a total rewrite of the IO pipe leading the PS5 to disk transfer speeds of 5.5 GB/s. Meaning a cold read of a database of 5 gig or less should take you less than a second. This thinking required looking at what was theoretically possible and breaking down the problem at each level. Thinking in terms of the speed of light.
Likewise there is another great talk from Grace Hopper which I referenced long before Alex introduced me to how fast disk io could be. Why networking is NEVER fast.
You can’t fight physics. Though at the same time computers are STUPID fast and when you re-examine base assumptions using modern data points you can write software which was not possible a decade ago. When Medium was written back in the day it had very tight boundaries. Early days of when Modeller was being concepted Anton looked at the scene graph and wanted to be able to build a tree with each leaf individually modelled. Which works out to about 10,000 leaves. Turns out most scene graphs are horribly slow but if you think about scene graphs from first principles with instancing and counting every bit, then it turns out you can make them blindingly fast.
This speed of light thinking requires you have a technical stack which is capable of being moulded to new concepts.
Edit Notes
Alex was kind enough in this tweet to point me at the original article:
Scalability! But at what COST? by Frank McSherry from 2015. There are some great follow ups as well.
Anton had a correction and more info in this tweet thread. Pasted below for readability.
I had a quick look to see if I can find that article, but no dice :( I'll try again later. What I remember that might help though:
- It was about performance of running social network graph algorithms (the fb one i think) on a laptop vs in the cloud
- I'm pretty sure it hit HN
Also it's "100k things" (leaves, bricks, etc) not 10k :) (and it's ballpark, a quick google reminds me that an average tree has ~200k leaves heh)
I find the "ops per byte" back of the napkin analysis here useful to get "into the ballpark". Assume ~20Gb/sec bandwidth on a typical CPU. Divide by 60 or whatever your fps target. That's how many bytes you can "touch" per frame. Then see if that seems like "enough" for your task
Next, figure out how many "ops per byte" you think you'll need to do. Know how many "ops per byte" you can do on your given CPU/GPU without being slower than memory (on PS4 GPU for example, it's 5 ops per byte iirc). If you're above that, you're cpu bound, below, mem bound.
For the case of moving transforms (ie 100k leaves), it turns out to be slightly cpu bound but not by much. It's about 2-4x vs just touching the mem. You can do experiments to figure this out. So from that, given your individual instance size, you know how many you can do.
It's much more than people expect :)
So if you're doing less than that, you should probably know why (as in, what are you getting in return). Not all apps need to go full rate, but good to be conscious about the choice to "make it easier to code" or "being safer" or whatever.
What about my own engine?
If you have the technical backing and more importantly the runway to create your own tech then that is great. Though I think you would be silly not to build on top of the shoulders of giants. Pull in any MIT or similar licensed tech when you can. Use it, improve it, delete it or rewrite it. The point is you don’t have to do it all day one.
Dreams went through multiple engine rewrites from day 1 to shipping and effectively went 10 years without producing Sony a dime. Modeller was built on top of Medium tech and even then was a large investment from a corporation the size of Adobe to rework and upgrade it.
Do I think small indie teams can compete in the creative/productivity space? Hell yeah! (future article)
Do I think they should build their engine from scratch? FUCK NO!
My suggestion: Use Godot as the technical seed to grow your XR application from.
Godot isn’t... X
Is it Unity or Unreal? No.
Out of the box perf is lower than Unreal and Unity.
There are shiny toys missing and the community, while powerful, is still growing. Tutorials and talent pools are smaller. If you are a 3d game developer with no internal programmer support would I recommend it over other engines? Probably not. Though if you have even one decent coder and you will need that if you are building tools. Then yes it is the right option.
Godot isn't speed of light
Yes I agree because it's general purpose. By design it is make to be approachable and extenable. Juan, a founder, outlines what he believes the pillars of Godot are in this tweet. That base gives the perfect stock car to rip apart and boost up for whatever drag race you specifically need to perform in. Seed thinking.
I don’t want to use GDScript
Well you don’t have to, I know devs that are 100% C++ but honestly it is a really nice speed boost. I would strongly recommend looking into it, working with it and would advise against the C# side. C++ or GDScript are ideal. C++ for core and systems, and GDScript for speed of development and prototyping.
It doesn’t do X?
Well there is likely an addon or plugin that will do it for you if it’s not core render tech say. Also you can build it yourself.
Dedicated XR Engine
This is true it is a general purpose engine but it’s XR implementation is one of the best in a general purpose stack. Also it’s easily exposed to be improved for future applications. Though I do want to be clear, dedicated XR tech stacks are the future. Again a future article. Though Godot is a great seed to build yours from.
It is unknown
Honestly the biggest hurdle is funding, and talent pool wise at the moment. I get it, but this is a temporary problem which is getting solved. I remember the shift when funders used to value internal tech, then the shift as they saw it as a risk. The pressure to abandon internal tech for middleware and now recently the business risk of being exposed to middleware. This will change, investors are getting more familiar with Godot and its position in the market.
What is Godot!
There are other options but at this point Godot 4.3, congrats to all on the release, is a robust MIT licensed engine written in clean C++ with a Vulkan renderer which runs on Quest and PCVR with strong support for a variety of features like multi window which app devs need that game devs don’t often support well.
It is not the most performant engine but it is easy to break into parts, replace parts and build on. The engine is very easy to understand with readability and re-workability key design features from the team.
Long term I think most productivity or creative tools end up building a lot of their own tech. Be it voxel based, mesh processing, stroke api or even text processing libs. As such you want to build it fast and portable so C++ is an obvious choice here and unlike Unity which has been many small companies goto choice for productivity VR it is FREE and a great seed from which your internal tech grows. Any investment you make into the technology is 100% yours. You may choose to push some of it back into the community but you are under no legal obligation to. That is massive for business.
Also the web side of Godot had a hiccup, too long to discuss but you can read here that they are fixing. Once fixed it is the perfect WebXR basis as well for more accessible situations. Like museum showcases or marketing pushes.
Edit Note: As of 4.3 single threaded option is there but this article is still worth the read about why the problem exsists.
Lots of companies are keeping it secret. Look at the lovely Cozy Blanket.
The only reason we know it’s Godot based is from this tweet
Because again they are under no obligation to share that and I’m pretty sure Uniform is built on it as well.
Godot is hands down the best OpenXR implementation I have seen and because it's all open source it’s really easy to add in support for OpenXR extensions and features not yet shipped. Just last week I added support to the engine for Logitech’s new Mx Ink pen over the course of a lunch break. Honestly take a look at this pen, I want to do a whole another post on creative inputs and this pen is a big part of it.
How to Extend Godot: Addons, Modules and Fork
In many small studios where we maintained our own tech stack the plan went as follows: Fork the internal tech stack, build and hack what we need for the title in development and then at the end of the project, or other milestones like another project due to start, we would examine what could be pulled back into the main branch tech engine.
Additionally sometimes games would cherry pick from each other. This is from an era when porting to new hardware was common, working with other studios' tech stacks was frequent and standardisation was rare. This is a good framework to think around extending Godot for games but tools are forever products.
My personal recommendation is ignore Godot Plugins development unless you're making it for use outside of your studio. Modules are great c++ development method for studio or game specific logic. It builds directly in the engine, and isolates it so that merge pain with Godot development.
The next leap is going from adding to using Godot as a seed. As you head into production fork Godot and take ownership. This means you have to fix all future bugs. You might want to pull in developments from the main branch but if you want to own the tech stack, control regressions and invest in tech then it comes with ownership of the stack and starting to think about the speed of light.
Conclusion
As before I put it up at the top but I will repeat it here.
- Speed of Light - What are the physics / hardware constraints
- Design for the possible, performance is a feature
- Owning your tech stack makes innovation possible
- I suggest using Godot as the seed to grow your tech from
- Ownership comes with responsibility, tools are forever.
Have a banana for your potassium 🍌