Today, I would not recommend anybody to go into graphics programming:
I started in 2001, when NVidias first Geforce 1 ("the Gigatexl shadercard") was first announced:
The field developed since then with so much speed and innovations, it blows my mind of. Compared to what we could do 25years ago, the tech today is just fu*ing impressive.
Though, with this impressiveness comes a big "but": The space is developing at a speed which is really really scary. Nvidia came up with AI-based effects to influence scene & assets on their own - back then, we wouldnt have even thought about that this will be possible some day in realtime.
I do not know if its possible at all to be a "decent pro" in this field now - let me use other words: "Where is todays Jon Carmack?" - he was famous for squeezing everything out of the hardware, using ideas very hidden in the community etc. - today, there is not any competitive moat for people like him (he actually lives on his legacy), and that is because the field is so vast and evolving so fast that there is no chance to become the next one
I really dislike people that got into a thing and then try to discourage others. “Don’t be like me! I wasted my entire life” which is bullshit from a jaded person that lost passion. Telling people to stay away from graphics programming is not how to entice tomorrow’s John Carmack.
So here’s another perspective. If all you have done is web apps and Kubernetes, for example, do get into graphics programming. The feedback cycle is exhilarating, and you get to appreciate how mind boggingly fast your average computer is. You’ll get to optimize things that are ultimately unimportant because you have never learned how quick things are at the low level. There are a ton of resources and the maths is not too bad. You might find that 3D modeling is a creative outlet you didn’t know you needed. Even if completely inapplicable to your day job, you’ll find new ways to appreciate the art of programming computers, and might just decide to never touch Kubernetes again and spend the next 5 years writing your own game engine in your spare time. There are a lot of crazy people like that, and the community of hobbyists that are not ground down by life and game dev as a career is just incredible. The Graphics Programming discord is a welcoming place if you want to check it out.
Graphics programming has this one very, very useful aspect, exponentially more valuable today: the matrix algebra pipelines, and then the requirement to 'think in matrix transforms' is a wonderful and visually engaging way to get your foundation for machine learning math.
I don't really see this with modern graphics programming, but I was highly amused that my 1980s-1990s graphics skills (in particular, coordinate transform math) were very useful when I started working in robotics in the 2010s-2020s (because forward and inverse kinematics are exactly the same thing as 2d/3d projections.)
The trick there is that they both have related physical analogs, and machine learning math doesn't really (in that while you can visualize them spatially, it doesn't seem to help solve any problems in that space.)
I mainly learned linear algebra via hands-on 3D graphics, and have a hard time thinking about a matrix as anything other than 4x4 and representing a linear transform...
How much do you even think about explicit matrix math when doing high-level ML?
I'm a data scientist and not a graphics programmer, but my guess is that it's just abstracted away more. If you're using ML/DL libraries, you're mostly just calling APIs that handle the linear algebra and calculus for you. Unless you're actively contributing to those libraries, you likely don't ever need to "touch" any of the underlying operations. Up to a point, it's useful to understand how things work under the hood, but where that point is kinda depends on your job. For instance, I could write code to do 'naive' matrix multiplication, but I couldn't, like, contribute to BLAS.
JC was a bit of an anomaly but also his image is mostly coming from players and journalists. Developers struggled to use the later id software engines (partly why UE won that war).
You don’t need to be JC to earn a decent living as a graphics/game programmer.
It's also a great way to not become the next "graphics programming legend" --I think a fast-moving field with lots of new developments is actually an exciting place to be a pro.
Feels like we try to turn anything we do into a career or job, especially with the odd ML angle. How about you "do graphics programming" instead of "being a graphics programmer"? Like start doing simple stuff until it clicks and you see it for being logistics to the GPU, then you can layer on top all the crazy concepts. Its like a small mountain you climb and suddenly everything clicks and you think like "oh my"... the possibilities and things to experiment with...
I would say being a long-time user of Photoshop and Blender helps a lot. It's not a main tool, but supplemental. Maybe AI will take over some of this though.
Why outsource my learning to Al? The whole point is the joy of the process. I could easily take a photo of a scene (since the inception of photography) instead of painting it, but people still paint. I'm tired, boss... I yearn for a past when we didn't have to end every conversation with a disclaimer about Al taking over.
Somewhat surprising there is no mention of basic design principles, or understanding the quirks of human perception. My brother was a production artist for some well-known computer games in the '90s-'00s, and continually complained about programmers and managers with zero visual sense, or curiosity about understanding the artists' side.
Graphics aren't my specialty, but as a musician, sound designer and producer, by far the most effective/influential audio DSP coders I'm aware of understand the basics of music, the physics/acoustics of sounds, and the gotchas at the interface between discrete digital processes and how we perceive and interpret stimuli.
I was a technical artist for a series of feature films during the early '00s. At a good studio they'll have art and design classes for the tech origin staff and scripting and bash classes for the art origin staff. I was both, and that was a ton of fun.
Exactly, Technical Artist is a distinct position that normally bridges the gap between pure programmers and artists and their needs. All TAs I've ever worked with had this incredible skill of knowing exactly what tech thing they need to achieve the outcome that the artists want.
This applies outside of creative industries too. I've seen my fair share of B2B/enterprise software where its clear the vendor has no clue how the industry they are selling to works, or how the users of that software think.
AI changed the calculus a bit (or at least, it has the potential to) but I think that was a huge part of the whole "learn to code" movement in the mid 2000s, to start treating software development as a "feature, not a product" of existing experts in their field so that the people most familiar with their domain are actually the ones making the software instead of having to translate the requirements down to a dev team.
Think more code monkeys for enterprise software consultancies, like Accenture, Tata, IBM Global Services, etc.
They needed warm bodies for their projects, as the usual source of manpower was grinding leetcode to work on bigtech at salaries that would make an accenture business type vomit in disgust.
trigonometry->Coordinate Geometry -> Linear Algebra applied to graphics
Once you have that intuition, the rest is all figuring out the stages of the graphics pipeline and the frameworks like opengl and their constituent data structures.
The most useful resources I've found for graphics are scratchapixel, UC Davis' graphics lectures, songho's articles, and Essential Math for Games and Interactive Applications. I highly recommend you read this last resource front to back. Seriously, its the best freaking math reference for graphics out there.
But knowing theory is not sufficient. You also need to get your hands dirty by writing code: learn how to build a software rasterizer (check out Tiny renderer) and a ray tracer (Ray Tracing in One Weekend series). Preferably in a language like C++. Then move onto APIs. I recommend you learn OpenGL, but if you're okay with being confused as all hell try Vulkan. Or WebGPU if you're a hipster (/s).
Finally, try to build some stuff. A simple engine. A non-trivial technique. A game. Whatever.
Unfortunately, you're unlikely to get hired working as a rendering engineer without having serious connections, or by having adjacent experience in the industry. Doubly so now that everyone is convinced junior engineers are unnecessary.
Thank you! I started recently as a greybeard engineer, and I found SDL3 GPU to be modern enough yet not too low level for a newbie compared to Vulkan. SDL in general is a fantastic framework. And if you use it from Odin, the dev experience is so smooth and enjoyable as everything you need to create graphical applications is builtin.
I will definitely check out Essential Math for Games and Interactive Applications, I feel I need some solid understanding of theory to see how it all fits together.
Today, I would not recommend anybody to go into graphics programming:
I started in 2001, when NVidias first Geforce 1 ("the Gigatexl shadercard") was first announced: The field developed since then with so much speed and innovations, it blows my mind of. Compared to what we could do 25years ago, the tech today is just fu*ing impressive.
Though, with this impressiveness comes a big "but": The space is developing at a speed which is really really scary. Nvidia came up with AI-based effects to influence scene & assets on their own - back then, we wouldnt have even thought about that this will be possible some day in realtime.
I do not know if its possible at all to be a "decent pro" in this field now - let me use other words: "Where is todays Jon Carmack?" - he was famous for squeezing everything out of the hardware, using ideas very hidden in the community etc. - today, there is not any competitive moat for people like him (he actually lives on his legacy), and that is because the field is so vast and evolving so fast that there is no chance to become the next one
I really dislike people that got into a thing and then try to discourage others. “Don’t be like me! I wasted my entire life” which is bullshit from a jaded person that lost passion. Telling people to stay away from graphics programming is not how to entice tomorrow’s John Carmack.
So here’s another perspective. If all you have done is web apps and Kubernetes, for example, do get into graphics programming. The feedback cycle is exhilarating, and you get to appreciate how mind boggingly fast your average computer is. You’ll get to optimize things that are ultimately unimportant because you have never learned how quick things are at the low level. There are a ton of resources and the maths is not too bad. You might find that 3D modeling is a creative outlet you didn’t know you needed. Even if completely inapplicable to your day job, you’ll find new ways to appreciate the art of programming computers, and might just decide to never touch Kubernetes again and spend the next 5 years writing your own game engine in your spare time. There are a lot of crazy people like that, and the community of hobbyists that are not ground down by life and game dev as a career is just incredible. The Graphics Programming discord is a welcoming place if you want to check it out.
Go for it!
Graphics programming has this one very, very useful aspect, exponentially more valuable today: the matrix algebra pipelines, and then the requirement to 'think in matrix transforms' is a wonderful and visually engaging way to get your foundation for machine learning math.
I don't really see this with modern graphics programming, but I was highly amused that my 1980s-1990s graphics skills (in particular, coordinate transform math) were very useful when I started working in robotics in the 2010s-2020s (because forward and inverse kinematics are exactly the same thing as 2d/3d projections.)
The trick there is that they both have related physical analogs, and machine learning math doesn't really (in that while you can visualize them spatially, it doesn't seem to help solve any problems in that space.)
This is like saying being a cashier prepares you for a job in high-finance because both involve arithmetic on dollars and cents.
I've been in ML for ~5 years in multiple FAANGs and I have never seen a rotation matrix.
I mainly learned linear algebra via hands-on 3D graphics, and have a hard time thinking about a matrix as anything other than 4x4 and representing a linear transform...
How much do you even think about explicit matrix math when doing high-level ML?
Doesn’t RoPE use 2D rotation matrices ?
congrats you've found literally the only example ("the exception that proves the rule").
Is the linear algebra of machine learning more complicated than that of graphics?
I'm a data scientist and not a graphics programmer, but my guess is that it's just abstracted away more. If you're using ML/DL libraries, you're mostly just calling APIs that handle the linear algebra and calculus for you. Unless you're actively contributing to those libraries, you likely don't ever need to "touch" any of the underlying operations. Up to a point, it's useful to understand how things work under the hood, but where that point is kinda depends on your job. For instance, I could write code to do 'naive' matrix multiplication, but I couldn't, like, contribute to BLAS.
i think so
TBF, I bet any graphics programmer would be a boon for a ML shop for their GPU/performance optimization knowledge alone.
What if I just want to program some rendering engine for a game that looks like DOOM 3 and its predecessors? I think that’s still quite doable?
JC was a bit of an anomaly but also his image is mostly coming from players and journalists. Developers struggled to use the later id software engines (partly why UE won that war).
You don’t need to be JC to earn a decent living as a graphics/game programmer.
Huh? Just because you're not going to become the next graphics programming legend you think it's not worth getting into graphics programming at all?
It's also a great way to not become the next "graphics programming legend" --I think a fast-moving field with lots of new developments is actually an exciting place to be a pro.
If anyone needs a quick tutorial on linear algebra, you can check out this printabale four pager that I wrote: https://minireference.com/static/tutorials/linear_algebra_in...
I also have some notebooks with SymPy code examples here: https://github.com/minireference/noBSLAnotebooks
Good Books :)
Feels like we try to turn anything we do into a career or job, especially with the odd ML angle. How about you "do graphics programming" instead of "being a graphics programmer"? Like start doing simple stuff until it clicks and you see it for being logistics to the GPU, then you can layer on top all the crazy concepts. Its like a small mountain you climb and suddenly everything clicks and you think like "oh my"... the possibilities and things to experiment with...
I think that Khan Academy has a lot of graphics programmers that you might be interested in seeing. They use processing js. https://www.khanacademy.org/computing/computer-programming/b...
This guy has some good art: https://www.khanacademy.org/profile/kaid_1019042693170894950...
And this person: https://www.khanacademy.org/profile/kaid_9428127706426004539...
unfortunately, khan academy has been deleting old accounts: https://kap-archive.bhavjit.com/view?p=6177161966469120
1. Familiarity with all GL APIs, but deep focus on 1 or 2.
If you want to work with Windows, probably DirectX.
2. Make awesome shaders. Check this out: https://fragcoord.xyz
I would say being a long-time user of Photoshop and Blender helps a lot. It's not a main tool, but supplemental. Maybe AI will take over some of this though.
Hell, maybe that other stuff too, hahaha!
Why outsource my learning to Al? The whole point is the joy of the process. I could easily take a photo of a scene (since the inception of photography) instead of painting it, but people still paint. I'm tired, boss... I yearn for a past when we didn't have to end every conversation with a disclaimer about Al taking over.
Somewhat surprising there is no mention of basic design principles, or understanding the quirks of human perception. My brother was a production artist for some well-known computer games in the '90s-'00s, and continually complained about programmers and managers with zero visual sense, or curiosity about understanding the artists' side.
Graphics aren't my specialty, but as a musician, sound designer and producer, by far the most effective/influential audio DSP coders I'm aware of understand the basics of music, the physics/acoustics of sounds, and the gotchas at the interface between discrete digital processes and how we perceive and interpret stimuli.
There’s a separate role that is more along the lines of what you’re saying, called a Technical Artist (that’s what I do)
I think graphics programmers benefit from having an artistic mindset, but they usually work so low level that it isn’t necessary to be successful.
I was a technical artist for a series of feature films during the early '00s. At a good studio they'll have art and design classes for the tech origin staff and scripting and bash classes for the art origin staff. I was both, and that was a ton of fun.
Exactly, Technical Artist is a distinct position that normally bridges the gap between pure programmers and artists and their needs. All TAs I've ever worked with had this incredible skill of knowing exactly what tech thing they need to achieve the outcome that the artists want.
Is this a viable field for employment?or did it collapse like alot of other digital art?
This applies outside of creative industries too. I've seen my fair share of B2B/enterprise software where its clear the vendor has no clue how the industry they are selling to works, or how the users of that software think.
AI changed the calculus a bit (or at least, it has the potential to) but I think that was a huge part of the whole "learn to code" movement in the mid 2000s, to start treating software development as a "feature, not a product" of existing experts in their field so that the people most familiar with their domain are actually the ones making the software instead of having to translate the requirements down to a dev team.
the learn to code movement was a psyop by big tech to get more javascript monkeys for cheap
I doubt most JavaScript monkeys would have got through their leet code style interview process ! :)
Think more code monkeys for enterprise software consultancies, like Accenture, Tata, IBM Global Services, etc.
They needed warm bodies for their projects, as the usual source of manpower was grinding leetcode to work on bigtech at salaries that would make an accenture business type vomit in disgust.
I see this all the time with audio too. The amount of bits you need to reserve a
trigonometry->Coordinate Geometry -> Linear Algebra applied to graphics
Once you have that intuition, the rest is all figuring out the stages of the graphics pipeline and the frameworks like opengl and their constituent data structures.
Immutability. Semantics.
I'm a graphics programmer.
The most useful resources I've found for graphics are scratchapixel, UC Davis' graphics lectures, songho's articles, and Essential Math for Games and Interactive Applications. I highly recommend you read this last resource front to back. Seriously, its the best freaking math reference for graphics out there.
But knowing theory is not sufficient. You also need to get your hands dirty by writing code: learn how to build a software rasterizer (check out Tiny renderer) and a ray tracer (Ray Tracing in One Weekend series). Preferably in a language like C++. Then move onto APIs. I recommend you learn OpenGL, but if you're okay with being confused as all hell try Vulkan. Or WebGPU if you're a hipster (/s).
Finally, try to build some stuff. A simple engine. A non-trivial technique. A game. Whatever.
Unfortunately, you're unlikely to get hired working as a rendering engineer without having serious connections, or by having adjacent experience in the industry. Doubly so now that everyone is convinced junior engineers are unnecessary.
Thank you! I started recently as a greybeard engineer, and I found SDL3 GPU to be modern enough yet not too low level for a newbie compared to Vulkan. SDL in general is a fantastic framework. And if you use it from Odin, the dev experience is so smooth and enjoyable as everything you need to create graphical applications is builtin.
I will definitely check out Essential Math for Games and Interactive Applications, I feel I need some solid understanding of theory to see how it all fits together.