The Programming Behind Cel Shading

As the inquisitive reader may have guessed, or simply, googled my name, I hail from Persia, where Prince of Persia (as most people here put it, pe-rance of per-sh-ee-a)  games and the movie are set. Some people complained that Persians are brown, and the fact that in the movie had hasted a Nordic actor to play the role of the Prince was rather insulting and racist, however, what these warriors choose to ignore is that Persians, and most other Iranic tribes at the time, such as Parthians (Parthia, where I live) and Medians were, in fact white. Let’s not beat around the bush here. For nationalistic reasons, I never played the reboot, or the trilogy, I just played the 2D platformer on a Sega Genesis emulator. Anyways, I thought the reboot was the first instance of cel shading in computer games, however, I was wrong. Ten years before, a Dreamcast game by the name of Jetset Radio (pictured above) had set the trend. I’m too young for Dreamcast and I’ve never seen one even. But it must have been helluva game judging by the videos I’ve seen of it.


Legend of Zelda: Skyward Sword, one of the better games using cel shading

But what is cel shading or as Graham Sellers puts it, cell shading [sic]? It’s basically the process of using tricks in the render pipelien to make everything appear as if they were drawn by hand. The alternative approach for naming this technique is toon shading.


Cinema 4D’s “Toon” Effect

To find out which games use this effect, I headed to my trusty hangout,, a wiki created using PMWiki and serving astray, bored media lovers for years… and in its cel shading page, it said:

Cel Shading applies first and foremost to the way the lighting is rendered.

This layman explanation is exactly, how I would describe it. Indeed, cel shading is rendering of the diffuse light channel. Texturing, and rendering. I know that light is a component of the material, and not something to be rendered, but using LUTs, we can achieve exactly this.

Anyways, what is  LUT?



A shot from Tintin’s Assets… I really don’t know what a plastic shader is. You could fill a data center with the things I don’t know!

Color LUT

If you have played around in Da Vinci Resolve, you’ll know what LUT is. It’s short for Look Up Table. Imagine you wish to change a series of colors to another series of colors. Each color must correspond to another. This is where Color LUT comes into play.



In OpenGL, each LUT is a 1D array that corresponds to a number between 0.0f and 1.0f. This number is the intensity of the diffuse component of the Phong lighting system (remember the last post? Phong lighting is different from Phong material). Imagine this, if you will.


\vec{N} is normal vector, \vec{L} is the light vector. the LUT maps each color to each intensity using the formula. \alpha is dependent on your code.

 Let’s put it all together and see what happens.

Note: These are taken from OpenGL Superbible 7ed, by Graham Sellers.


Cel Shading in OpenGL

First, our front-end, at least, a part of it:

Listing 1: OpenGL Cel Shading Front End


//declare our LUT
static const GLubyte toon_tex_data[] =
    0x44, 0x00, 0x00, 0x00,
    0x88, 0x00, 0x00, 0x00,
    0xCC, 0x00, 0x00, 0x00,
    0xFF, 0x00, 0x00, 0x00

glGenTextures(1, &tex_toon); //Generate texture
glBindTexture(GL_TEXTURE_1D, tex_toon); //Bind texture, a 1D texture
glTexStorage1D(GL_TEXTURE_1D, 1, GL_RGB8, sizeof(toon_tex_data) / 4);
glTexSubImage1D(GL_TEXTURE_1D, 0,
                0, sizeof(toon_tex_data) / 4,
                GL_RGBA, GL_UNSIGNED_BYTE,
                toon_tex_data); //Pass the data
glTexParameteri(GL_TEXTURE_1D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); //Conditions

Note that all the codes are explained in comments, and this is an introductory article, and not a tutorial, so I don’t see fit to explain the code in detail. You can buy OpenGL Superbible and see for yourself.

Listing 2: Partial of Cel Shading Vertex Shader


#version 420 core

uniform mat4 mv_matrix;
uniform mat4 proj_matrix;

layout (location = 0) in vec4 position;
layout (location = 1) in vec3 normal;

out VS_OUT
    vec3 normal;
    vec3 view;
} vs_out;

void main(void)
    vec4 pos_vs = mv_matrix * position;

    // Calculate eye-space normal and position
    vs_out.normal = mat3(mv_matrix) * normal;
    vs_out.view =;
// Send clip-space position to primitive assembly
    gl_Position = proj_matrix * pos_vs;

And finally, our fragment shader.

Listing 3: Cel shading fragment shader.


#version 420 core

layout (binding = 0) uniform sampler1D tex_toon;

uniform vec3 light_pos = vec3(30.0, 30.0, 100.0);

    vec3 normal;
    vec3 view;
} fs_in;

out vec4 color;

void main(void)
    // Calculate per-pixel normal and light vector
    vec3 N = normalize(fs_in.normal);
    vec3 L = normalize(light_pos - fs_in.view);
// Simple N dot L diffuse lighting
    float tc = pow(max(0.0, dot(N, L)), 5.0);

    // Sample from cell shading texture
    color = texture(tex_toon, tc) * (tc * 0.8 + 0.2);

If we load a donut model into the program, this is what we’ll get:


Well, hope you enjoyed it! I’m currently learning After Effects. And I may write a book about it. What do you think?

Leave a Reply

Your email address will not be published. Required fields are marked *