vtable And Other Monsters
Casting an object to another one or calling a virtual function are probably two of the most common operations you can think of in C++, so it is a shame that such simple operations are not fully understood by everyone.
I have been programming in
C for many years, and one of the benefits of
C is that there is no real magic: when you define a structure, you can see the layout in memory, you can feel the pointers being padded, and every other operations, such as casting a pointer to another one (something you should only do with great care, by the way) are just re-interpreting memory addresses from one type to another one or shifting addresses a bit. No magic. No surprises. Accessing the second member of a pointer array ? This is just basic pointer arithmetic (indirect move with shifting and index, something that even the venerable
68000 processor had if my memory is correct)
Some people go straight in
C++ (or even in
Java) when they start learning programming – which is in my point of view a bad thing – and tend to lose their focus on what’s really happening behind the curtains (ie. inside the memory)
But things are generally much simpler that you thought.
Back To Basis
Let’s first have a look at a very simple object. A C++ one:
1 2 3 4 5 6 7 8 9
This class contains only a constructor, and has one member (an integer). These two things are not of the same nature: variable members and functions are of different kind (yes, this is a really smart and insightful remark, I know).
To make things clearer, we can view this class as a C-structure, and its functions as global functions taking a (hidden) “this” member as first argument (the pointer to the structure itself). The instance members can be accessed implicitly, without the
this-> pointer indirection, by the way: C++ will add the missing bits.
The code above can be rewritten in pure C as follows:
1 2 3 4 5 6 7 8 9 10
… and, rather that just declaring:
… you’ll need to call the constructor manually:
Here it is! C++ in plain C! Of course a C++ compiler will hide everything behind the curtains when using regular classes, but the actual compiled code will be more or less the same.
If you add more members and/or more methods in this class, the principle is the same: extend the structure layout, and/or add more global functions that take an hidden this first argument.
Quizz: what is the size of a
class A object instance ? The answer is pretty simple: the class only contains one member (
id), which is an integer (32-bit here) ; the
sizeof(A) is therefore 4 (bytes). No magic.
Quizz: what will be the size of a
class A object instance if a regular member function is added ? The answer is pretty simple, too: its size won’t change, as it still contains only one member – only an additional global function will be added in the code.
What About Inheritance ? This Seems a Bit More Complicated!
No, no, not at all.
Let’s take a simple (yet representative) example: our previous A class, another B class with a single “age” member, and at last a final class C with a member “mode”, but inheriting from A and B.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Instantiating an object from C (
C c;) will print something like:
1 2 3
class C constructor called first the upper inherited classes constructors (
class A constructor and
class B constructor), and therefore prints their information first. Here again,
C++ hides everything behind curtains: the calls to base classes constructors are generated automatically, before your actual
class C constructor code.
You can note that the address seen by the constructors of A and C are identical, but not the one seen by B. How can it be ?
Let’s rewrite everything in plain C:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
struct C layout is pretty straightforward: the first element is the
struct A members – the reason why constructors from class A and class C did see the same address, the second member is the member(s) of class B, and finally comes the member(s) of C.
In both C++ and C variants,
- the size of C is 12 bytes (the three integers)
- the location of A within C is at the beginning (the first four bytes)
- the location of B within C is after the area dedicated to A (the next four bytes)
- the location of members specific to C then follows
Casting will provide the same behavior, by the way:
Okay, But Virtual Functions ? Can You Do That In Plain C ?
Well, let’s consider a simple example: simply add a virtual function
cast the high-level
class C instance into
B*, we’ll see the same behavior.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
If we create an instance of
C c;), the following calls will produce exactly the same output:
Where is the magic ? Let’s first have a look at the output, when creating this object:
1 2 3
We can see that
B are not anymore 4 bytes, but 16. And C is 32 bytes! Somehow, the
C++ compiler did add some bytes in our objects without telling us!
Let’s have a look at each object instance actual contents within each classes, by dumping data as pointers in constructors, to figure ou what has been added inside our beloved
1 2 3 4 5 6 7 8 9
print_object(__FUNCTION__, this, sizeof(*this)); inside each constructor.
Remember that the constructors are called as follows when creating an instance of C:
- base class A constructor is called magically (
thispoints to the beginning of the allocated C structure)
- base class B constructor is called magically (
thisstarts at 16 bytes from the beginning of the allocated C structure)
- top level C constructor user code is executed (
thispoints to the beginning of the allocated C structure)
Here’s the result:
1 2 3 4 5 6 7 8 9 10 11
We can see that:
- A contains an unknown additional pointer (
0x400f20), and the id variable (
42) padded to 8 bytes
- B contains another unknown additional pointer (
0x400f00), and the age variable (
7) padded to 8 bytes
- C embeds A and B, and the mode variable (-1): these are seen as
00000007for the age variable, and
fffffffffor the mode variable:
0xffffffffin two complement’s world) – and the two additional pointers
So it seems that the structure size change reason is that an hidden pointer member (8 bytes in the 64-bit world) has been added to the class instance – and the compiler probably padded the structure to 16 bytes, because you want to have 8-byte alignment when dealing with arrays: if you have an array of two objects, the second object hidden pointer has to be padded to 8 bytes, enforcing the structure to be a multiple of 8 bytes. The 4 bytes used for padding are unused, but useful to maintain the alignment.
Hey, wait! The two pointers are different, when printed inside the C constructor! Yes, good catch: the additional A pointer which was
0x400f20 inside the
class A constructor has been changed into
0x400ed0 inside the
class C constructor, and the additional B pointer which was
0x400f00 inside the
class B constructor has been changed into
0x400ee8 inside the
class C constructor.
Oh, and a last note: C is only 32 bytes.
- there is no additional hidden pointer for C!
- the compiler apparently used the padding area of
class Bfor the
modevariable (smart, isn’t it ?)
Okay, so two magical pointers have been added because we defined a virtual function. What does it mean ?
Let’s go back to basic: how would you implement this class hierarchy in pure C ?
Here’s what you may do:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
Okay, looks more or less what we had previously. But what about the virtual function ? We have to be sure that we call the right function, even if we only have a A* or a B* pointer (and we don’t know that this is actually a C object class behind).
A solution can be to add a function pointer for every virtual function in each classes, and initialize inside the constructor!
Something like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
What did we do here ?
- A and B constructors initialize their versions of
- C constructor initialize its own version of
Neat, isn’t it ?
Now, whatever pointer you have, these three lines will produce identical behavior:
Hey, by the way, we do not need an additional pointer for C, because we already have the correct version in two locations:
b.print. Let’s remove this useless pointer in C (and the useless initialization), and use:
There is a little problem, however, when using a C instance cast in A* or B*. When you call
a pointer is of type A*. And therefore the assignment
this->a.print = C_print will produce a warning. Similarly, when you call b->print(b), the
this pointer will have a type of B*, and not only it is of the wrong type, but it has a different address, as c->b is located sixteen bytes after the beginning of C.
We need specialized versions with correct casts!
- casting from A* to C* is straightforward: we know that the address is the same, and the type can be safely cast
- casting from B* to C* requires to adjust the address to find the beginning of C, and a cast (B is in the middle of C, remember)
Here we go:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
With these adjustments, things are smooth:
c.a.print(&c.a)==> I am C(42, 7, -1)
c.b.print(&c.b)==> I am C(42, 7, -1)
c.print(&c)==> I am C(42, 7, -1)
Perfect, isn’t it ?
Not yet: what if we add more virtual functions in our structures ? This will increase the object size, and require more pointer initialization in constructors! What a waste of space and time: we are assigning the same pointers each time. Could we prepare the set of pointers for each classes in a static, global structure, and just initialize a pointer to these static data inside the constructors ? Neat, isn’t it ?
Here’s the modified version – let’s call the global structure tables containing specialized function pointers
vtbl, as in virtual table, because this is actually what they are: table for virtual functions!
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87
Neat, isn’t it ?
Well, I’m telling you a secret: this is more or less what the C++ compiler does (yes, behind the curtains) when implementing virtual functions. The virtual function tables are called virtual tables, and are static structures inside the compiled code. And inside each corresponding class constructor, additional code is inserted just between base class initialization calls, and user code, to set all virtual table pointers corresponding to the class type.
Some additional remarks on this design:
if you call a virtual function inside a base class constructor, you will not call the top-level function version, because the virtual table pointer has not yet been initialized (it will be initialized after the end of the base class constructor)
you need one static structure containing all necessary stuff for a given class
you can put additional information on these virtual table structures, which can be useful!
- the class name
- type information functions (what is this object’s real class ?), for example by implementing a hidden
- dynamic cast support
… and this is exactly what most compiler do, actually: Runtime Type Information data is generally included as soon as a virtual table needs to be created for a given class (ie. you can use dynamic_cast<>())
Now you know what’s going on when you define a virtual function: rather than calling the function directly, an indirection through a virtual table is done.
This has some drawbacks:
a little function call impact on performances, because you need to read the actual function pointer in a static table
a potentially huge impact on performances because you can not inline anymore function calls
A Deeper Look
Let’s dig a little bit more, using GCC version of the virtual tables.
If you run your program under GDB and examine an instance of
class C, you’ll see:
_vptr.A = 0x400e70 and
_vptr.B = 0x400e88 : these are the virtual tables.
What is behind these pointers ? Recent GCC releases allows you to dig a bit into an object’s virtual table:
1 2 3 4 5 6
Otherwise, you can easily guess that each vtables starts with the same kind of function array we created for our C version:
1 2 3 4 5
Now you understand a bit more the concept or vtable, let’s see a dirty-but-fun hack (valid for GCC, possibly for Visual C++ too): how to override a specific object’s virtual function table, and rewire virtual functions at runtime
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
And yes, your object was hacked:
1 2 3
Neat, isn’t it ? (yes, yes, not really portable though)
For Windows coders, the Reversing Microsoft Visual C++ Part II page will be a goldmine, too
TL;DR: remember that C++ used to be enhanced C a long time ago, and most basic features can be easily implemented and understood through plain C code!