I've started writing a simple graphical debugger for the limited subset of C++ that Casey Muratori uses in Handmade Hero and I've already made two big mistakes. The first one is thinking that I could write a better GDB frontend than those available today, when the truth of the matter is that GDB does not want graphical frontends. The second one is using libdwarf to parse the DWARF debug information of ELF binaries.
What drove me to use libdwarf was Eli Bendersky's third article on debugging and linux (the other two talk about ptrace and implementing breakpoints) That article advises against trying to parse the DWARF section of an executable manually, arguing that DWARF is a very complex format, requiring the implementation of two "specialized virtual machines" to decode location data and line number data. I read that and thought "sure, let's give libdwarf a try".
While a cursory look at the DWARF spec makes me believe that libdwarf does, in fact, hide some of the complexity of the DWARF format, I've found that its API carries its own set of problems, particularly on the memory management side of things. The main complaint I have is that every time I make a call to libdwarf, it allocates space for the response under an opaque pointer and asks me to remember the type of the request, so that later I can ask the library to free that memory. The natural consequence of such a design is that two thirds of the code that I write to interface with libdwarf end up being spent managing memory outside of my control.
This example, extracted from the official documentation, illustrates my point (I've added the last else if branch to avoid leaking memory through the error descriptor):
void example1(Dwarf_Die somedie) {
Dwarf_Debug dbg = 0;
Dwarf_Signed atcount;
Dwarf_Attribute *atlist;
Dwarf_Error error = 0;
Dwarf_Signed i = 0;
int errv;
errv = dwarf_attrlist(somedie, &atlist, &atcount, &error);
if (errv == DW_DLV_OK) {
for (i = 0; i < atcount; ++i) {
/* use atlist[i] */
dwarf_dealloc(dbg, atlist[i], DW_DLA_ATTR);
}
dwarf_dealloc(dbg, atlist, DW_DLA_LIST);
}
else if(errv == DW_DLV_ERROR){
/* use error */
dwarf_dealloc(dbg, error, DW_DLA_ERROR);
}
}
This piece of code takes a DWARF Debug Information Entry and asks about its attributes (key-value pairs describing it, such as name, size, ...). If the number of attributes returned by the function is n, then the number of dwarf_dealloc calls needed will be n+1, one for each of the attributes and another one for the list itself.
Also, notice how libdwarf fails to keep track of the types of pointers it allocates, transferring that burden onto the caller. This means that every call to dwarf_dealloc needs to specify one among these identifiers: DW_DLA_STRING, DW_DLA_LOC, DW_DLA_LOCDESC, DW_DLA_ELLIST, DW_DLA_BOUNDS, DW_DLA_BLOCK, DW_DLA_DEBUG, DW_DLA_DIE, DW_DLA_LINE, DW_DLA_ATTR, DW_DLA_TYPE, DW_DLA_SUBSCR, DW_DLA_GLOBAL_CONTEXT, DW_DLA_ERROR, DW_DLA_LIST, DW_DLA_LINEBUF, DW_DLA_ARANGE, DW_DLA_ABBREV, DW_DLA_FRAME_OP, DW_DLA_CIE, DW_DLA_FDE, DW_DLA_LOC_BLOCK, DW_DLA_FRAME_BLOCK, DW_DLA_FUNC_CONTEXT, DW_DLA_TYPENAME_CONTEXT, DW_DLA_VAR_CONTEXT, DW_DLA_WEAK_CONTEXT and DW_DLA_PUBTYPES_CONTEXT.
That's a clear violation of the eighth item of Casey Muratori's 2004 Designing and Evaluating Reusable Components API design checklist (which, by the way, deserves a wider audience):
Use of the component's resource management (memory, file, string, etc.) is completely optional.
In order to sidestep this issue, I've decided to modify libdwarf and make it use my own custom allocator functions, effectively taking control of memory back from the library. The last revision of libdwarf (January, 2016) makes 34 calls to malloc, 10 to calloc, 1 call to realloc and under 100 calls to free. I've gotten rid of the realloc call, which was only there because of some sloppy coding, and I've replaced the rest with calls to functions specified by the user through three extra parameters I've added to dwarf_init.
My use case for libdwarf consists on parsing the whole binary in one go, building my own intermediate representation of the debugging information and, once I'm done, discarding all memory allocated through that library. That allows me to make my malloc and calloc functions use a simple stack allocator and to replace free with an empty function. Here's a barebones example of the way I use the tweaked libdwarf:
/* Compile with: clang -std=c++11 -Wl,-export-dynamic,-rpath=. libdwarf.so \
example.cc -o example -lelf */
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <assert.h>
#include "libdwarf.h"
struct MemoryArena{
uint8_t *base;
size_t size;
size_t used;
};
void * push_size(MemoryArena *arena, size_t size, bool clear=false){
assert((arena->used + size) <= arena->size);
void *result = arena->base + arena->used;
arena->used += size;
if(clear) memset(result, 0, size);
return result;
}
static MemoryArena dwarf_arena;
void * stack_malloc(size_t size){ return push_size(&dwarf_arena, size); }
void * stack_calloc(size_t nmemb, size_t size){ return push_size(&dwarf_arena, nmemb*size, true); }
void stack_free(void *){}
int main(void){
// Memory arena initialization
int bytes_to_allocate = 1024*1024;
dwarf_arena.base = (uint8_t *) malloc(bytes_to_allocate);
dwarf_arena.size = bytes_to_allocate;
dwarf_arena.used = 0;
int fd = open("inferior", O_RDONLY); // ELF target
Dwarf_Debug dbg;
Dwarf_Error err;
dwarf_init(fd, DW_DLC_READ, 0, 0, &dbg, &err, stack_malloc, stack_calloc, stack_free);
// Call as many dwarf_* functions as necessary to build
// my own representation of debug information without
// worrying about dwarf_dealloc or dwarf_finish
// Recycling of dwarf memory arena for other purposes
dwarf_arena.used = 0;
MemoryArena some_other_arena = dwarf_arena;
dwarf_arena = {};
// Code that does something useful with the debug symbols
return 0;
}
If you ever find yourself in a situation similar to mine, you can get the source of the modified libdwarf from this github repository. You will also need to install the libelf-dev package that comes with you Linux distribution. However, I urge you to reconsider using libdwarf. The obtuseness of it's memory allocation scheme is just my main complaint about it, but my dislike for its API does not stop there.
Also, remember those two terribly complicated "specialized virtual machines" Eli Bendersky mentioned? Turns out that libdwarf only implements one of them. Here's what libdwarf's documentation has to say about the other one:
6.22 Location Expression Evaluation
An "interpreter" which evaluates a location expression is required in any debugger. There is no interface defined here at this time.
One problem with defining an interface is that operations are machine dependent: they depend on the interpretation of register numbers and the methods of getting values from the environment the expression is applied to.
It would be desirable to specify an interface.