debiatan - webloghttps://blog.debiatan.net/2016-02-23T19:00:00+01:00Libdwarf with custom memory allocators2016-02-23T19:00:00+01:002016-02-23T19:00:00+01:00Miguel Lechóntag:blog.debiatan.net,2016-02-23:/libdwarf_with_custom_memory_allocators.html<!-- :subtitle: (well, one of them) -->
<p>I've started writing a simple graphical debugger for the limited subset of C++ that Casey Muratori uses in <a class="reference external" href="https://handmadehero.org/">Handmade Hero</a> and I've already made two big mistakes. The first one is thinking that I could write a better GDB frontend than those available today, when the truth of the matter …</p><!-- :subtitle: (well, one of them) -->
<p>I've started writing a simple graphical debugger for the limited subset of C++ that Casey Muratori uses in <a class="reference external" href="https://handmadehero.org/">Handmade Hero</a> and I've already made two big mistakes. The first one is thinking that I could write a better GDB frontend than those available today, when the truth of the matter is that <a class="reference external" href="http://blog.debiatan.net/a_limitation_of_gdb_frontends.html">GDB does not want graphical frontends</a>. The second one is using <a class="reference external" href="http://www.prevanders.net/dwarf.html">libdwarf</a> to parse the DWARF debug information of ELF binaries.</p>
<p>What drove me to use libdwarf was Eli Bendersky's <a class="reference external" href="http://eli.thegreenplace.net/2011/02/07/how-debuggers-work-part-3-debugging-information">third article on debugging and linux</a> (the other two talk about <a class="reference external" href="http://eli.thegreenplace.net/2011/01/23/how-debuggers-work-part-1">ptrace</a> and <a class="reference external" href="http://eli.thegreenplace.net/2011/01/27/how-debuggers-work-part-2-breakpoints">implementing breakpoints</a>) That article advises against trying to parse the DWARF section of an executable manually, arguing that DWARF is a very complex format, requiring the implementation of two "specialized virtual machines" to decode location data and line number data. I read that and thought "sure, let's give libdwarf a try".</p>
<p>While a cursory look at the DWARF spec makes me believe that libdwarf does, in fact, hide some of the complexity of the DWARF format, I've found that its API carries its own set of problems, particularly on the memory management side of things. The main complaint I have is that every time I make a call to libdwarf, it allocates space for the response under an opaque pointer and asks <em>me</em> to remember the type of the request, so that later I can ask the library to free that memory. The natural consequence of such a design is that two thirds of the code that I write to interface with libdwarf end up being spent managing memory outside of my control.</p>
<p>This example, extracted from the official documentation, illustrates my point (I've added the last <tt class="docutils literal">else if</tt> branch to avoid leaking memory through the error descriptor):</p>
<div class="highlight"><pre><span></span><span class="kt">void</span> <span class="nf">example1</span><span class="p">(</span><span class="n">Dwarf_Die</span> <span class="n">somedie</span><span class="p">)</span> <span class="p">{</span>
<span class="n">Dwarf_Debug</span> <span class="n">dbg</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="n">Dwarf_Signed</span> <span class="n">atcount</span><span class="p">;</span>
<span class="n">Dwarf_Attribute</span> <span class="o">*</span><span class="n">atlist</span><span class="p">;</span>
<span class="n">Dwarf_Error</span> <span class="n">error</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="n">Dwarf_Signed</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">errv</span><span class="p">;</span>
<span class="n">errv</span> <span class="o">=</span> <span class="n">dwarf_attrlist</span><span class="p">(</span><span class="n">somedie</span><span class="p">,</span> <span class="o">&</span><span class="n">atlist</span><span class="p">,</span> <span class="o">&</span><span class="n">atcount</span><span class="p">,</span> <span class="o">&</span><span class="n">error</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">errv</span> <span class="o">==</span> <span class="n">DW_DLV_OK</span><span class="p">)</span> <span class="p">{</span>
<span class="k">for</span> <span class="p">(</span><span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">atcount</span><span class="p">;</span> <span class="o">++</span><span class="n">i</span><span class="p">)</span> <span class="p">{</span>
<span class="cm">/* use atlist[i] */</span>
<span class="n">dwarf_dealloc</span><span class="p">(</span><span class="n">dbg</span><span class="p">,</span> <span class="n">atlist</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">DW_DLA_ATTR</span><span class="p">);</span>
<span class="p">}</span>
<span class="n">dwarf_dealloc</span><span class="p">(</span><span class="n">dbg</span><span class="p">,</span> <span class="n">atlist</span><span class="p">,</span> <span class="n">DW_DLA_LIST</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">else</span> <span class="k">if</span><span class="p">(</span><span class="n">errv</span> <span class="o">==</span> <span class="n">DW_DLV_ERROR</span><span class="p">){</span>
<span class="cm">/* use error */</span>
<span class="n">dwarf_dealloc</span><span class="p">(</span><span class="n">dbg</span><span class="p">,</span> <span class="n">error</span><span class="p">,</span> <span class="n">DW_DLA_ERROR</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
<!-- ** -->
<p>This piece of code takes a DWARF Debug Information Entry and asks about its attributes (key-value pairs describing it, such as name, size, ...). If the number of attributes returned by the function is <em>n</em>, then the number of <tt class="docutils literal">dwarf_dealloc</tt> calls needed will be <em>n+1</em>, one for each of the attributes and another one for the list itself.</p>
<p>Also, notice how libdwarf fails to keep track of the types of pointers it allocates, transferring that burden onto the caller. This means that every call to <tt class="docutils literal">dwarf_dealloc</tt> needs to specify one among these identifiers: <tt class="docutils literal">DW_DLA_STRING</tt>, <tt class="docutils literal">DW_DLA_LOC</tt>, <tt class="docutils literal">DW_DLA_LOCDESC</tt>, <tt class="docutils literal">DW_DLA_ELLIST</tt>, <tt class="docutils literal">DW_DLA_BOUNDS</tt>, <tt class="docutils literal">DW_DLA_BLOCK</tt>, <tt class="docutils literal">DW_DLA_DEBUG</tt>, <tt class="docutils literal">DW_DLA_DIE</tt>, <tt class="docutils literal">DW_DLA_LINE</tt>, <tt class="docutils literal">DW_DLA_ATTR</tt>, <tt class="docutils literal">DW_DLA_TYPE</tt>, <tt class="docutils literal">DW_DLA_SUBSCR</tt>, <tt class="docutils literal">DW_DLA_GLOBAL_CONTEXT</tt>, <tt class="docutils literal">DW_DLA_ERROR</tt>, <tt class="docutils literal">DW_DLA_LIST</tt>, <tt class="docutils literal">DW_DLA_LINEBUF</tt>, <tt class="docutils literal">DW_DLA_ARANGE</tt>, <tt class="docutils literal">DW_DLA_ABBREV</tt>, <tt class="docutils literal">DW_DLA_FRAME_OP</tt>, <tt class="docutils literal">DW_DLA_CIE</tt>, <tt class="docutils literal">DW_DLA_FDE</tt>, <tt class="docutils literal">DW_DLA_LOC_BLOCK</tt>, <tt class="docutils literal">DW_DLA_FRAME_BLOCK</tt>, <tt class="docutils literal">DW_DLA_FUNC_CONTEXT</tt>, <tt class="docutils literal">DW_DLA_TYPENAME_CONTEXT</tt>, <tt class="docutils literal">DW_DLA_VAR_CONTEXT</tt>, <tt class="docutils literal">DW_DLA_WEAK_CONTEXT</tt> and <tt class="docutils literal">DW_DLA_PUBTYPES_CONTEXT</tt>.</p>
<p>That's a clear violation of the eighth item of Casey Muratori's 2004 <a class="reference external" href="http://mollyrocket.com/873">Designing and Evaluating Reusable Components</a> API design checklist (which, by the way, deserves a wider audience):</p>
<blockquote>
Use of the component's resource management (memory, file, string, etc.) is completely optional.</blockquote>
<p>In order to sidestep this issue, I've decided to modify libdwarf and make it use my own custom allocator functions, effectively taking control of memory back from the library. The last revision of libdwarf (January, 2016) makes 34 calls to <tt class="docutils literal">malloc</tt>, 10 to <tt class="docutils literal">calloc</tt>, 1 call to <tt class="docutils literal">realloc</tt> and under 100 calls to <tt class="docutils literal">free</tt>. I've gotten rid of the <tt class="docutils literal">realloc</tt> call, which was only there because of some sloppy coding, and I've replaced the rest with calls to functions specified by the user through three extra parameters I've added to <tt class="docutils literal">dwarf_init</tt>.</p>
<p>My use case for libdwarf consists on parsing the whole binary in one go, building my own intermediate representation of the debugging information and, once I'm done, discarding all memory allocated through that library. That allows me to make my <tt class="docutils literal">malloc</tt> and <tt class="docutils literal">calloc</tt> functions use a simple stack allocator and to replace <tt class="docutils literal">free</tt> with an empty function. Here's a barebones example of the way I use the tweaked libdwarf:</p>
<div class="highlight"><pre><span></span><span class="cm">/* Compile with: clang -std=c++11 -Wl,-export-dynamic,-rpath=. libdwarf.so \</span>
<span class="cm"> example.cc -o example -lelf */</span>
<span class="cp">#include</span> <span class="cpf"><stdint.h></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><stdlib.h></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><string.h></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><fcntl.h></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf"><assert.h></span><span class="cp"></span>
<span class="cp">#include</span> <span class="cpf">"libdwarf.h"</span><span class="cp"></span>
<span class="k">struct</span> <span class="n">MemoryArena</span><span class="p">{</span>
<span class="kt">uint8_t</span> <span class="o">*</span><span class="n">base</span><span class="p">;</span>
<span class="kt">size_t</span> <span class="n">size</span><span class="p">;</span>
<span class="kt">size_t</span> <span class="n">used</span><span class="p">;</span>
<span class="p">};</span>
<span class="kt">void</span> <span class="o">*</span> <span class="nf">push_size</span><span class="p">(</span><span class="n">MemoryArena</span> <span class="o">*</span><span class="n">arena</span><span class="p">,</span> <span class="kt">size_t</span> <span class="n">size</span><span class="p">,</span> <span class="kt">bool</span> <span class="n">clear</span><span class="o">=</span><span class="nb">false</span><span class="p">){</span>
<span class="n">assert</span><span class="p">((</span><span class="n">arena</span><span class="o">-></span><span class="n">used</span> <span class="o">+</span> <span class="n">size</span><span class="p">)</span> <span class="o"><=</span> <span class="n">arena</span><span class="o">-></span><span class="n">size</span><span class="p">);</span>
<span class="kt">void</span> <span class="o">*</span><span class="n">result</span> <span class="o">=</span> <span class="n">arena</span><span class="o">-></span><span class="n">base</span> <span class="o">+</span> <span class="n">arena</span><span class="o">-></span><span class="n">used</span><span class="p">;</span>
<span class="n">arena</span><span class="o">-></span><span class="n">used</span> <span class="o">+=</span> <span class="n">size</span><span class="p">;</span>
<span class="k">if</span><span class="p">(</span><span class="n">clear</span><span class="p">)</span> <span class="n">memset</span><span class="p">(</span><span class="n">result</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="n">size</span><span class="p">);</span>
<span class="k">return</span> <span class="n">result</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">static</span> <span class="n">MemoryArena</span> <span class="n">dwarf_arena</span><span class="p">;</span>
<span class="kt">void</span> <span class="o">*</span> <span class="nf">stack_malloc</span><span class="p">(</span><span class="kt">size_t</span> <span class="n">size</span><span class="p">){</span> <span class="k">return</span> <span class="n">push_size</span><span class="p">(</span><span class="o">&</span><span class="n">dwarf_arena</span><span class="p">,</span> <span class="n">size</span><span class="p">);</span> <span class="p">}</span>
<span class="kt">void</span> <span class="o">*</span> <span class="nf">stack_calloc</span><span class="p">(</span><span class="kt">size_t</span> <span class="n">nmemb</span><span class="p">,</span> <span class="kt">size_t</span> <span class="n">size</span><span class="p">){</span> <span class="k">return</span> <span class="n">push_size</span><span class="p">(</span><span class="o">&</span><span class="n">dwarf_arena</span><span class="p">,</span> <span class="n">nmemb</span><span class="o">*</span><span class="n">size</span><span class="p">,</span> <span class="nb">true</span><span class="p">);</span> <span class="p">}</span>
<span class="kt">void</span> <span class="nf">stack_free</span><span class="p">(</span><span class="kt">void</span> <span class="o">*</span><span class="p">){}</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="kt">void</span><span class="p">){</span>
<span class="c1">// Memory arena initialization</span>
<span class="kt">int</span> <span class="n">bytes_to_allocate</span> <span class="o">=</span> <span class="mi">1024</span><span class="o">*</span><span class="mi">1024</span><span class="p">;</span>
<span class="n">dwarf_arena</span><span class="p">.</span><span class="n">base</span> <span class="o">=</span> <span class="p">(</span><span class="kt">uint8_t</span> <span class="o">*</span><span class="p">)</span> <span class="n">malloc</span><span class="p">(</span><span class="n">bytes_to_allocate</span><span class="p">);</span>
<span class="n">dwarf_arena</span><span class="p">.</span><span class="n">size</span> <span class="o">=</span> <span class="n">bytes_to_allocate</span><span class="p">;</span>
<span class="n">dwarf_arena</span><span class="p">.</span><span class="n">used</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">fd</span> <span class="o">=</span> <span class="n">open</span><span class="p">(</span><span class="s">"inferior"</span><span class="p">,</span> <span class="n">O_RDONLY</span><span class="p">);</span> <span class="c1">// ELF target</span>
<span class="n">Dwarf_Debug</span> <span class="n">dbg</span><span class="p">;</span>
<span class="n">Dwarf_Error</span> <span class="n">err</span><span class="p">;</span>
<span class="n">dwarf_init</span><span class="p">(</span><span class="n">fd</span><span class="p">,</span> <span class="n">DW_DLC_READ</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="o">&</span><span class="n">dbg</span><span class="p">,</span> <span class="o">&</span><span class="n">err</span><span class="p">,</span> <span class="n">stack_malloc</span><span class="p">,</span> <span class="n">stack_calloc</span><span class="p">,</span> <span class="n">stack_free</span><span class="p">);</span>
<span class="c1">// Call as many dwarf_* functions as necessary to build</span>
<span class="c1">// my own representation of debug information without</span>
<span class="c1">// worrying about dwarf_dealloc or dwarf_finish</span>
<span class="c1">// Recycling of dwarf memory arena for other purposes</span>
<span class="n">dwarf_arena</span><span class="p">.</span><span class="n">used</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="n">MemoryArena</span> <span class="n">some_other_arena</span> <span class="o">=</span> <span class="n">dwarf_arena</span><span class="p">;</span>
<span class="n">dwarf_arena</span> <span class="o">=</span> <span class="p">{};</span>
<span class="c1">// Code that does something useful with the debug symbols</span>
<span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
<!-- ** -->
<p>If you ever find yourself in a situation similar to mine, you can get the source of the modified libdwarf from <a class="reference external" href="https://github.com/debiatan/libdwarf_custom_allocator">this github repository</a>. You will also need to install the <tt class="docutils literal"><span class="pre">libelf-dev</span></tt> package that comes with you Linux distribution. <strong>However, I urge you to reconsider using libdwarf</strong>. The obtuseness of it's memory allocation scheme is just my main complaint about it, but my dislike for its API does not stop there.</p>
<p>Also, remember those two <em>terribly complicated</em> "specialized virtual machines" Eli Bendersky mentioned? Turns out that libdwarf only implements one of them. Here's what libdwarf's documentation has to say about the other one:</p>
<blockquote>
<p>6.22 Location Expression Evaluation</p>
<p>An "interpreter" which evaluates a location expression is required in any debugger. There is no interface
defined here at this time.</p>
<p>One problem with defining an interface is that operations are machine dependent: they depend on the
interpretation of register numbers and the methods of getting values from the environment the expression is
applied to.</p>
<p>It would be desirable to specify an interface.</p>
</blockquote>
A limitation of GDB frontends2016-02-17T19:00:00+01:002016-02-17T19:00:00+01:00Miguel Lechóntag:blog.debiatan.net,2016-02-17:/a_limitation_of_gdb_frontends.html<!-- :subtitle: (well, one of them) -->
<p>GDB is an OK command-line debugger, but its not a solid foundation for graphical debuggers. According to the GDB documentation, the recommended way of interacting with the debugger is through <a class="reference external" href="https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI.html">the GDB/MI interface</a>, which offers almost the same functionality as the regular GDB command-line through an alternative text protocol …</p><!-- :subtitle: (well, one of them) -->
<p>GDB is an OK command-line debugger, but its not a solid foundation for graphical debuggers. According to the GDB documentation, the recommended way of interacting with the debugger is through <a class="reference external" href="https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI.html">the GDB/MI interface</a>, which offers almost the same functionality as the regular GDB command-line through an alternative text protocol with a well-defined but somewhat obtuse grammar. If all we need is the ability to automate a GDB session, then GDB/MI is probably enough, but if we want our frontend to provide functionality not present in GDB, we may be out of luck.</p>
<p>As an example, let's imagine we want to ask GDB about the value associated to a variable. In a regular command-line debugging session, we would issue a print command specifying the name of the variable and the debugger would look for it inside the blocks encompassing our current line to provide an answer.</p>
<p>What happens, however, when we try to implement mouseover variable inspection? Suddenly, the current line can't act as an implicit location for our queries and the variable we want to inspect may not be uniquely identified by its name. Take for instance this run-of-the-mill piece of C++ code, representative of my coding style on Friday afternoons:</p>
<table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20</pre></div></td><td class="code"><div class="highlight"><pre><span></span> <span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="kt">void</span><span class="p">){</span>
<span class="kt">float</span> <span class="n">thing</span> <span class="o">=</span> <span class="mf">2.f</span><span class="p">;</span>
<span class="p">{</span>
<span class="kt">int</span> <span class="n">thing</span> <span class="o">=</span> <span class="mi">3</span><span class="p">;</span>
<span class="kt">int</span> <span class="n">lets_break_here</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">{</span>
<span class="kt">char</span> <span class="n">thing</span> <span class="o">=</span> <span class="sc">'a'</span><span class="p">;</span>
<span class="c1">// thing in a comment</span>
<span class="cp">#if 0</span><span class="c"></span>
<span class="c"> thing</span>
<span class="cp"> #endif</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span><span class="o"><</span><span class="mi">10</span><span class="p">;</span> <span class="o">++</span><span class="n">i</span><span class="p">){</span>
<span class="n">thing</span> <span class="o">=</span> <span class="n">thing</span> <span class="o">*</span> <span class="n">thing</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
</td></tr></table><p>Let's say we decide to stop on line five and inspect the values associated to all available <tt class="docutils literal">thing</tt>s using Emacs' gud-tooltip-mode.</p>
<div class="figure align-center">
<img alt="Testing gdb-emacs" src="images/gdb_emacs.png" style="width: 599.0px; height: 484.0px;" />
</div>
<p>The result are wrong most of the time. Maybe KDevelop has other thoughts on the matter...</p>
<div class="figure align-center">
<img alt="Testing gdb-kdevelop" src="images/gdb_kdevelop.png" style="width: 603.54px; height: 491.82px;" />
</div>
<p>Not really. Qt-creator?</p>
<div class="figure align-center">
<img alt="Testing gdb-qtcreator" src="images/gdb_qtcreator.png" style="width: 612.72px; height: 513.36px;" />
</div>
<p>This one at least does not fall for the commented <tt class="docutils literal">thing</tt> and even acknowledges the existence of other types of <tt class="docutils literal">thing</tt>s somewhere else in the program. But still, I'm beginning to suspect that GDB frontend programmers don't usually debug the sort of code I tend to work on. The results are the same for Eclipse's CDT standalone debugger and for DDD. I can't say what CLion and Nemiver do, because they refuse to work on my machine.</p>
<p>Figuring out the lexical scope of variables in C programs is not <em>that</em> hard. All these frontends fail to implement it properly because GDB makes it hard for them. Sure, they could issue an <tt class="docutils literal">info scope 7</tt> command and find out the addresses and sizes of all variables relative to the stack base pointer for all scopes encompassing line seven:</p>
<pre class="literal-block">
Scope for 7:
Symbol thing is a variable at frame base reg $rbp offset 0+-17, length 1.
Symbol thing is a variable at frame base reg $rbp offset 0+-12, length 4.
Symbol lets_break_here is a variable at frame base reg $rbp offset 0+-16, length 4.
Symbol thing is a variable at frame base reg $rbp offset 0+-8, length 4.
</pre>
<p>Then, they would only need to infer somehow which <tt class="docutils literal">thing</tt>'s which, find its type and format its value accordingly. At this point it may be easier to read the DWARF debug information directly and avoid GDB altogether.</p>
<p>Or we could just purchase a Windows license, download Visual Studio Community Whatever, register it and...</p>
<div class="figure align-center">
<img alt="Testing Visual Studio's debugger" src="images/gdb_visual_studio.png" style="width: 613.8px; height: 644.4px;" />
<p class="caption">Oops!</p>
</div>
<p>I suppose the problem here is that the Visual Studio team cares mostly about C#, and since C# does not allow variables with the same name in nested scopes...</p>
inDRM: Copy control with a personal touch2015-04-01T10:00:00+02:002015-04-01T10:00:00+02:00Miguel Lechóntag:blog.debiatan.net,2015-04-01:/inDRM.html<p>I was born in 1981, but I still got late to the PC party. My first computer was a Pentium 100, 8 MB RAM, 850 MB HDD, 14-inch CRT beast. My parents bought it for my brother and me on December 1995.</p>
<p>Some weeks before lugging it home, I was …</p><p>I was born in 1981, but I still got late to the PC party. My first computer was a Pentium 100, 8 MB RAM, 850 MB HDD, 14-inch CRT beast. My parents bought it for my brother and me on December 1995.</p>
<p>Some weeks before lugging it home, I was already making preparations. I bought a case of ten 3.5-inch floppies and headed to the local chess club. They had a 486 that people were supposed to use to study and play chess, but it also moonlighted as a software stash for club members under 25. That afternoon I spent twenty minutes swapping floppies in and out of the computer, labeling them as <tt class="docutils literal">Mortal Kombat II RAR Volume 1/8</tt>, etc. That same procedure would have to be reenacted backwards at home, once a beige box graced my desk with its presence.</p>
<p><em>Was it worth it?</em> Yes. <em>Did I play the game much?</em> I played the hell out of it that Xmas. <em>Why are you telling me this stuff?</em> Because I want to talk about...</p>
<div class="section" id="compromise">
<h2>Compromise</h2>
<p>Look. In order to play that thing, I had to:</p>
<ul class="simple">
<li>Walk twenty minutes to the chess club</li>
<li>Pester someone to get my turn at the computer</li>
<li>Wait twenty more minutes in front of it</li>
<li>Walk back home</li>
<li>Spend twenty extra minutes unRARing the stuff</li>
</ul>
<p>After all that, I would have played the game even if it sucked! Maybe it sucked and I didn't notice!</p>
<p>I invested some of my time in getting it running so, had it failed to meet my expectations, I would have been left with these two options:</p>
<ul class="simple">
<li>Deciding that I was stupid for wasting a perfectly good afternoon</li>
<li>Tricking myself into thinking that the game was good</li>
</ul>
<p>Back then, I didn't think of myself as a stupid person, so I would have gone for the second one. Of course, the game was <em>actually</em> good, so it's a moot point. But anyways, what's the moral of the story?</p>
</div>
<div class="section" id="wasting-the-time-of-players-increases-the-subjective-appeal-of-games">
<h2>Wasting the <strong>time</strong> of players increases the subjective appeal of games</h2>
<p>Which is kind of tragic, given today's sorry state of software distribution. People don't go to game stores anymore. They don't ask their friends for games, either. They just wait for the <em>Steam</em> sale, whip out a credit card and pay a visit to the fridge while their game downloads. Where's the compromise in that?</p>
<p>We can do better. Up for some unsolicited advice? Thought so.</p>
<p>If you're a game developer and want your stuff to be noticed, forget <em>greenlight</em>. What you need is <strong>20th century game distribution</strong> and you know it. The days of tapes and floppies may be long gone, but when everything seems lost, you can always count on good old...</p>
</div>
<div class="section" id="d-r-m">
<h2>D.R.M.</h2>
<p>Here's my proposal:</p>
<ul class="simple">
<li>Upload your game somewhere and force your first users to contact you to get an activation key. (<tt class="docutils literal">wasted time + social interaction = ugly experience</tt>).</li>
<li>Since you don't want to spend all day handing keys over, unload that burden on your players. Make it so that <strong>every unlocked copy of your game is capable of generating valid activation keys</strong> (<tt class="docutils literal">oh this game has to be reaally good if im answering peoples emails just because i wrote in a forum that i was playing it</tt>).</li>
<li>...</li>
<li>Profit!</li>
</ul>
<p>That's the gist of the <tt class="docutils literal">inDRM</tt> copy control scheme, my contribution to <a class="reference external" href="http://www.sigbovik.org/2015/">this year's SIGBOVIK conference</a>.</p>
<p>If you want to know more, check <a class="reference external" href="https://github.com/debiatan/inDRM/blob/master/doc/article.pdf">the article</a> and browse the <a class="reference external" href="https://github.com/debiatan/inDRM">associated GitHub repository</a>.</p>
</div>
<div class="section" id="list-of-games-using-indrm">
<h2>List of games using inDRM</h2>
<p>None so far, but they're coming.</p>
</div>
Five weeks of Handmade Hero2015-03-13T13:00:00+01:002015-03-13T13:00:00+01:00Miguel Lechóntag:blog.debiatan.net,2015-03-13:/five-weeks-of-handmade-hero.html<p>You may be aware that Barcelona holds a couple dozen Mobile World Congresses a year. However, you probably ignore that there are other technological meetings in Barcelona that far surpass the MWC by any metric you can think of. The <a class="reference external" href="http://www.qidv.org/">QIDV</a> (<em>Quedadas informales de desarrollo de videojuegos</em>, Meetups about Game …</p><p>You may be aware that Barcelona holds a couple dozen Mobile World Congresses a year. However, you probably ignore that there are other technological meetings in Barcelona that far surpass the MWC by any metric you can think of. The <a class="reference external" href="http://www.qidv.org/">QIDV</a> (<em>Quedadas informales de desarrollo de videojuegos</em>, Meetups about Game Development) are the most prominent example.</p>
<p>For the tenth edition of these gatherings, I decided to give a talk to raise awareness of Casey Muratori's <a class="reference external" href="https://handmadehero.org/">Handmade Hero</a> project. I tried to summarize the highlights of the first five weeks in just fifteen minutes, explaining mainly the architectural decisions that support the live code editing capabilities of his platform layer.</p>
<p>If you couldn't come but understand Spanish, I invite you to take a look at the recording of the session:</p>
<p>Here's just the fun part (the demo) at the end:</p>
<div class="flex-video widescreen" style="margin: 0 auto;text-align:center;">
<iframe src="https://www.youtube.com/embed/0vHt8qBOumc?start=758&feature=oembed" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>
<p>And here's the whole thing:</p>
<div class="flex-video widescreen" style="margin: 0 auto;text-align:center;">
<iframe src="https://www.youtube.com/embed/0vHt8qBOumc?start=228&feature=oembed" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>
<p>If you don't feel satisfied by my bare bones explanation of the technical details, please resort to this <a class="reference external" href="http://nullprogram.com/blog/2014/12/23/">nicely written post about the subject</a>. If that still doesn't do the trick, you can always watch <a class="reference external" href="https://youtu.be/oijEnriqqcs">the original Handmade Hero video</a>.</p>
<p>Enjoy!</p>
Acampa como puedas2015-01-27T07:00:00+01:002015-01-27T07:00:00+01:00Miguel Lechóntag:blog.debiatan.net,2015-01-27:/acampa-como-puedas.html<p>This weekend I took part in the seventh edition of the <a class="reference external" href="http://globalgamejam.org/">Global Game Jam</a>. The goal of the event is to create a game related in some way to a given theme in 48 hours. This year's theme was <em>What do we do now?</em>.</p>
<p>Instead of flying solo, as I …</p><p>This weekend I took part in the seventh edition of the <a class="reference external" href="http://globalgamejam.org/">Global Game Jam</a>. The goal of the event is to create a game related in some way to a given theme in 48 hours. This year's theme was <em>What do we do now?</em>.</p>
<p>Instead of flying solo, as I did last year, I joined two friends: <a class="reference external" href="http://tamats.com/blog/">Javi</a> and Esteban. It payed off big time, not only because <em>jamming alone</em> doesn't make any sense (I can code by myself any weekend I so desire -- which is most of them), but also because I jammed in good company and <a class="reference external" href="https://gamejambcn.com/en/resultats-danys-anteriors/resultats-2015/">we fucking won the local Barcelona section of the jam</a>.</p>
<div class="figure align-center">
<img alt="Gorilla-bears chasing dinner." src="images/acampa_attack.jpg" style="width: 372.0px; height: 232.0px;" />
<p class="caption">Gorilla-bears chasing dinner.</p>
</div>
<p>If you want a detailed account on how it went or some technical tips to play the game, I refer you to <a class="reference external" href="http://www.tamats.com/blog/?p=665">Javi's write-up</a>. Other than that, here's the <a class="reference external" href="http://globalgamejam.org/2015/games/acampa-como-puedas">profile of the game at the GGJ web page</a> and <a class="reference external" href="http://tamats.com/games/ggj15">a link to the game</a>. Enjoy!</p>
<p>(UPDATE) Here's the presentation of the game Javi gave at the end of the jam:</p>
<div class="flex-video widescreen" style="margin: 0 auto;text-align:center;">
<iframe src="https://www.youtube.com/embed/ZnF-WKUu5Tw?start=412&feature=oembed" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>
Oscilloscope2014-11-25T18:00:00+01:002014-11-25T18:00:00+01:00Miguel Lechóntag:blog.debiatan.net,2014-11-25:/oscilloscope.html<p>It seems that building an oscilloscope on top of an Arduino board is some sort of rite of passage. Do a <a class="reference external" href="https://duckduckgo.com/?q=arduino+oscilloscope">duckduckgo</a> search and you will see that all the good names for this kind of project are already taken, so I have decided to name mine just <em>oscilloscope</em>. In …</p><p>It seems that building an oscilloscope on top of an Arduino board is some sort of rite of passage. Do a <a class="reference external" href="https://duckduckgo.com/?q=arduino+oscilloscope">duckduckgo</a> search and you will see that all the good names for this kind of project are already taken, so I have decided to name mine just <em>oscilloscope</em>. In my defense, I'll say that I have written mine because I needed the most basic of oscilloscopes and it's just easier to build one from scratch than to adapt some already existing code.</p>
<p>I've written my oscilloscope code it in order to inspect a single-channel periodic signal with a frequency close to 250 Hz. My first trivial attempt at the task already recorded the signal at 5 kHz and transmitted it over USB without the need to buffer data in the RAM of the microcontroller, so I haven't bothered speeding it up.</p>
<p>The main bottleneck in my case is the serial transmission. Trying to collect more than 5800 samples per second will likely result in the loss of some of them. I've written some extra code that checks for missing samples, but it never detects problems at 5 kHz and, frankly, I'm OK with twenty samples per cycle.</p>
<p>All in all, putting together the <tt class="docutils literal">C</tt> firmware and the <tt class="docutils literal">Python</tt> client, the project takes less than 130 lines. The program that checks for periodicity, finds the frequency and aligns the cycles to display them adds 90 extra lines. The output of this program looks like this:</p>
<div class="figure align-center">
<img alt="Jiggling periodic wave" src="images/oscilloscope.gif" style="width: 406.0px; height: 306.0px;" />
<p class="caption">Visualization of several cycles of a PWM signal at 250 Hz after going through a simple RC circuit.</p>
</div>
<p>Here's the <a class="reference external" href="https://github.com/debiatan/oscilloscope">link to the github repository</a>.</p>
Distortion Vector Field2014-08-25T08:00:00+02:002014-08-25T08:00:00+02:00Miguel Lechóntag:blog.debiatan.net,2014-08-25:/guybrush.html<p>Just to keep the blog rolling, I'm posting this ten-second clip of Guybrush Threepwood coming into existence from some sort of primordial soup of pixels.</p>
<div class="flex-video widescreen" style="margin: 0 auto;text-align:center;">
<iframe src="https://player.vimeo.com/video/104247325?app_id=122963" frameborder="0" title="Guybrush materializes" allow="autoplay; fullscreen" allowfullscreen></iframe>
</div>
<p>On the left, you can see the distortion field applied to the original image to deform it. It consists of a field of vectors, each …</p><p>Just to keep the blog rolling, I'm posting this ten-second clip of Guybrush Threepwood coming into existence from some sort of primordial soup of pixels.</p>
<div class="flex-video widescreen" style="margin: 0 auto;text-align:center;">
<iframe src="https://player.vimeo.com/video/104247325?app_id=122963" frameborder="0" title="Guybrush materializes" allow="autoplay; fullscreen" allowfullscreen></iframe>
</div>
<p>On the left, you can see the distortion field applied to the original image to deform it. It consists of a field of vectors, each of them centered on a pixel of the image, following a 2D <a class="reference external" href="https://en.wikipedia.org/wiki/Ornstein–Uhlenbeck_process">Ornstein-Uhlenbeck process</a>. They are convolved with a Gaussian kernel that grows as the video steps forward.</p>
Unit-Test-Based Programming2014-04-01T10:00:00+02:002014-04-01T10:00:00+02:00Miguel Lechóntag:blog.debiatan.net,2014-04-01:/utbp.html<p>UTBP is a new subparadigm of Declarative Programming, in which code is generated from a description of its intended behavior, specified through unit tests. This work was presented at <a class="reference external" href="http://www.sigbovik.org/2014/">SIGBOVIK 2014</a> and received the <em>Most Frighteningly Like Real Research</em> Award.</p>
<p>For an in-depth tour of UTBP, you can check <a class="reference external" href="https://github.com/debiatan/utbp/blob/master/doc/article.pdf">the …</a></p><p>UTBP is a new subparadigm of Declarative Programming, in which code is generated from a description of its intended behavior, specified through unit tests. This work was presented at <a class="reference external" href="http://www.sigbovik.org/2014/">SIGBOVIK 2014</a> and received the <em>Most Frighteningly Like Real Research</em> Award.</p>
<p>For an in-depth tour of UTBP, you can check <a class="reference external" href="https://github.com/debiatan/utbp/blob/master/doc/article.pdf">the article</a> and browse the <a class="reference external" href="https://github.com/debiatan/utbp">associated GitHub repository</a>. If you prefer a light introduction, you can choose between two versions of the same video. This one is dubbed:</p>
<div class="flex-video widescreen" style="margin: 0 auto;text-align:center;">
<iframe src="https://player.vimeo.com/video/89179353?app_id=122963" frameborder="0" title="Unit-Test-Based Programming -- English dubbing" allow="autoplay; fullscreen" allowfullscreen></iframe>
</div>
<p>This one is subbed:</p>
<div class="flex-video widescreen" style="margin: 0 auto;text-align:center;">
<iframe src="https://player.vimeo.com/video/89182858?app_id=122963" frameborder="0" title="Unit-Test-Based Programming -- English subtitles" allow="autoplay; fullscreen" allowfullscreen></iframe>
</div>
Gradius meets Twister2014-01-27T20:00:00+01:002014-01-27T20:00:00+01:00Miguel Lechóntag:blog.debiatan.net,2014-01-27:/gradius-meets-twister.html<p>This weekend I participated in the sixth edition of the <a class="reference external" href="http://globalgamejam.org/">Global Game Jam</a> . The goal of the event was to create a game related in some way to a given theme in 48 hours. This year's theme was <em>We don't see things as they are, we see them as we …</em></p><p>This weekend I participated in the sixth edition of the <a class="reference external" href="http://globalgamejam.org/">Global Game Jam</a> . The goal of the event was to create a game related in some way to a given theme in 48 hours. This year's theme was <em>We don't see things as they are, we see them as we are</em>.</p>
<p>I decided to go for a multiplayer Gradius clone in which collisions with enemies remap your controls. I developed it using <a class="reference external" href="http://www.codeskulptor.org/">CodeSkulptor</a>, which is basically a Python interpreter running on top of Javascript. Needless to say, it's no speed demon. See for yourself:</p>
<div class="flex-video widescreen" style="margin: 0 auto;text-align:center;">
<iframe src="https://player.vimeo.com/video/85182232?app_id=122963" frameborder="0" title="Gradius meets Twister" allow="autoplay; fullscreen" allowfullscreen></iframe>
</div>
<p>If you want to give it a try, I have two suggestions to make:</p>
<ul class="simple">
<li>Find someone to play against (otherwise is going to look as sad as my single player game above).</li>
<li>Run it on Chrome.</li>
</ul>
<p>Here's the <a class="reference external" href="http://www.globalgamejam.org/2014/games/gradius-meets-twister">profile of the game at the GGJ webpage</a> and <a class="reference external" href="http://www.codeskulptor.org/#user36_Z3QPgd5jeF_98.py">a link to the game</a>. Enjoy!</p>
<p><strong>Update</strong>: Looks like <a class="reference external" href="https://gamejambcn.com/gjb-2014/resultats/">my game placed third</a> on the local vote we held after the Jam! Hooray!</p>
xrobot2014-01-02T10:00:00+01:002014-01-02T10:00:00+01:00Miguel Lechóntag:blog.debiatan.net,2014-01-02:/xrobot.html<p>Every once in a while, I need to step outside the command line. Sometimes I'm even forced to interact with <em>deaf</em> graphical programs, those that do not listen to standard input or a meager HTTP port. In those desperate times, were it not for tools such as <a class="reference external" href="http://xautomation.sf.net/">XAUT</a> or <a class="reference external" href="http://www.semicomplete.com/blog/projects/xdotool/">xdotool …</a></p><p>Every once in a while, I need to step outside the command line. Sometimes I'm even forced to interact with <em>deaf</em> graphical programs, those that do not listen to standard input or a meager HTTP port. In those desperate times, were it not for tools such as <a class="reference external" href="http://xautomation.sf.net/">XAUT</a> or <a class="reference external" href="http://www.semicomplete.com/blog/projects/xdotool/">xdotool</a>, I would have to type and click outside of <tt class="docutils literal">VIM</tt>, like cavemen probably did.</p>
<p>Those two little programs are enough to make me happy when confronted with an X11 server. However, my computer, of a more whimsical nature, is reluctant to execute binaries other than a Python interpreter (like any other well-meaning general-purpose device assembled during the 21st century, really). That is why I have decided to write the simplest Python library I could think of that is able to:</p>
<ul class="simple">
<li>Find out the position of the mouse pointer</li>
<li>Move the mouse pointer around the screen</li>
<li>Press and release mouse buttons</li>
<li>Press and release keys in the keyboard</li>
<li>Capture the screen</li>
</ul>
<p>The <tt class="docutils literal">xrobot</tt> library is lean, simple and Python[23]-compliant. It is just a wrapper around functions defined inside <tt class="docutils literal"><span class="pre">python-xlib</span></tt>. Since Xlib screen capture is painfully slow, the <tt class="docutils literal"><span class="pre">python-gtk</span></tt> bindings are used instead, if present. I have decided to return images as <tt class="docutils literal">numpy</tt> arrays for my convenience; if you find that dependency unbearable, you can root it out easily from the code.</p>
<div class="flex-video widescreen" style="margin: 0 auto;text-align:center;">
<iframe src="https://player.vimeo.com/video/83220240?app_id=122963" frameborder="0" title="5000 clicks" allow="autoplay; fullscreen" allowfullscreen></iframe>
</div>
<p class="caption-text">Here's me, at five hundred clicks/second.</p>
<p>I leave you with a <a class="reference external" href="https://github.com/debiatan/xrobot">link to the xrobot github repository</a> and some sample code:</p>
<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">xrobot</span>
<span class="n">xr</span> <span class="o">=</span> <span class="n">xrobot</span><span class="o">.</span><span class="n">XRobot</span><span class="p">()</span>
<span class="n">xr</span><span class="o">.</span><span class="n">move</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
<span class="n">robot</span> <span class="o">=</span> <span class="n">XRobot</span><span class="p">()</span>
<span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">robot</span><span class="o">.</span><span class="n">mouse_pos</span><span class="p">()</span>
<span class="k">print</span><span class="p">(</span><span class="s1">'Current mouse position: x ='</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="s1">'y ='</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="n">robot</span><span class="o">.</span><span class="n">move</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
<span class="n">robot</span><span class="o">.</span><span class="n">click</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="n">robot</span><span class="o">.</span><span class="n">key</span><span class="p">(</span><span class="s1">'a'</span><span class="p">)</span> <span class="c1"># Press and release 'a'</span>
<span class="n">robot</span><span class="o">.</span><span class="n">key_down</span><span class="p">(</span><span class="s1">'comma'</span><span class="p">)</span> <span class="c1"># Press ','</span>
<span class="n">robot</span><span class="o">.</span><span class="n">key_up</span><span class="p">(</span><span class="s1">'comma'</span><span class="p">)</span> <span class="c1"># Release ','</span>
<span class="n">width</span><span class="p">,</span> <span class="n">height</span> <span class="o">=</span> <span class="n">robot</span><span class="o">.</span><span class="n">screen_resolution</span><span class="p">()</span>
<span class="k">print</span><span class="p">(</span><span class="s1">'Screen width:'</span><span class="p">,</span> <span class="n">width</span><span class="p">,</span> <span class="s1">'Screen height:'</span><span class="p">,</span> <span class="n">height</span><span class="p">)</span>
<span class="n">img</span> <span class="o">=</span> <span class="n">robot</span><span class="o">.</span><span class="n">capture_screen</span><span class="p">()</span>
<span class="kn">import</span> <span class="nn">pylab</span> <span class="kn">as</span> <span class="nn">pl</span>
<span class="n">pl</span><span class="o">.</span><span class="n">imshow</span><span class="p">(</span><span class="n">img</span><span class="p">)</span>
<span class="n">pl</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>