LD_DEBUG: debugging ld.so on linux

David Baraff told me about this awesome way to see what ld.so is doing on linux.

    $ LD_DEBUG=help /bin/cat
    Valid options for the LD_DEBUG environment variable are:

    libs display library search paths
    reloc display relocation processing
    files display progress for input file
    symbols display symbol table processing
    bindings display information about symbol binding
    versions display version dependencies
    all all previous options combined
    statistics display relocation statistics
    unused determined unused DSOs
    help display this help message and exit

    To direct the debugging output into a file instead of standard output
    a filename can be specified using the LD_DEBUG_OUTPUT environment variable.

John Backus on debugging

John Backus describes debugging, circa 1957:

In order to produce a program with built-in debugging facilities, it is a

simple matter for the programmer to write various PRINT statements, which

cause "snapshots" of pertinent information to be take at appropriate points in

his procedure, and insert these in the deck of cards comprising his original

FORTRAN program.  After compiling this program, running the resulting machine

program, and comparing the resulting snapshots with hand-calculated or

known values, the programmer can localize the specific area in his FORTRAN

program which is causing the difficulty.  After making the appropriate corrections

in the FORTRAN program he may remove the snap-shot cards and recompile

the final program or leave them in and recompile if the program is

not yet fully checked.

The FORTRAN Automatic Coding System

Proceedings of the Western Joint Computer Conference

Los Angeles, California, February 1957

Stopping in a C++ function called via Python

How do i tell totalview to stop at a line number in a particular file? This is a C++ function in a SO which is being loaded as part of a python import.

Leaf Nunes has the most awesome answer:

I 'm not sure how you tell totalview via totalview, but you can tell the C++ to stop at a line by adding:

asm("int $3");

This is the only way i've seen to pre-set a breakpoint in C code referenced by a python binary running under totalview. Once it's stopped, you should be able to look around all the libraries the python binary has loaded and set breakpoints as you desire.

Arun Rao:

If you run the python program until it finishes, then you can set the breakpoint because the library will have been loaded, and restart the process.

Introspecting python variables (to print a debug message)

I asked about introspecting python variables. I was looking for something similar to Tcl's upvar command. I received a couple of good responses. I didn't ever end up using them for debugging, but I sure learned a lot more about Python introspection than I knew previously.

Alex Mohr:

We have to remember that an arbitrary number of variables, all with different names, can refer to the same object. For example, 

foo = 'something'

bar = foo

Now suppose I say, printvars('message', bar). The printvars function will get a reference to the 'something' string, but it doesn't know that the name of the reference I called you with was 'bar' rather than 'foo'. This function will take your argument, and look through globals() and find all the names that refer to that object, and then print those out, but this is almost certainly slow, fragile and useless:

def printvars(msg, *args):
    print msg, ':',
    for arg in args:
        matches = []
        for name, val in globals().items():
            if arg is val:
                matches.append(name)
        for match in matches:            print match+',',
        print '=', repr(arg)+';',
    print ''

If I use this as with my hypothetical example above, it produces this output:

>>> printvars('message', bar)
message : bar, foo, = 'something' 

Erick Tryzelaar:

This is doable with scary python black magic:

import sys
def foo(msg, *args):
   caller = sys._getframe(1)
   print msg, ' '.join(['%s=%s' % (x, caller.f_locals[x]) for x in args])
x = 5
y = 6
foo('hello', 'x', 'y')
hello x=5 y=6

Lars Damerow:

I agree with Erick and Alex here -- you can't get away from passing the variable names in as strings, since Python just considers variable names to be string keys for the local and global scope dictionaries. I also second Erick's opinion that using sys._getframe is not a good idea. It may look redundant, but the simplest and most bulletproof approach is probably something like this:

def printvars(msg, *args):
    print msg, ' '.join(['%s=%s' % (x, v) for (x, v) in args])
>>> x = 1
>>> y = 2
>>> printvars("hello", ("x", x), ("y", y))
hello x=1 y=2

We could make that a lot cleaner if Python had macros, but it doesn't. If we knew that all of the desired variables are in the same dictionary (which includes locals() and globals()), we could obviously clean up the calls like this:

def printvars(msg, scope, *args):
    print msg, ' '.join(['%s=%s' % (x, scope[x]) for x in args])
>>> x = 1
>>> y = 2
>>> printvars("hello", locals(), "x", "y")
hello x=1 y=2

Jim Atkinson:

Although you cannot avoid giving the name as well as the value for the variable. You can use keyword syntax to pass them and avoid having to type quotes around the name.

def printvars(msg, **kw):
    print msg,
    keys = kw.keys()
    keys.sort()
    for key in keys:
    print "%s=%r", (key, kw[key])
    print

x = 1 
y = 2 
printvars ("hello", x=x, y=y)
hello x=1 y=2 

The only downside is that you cannot control the order of the variables, it is always alphabetical.

Omnigraffle: Don't write bundles!

Sometimes omnigraffle will rewrite a file into a bundle. It can be configured not to do this:

defaults write com.omnigroup.OmniGraffle PrivateGraffleFlatFile 0
defaults write com.omnigroup.OmniGrafflePro PrivateGraffleFlatFile 0

Oracle: row-wise comparison

I had a query that needed to compare two columns in a row andwas making two queries to do so. All hail the row-wise comparison!

select id from p4_versions
where (versionof, version) = (select id, headver from p4_files
      where p4path like '%/date.txt');

How to Create a REST Protocol

"REST is an architectural style that can be used to guide the construction of web services. Recently, there have been attempts to create such services that have met with mixed success. This article outlines a series of steps you can follow in creating your protocol--guidance that will help you get all the benefits that REST has to offer, while avoiding common pitfalls."

How to Create a REST Protocol

Python: retrieving the oracle error code

Anthony Tuininga gave me this hint on retrieving the Oracle error code from a cx_python program:

try:
    cursor.execute("select 1 / 0 from dual")
except cx_Oracle.DatabaseError, e:
    print "Exception:", e    # this is what you are looking at right now
    errorObj, = e.args       # this is the actual "error" that cx_Oracle raises
    print "ErrorCode:", errorObj.code
    print "ErrorMessage:", errorObj.message

Causing Guido van Rossum to ask

This is overly subtle. Why not write

    errorObj = e.args[0]

and received the response

No particularly good reason -- it just happens to be a way of indicating that there should be one and only one entry in the list. An exception is raised if there is not whereas args[0] does not raise the exception. Its an open question which is the better code.