Work Blogging: compiler architecture

About a month ago, f2py quit working on my work laptop.

For those of you who do not know, f2py is a neat little compiler that allows you to compile Fortran code in such a way as to allow it to be called as a function by python scripts. I mostly program in python, because it's an incredibly intuitive language, and it doesn't have a million "gotcha" errors like C/fortran/perl/etc. However, Fortran is really great for numerical computation, because it's very low-level, and, as a result, is very very quick. So, in my current project, I'm using python as a wrapper around a wonderfully chocolaty Fortran center. Like a tootsie pop.

Now, when f2py quit working, all of a sudden I had to update all my compilers and programming tools, which I couldn't do until I updated my operating system (from OSX 10.4.x to 10.7.x; yes, I was that behind). Once the OS was upgraded, I went about re-installing everything: GCC, python and all its modules, etc. So far, so good, everything's installed, says it's working, time to try running f2py, which successfully compiles my fortran code. Yay! Now to run the compiled executable from my python code and get on with my modeling!

FAIL!

Terminal tells me that the two programs have mismatching architecture, and therefore cannot be run together. Google tells me this happens occasionally, and that it has to do with the fortran compiler called by f2py (in this case, gfortran, which is GCC's fortran compiler) compiling to i386 architecture, and python compiling to x86_64 archicture. This is important, since i386 is 32-bt, and x86_64 is 64-bit. Google also gives me a neat little command to verify this.

>>lipo -info /path/to/file


which gives:

>>lipo -info /usr/local/bin/gfortran
Non-fat file: /opt/local/bin/gfortran is architecture: i386
>>lipo -info /opt/local/bin/python
Non-fat file: /opt/local/bin/gfortran is architecture: x86_64


Neat, huh?

As an aside, the history here is actually pretty interesting. Intel's 32-bit chips ran on an i386 architecture (hence the i), whereas AMD's chips ran on a different architecture. When AMD created a 64-bit chip, they had to produce a new architecture, which they called x86_64. As Intel was developing their 64-bit chip, they came to an agreement with AMD to use their architecture as a universal standard. Now, Macbook Pros, like mine, have the new Intel chipset in them, which support both i386 and x86_64, but need to use x86_64 to function as 64-bit machines.

So, at this point, I'm left wondering, why is my gfortran running i386? I installed it through GCC, which should install a x86_64 version of the compiler. I didn't ask for "universal" (read, i386) variants on GCC or python. So, I call over my co-worker Jonathan to take a look at it.

He instantly sees the problem.

>>which gfortran
/usr/local/bin

>>which python
/opt/local/bin


This is a legacy problem from two years ago when I attempted to install python as a package with relative naivete about how linux worked, and ended up with a bunch of crap in my user account, stored now at /usr/local/bin. I had completely forgotten these things existed, and now, when I was calling gfortran, my computer was getting confused about which gfortran I actually wanted, and was sending me to the bad install I did years ago.

A couple simple commands later, I got everything routed to the right places, then spent some time deleting these useless files (general rule of thumb, as I understand it, is that everything in /usr/local/bin can be deleted; in my case, there are some NREL-specific files that I ought to leave, but that's another story). And now, everything works, and I know a great deal more about compilers. More than I ever wanted to, in fact.

Yay!