Category Archives: Programming

Python on Windows 8.1

Recently I acquired a Microsoft Surface Pro 3 and of course I installed Python.
It turned out that installing Python on Windows has its own caveats – similar to the particularities of Python on OS X.

Some sites (like SciPy.org or iPython.org) recommend going with a specialised Distribution like anaconda on Windows Systems.

I wanted to have a little more control and decided to install the packages myself, starting with Python from http://python.org/.

There are two important questions:

  1. Python 2 or Python 3?
  2. 32bit or 64bit?

You will have to answer these for youself, based on your particular requirements.
Help for you decision can be found on the python.org wiki site or
on this DataNitro blog entry.

I decided to install the 32bit version of Python 2.7 for compatibility reasons.
You can install Python 3.4 in parallel. It will install into its own directory and it will have its own uninstall entry in the system panel.

Like on Mac OS X I installed virtualenv to manage multiple Environments with different sets of installed packages.
A good Windows specific how-to can be found on Tyler Butler’s page.
He also explains how to integrate with PowerShell via the virtualenvwrapper-powershell package.
If you are using PowerShell for the first time he also Points out how to enable the execution of scripts (see this TechNet article in general and this help page on Microsoft TechNet in particular). You have to explicitly Permit the execution of unsigned scripts. You can list your machine’s policies with this command in PowerShell:

Get-ExecutionPolicy -List

My policies look like this:

Scope            ExecutionPolicy
-----            ---------------
MachinePolicy    Undefined
UserPolicy       Undefined
Process          Undefined
CurrentUser      RemoteSigned
LocalMachine     Undefined

The RemoteSigned policy for the CurrentUser scope means that in my user’s context the execution of local unsigned scripts is allowed (while remote scripts have to be signed).
You can switch to this policy by issuing this command in PowerShell:

Set-ExecutionPolicy RemoteSigned -Scope CurrentUser

If you follow Tyler’s instructions you should be able to use virtualenv. You can switch to an environment with the workon command and you can exit the current environment with the command deactivate.
For more information about working with virtualenv have a look at this virtualenvwrapper command reference.

After Setting up virtualenv I wanted to install some packages that I usually Need, in particular numpy, scipy, matplotlib, pandas and iPython.
Usually a pip install numpy should work. But here it did not but pip stopped with an error message.
The SciPy.org page again recommends a pre-built distribution.
Alternatively it points to the maintainers of the single packages.
At this point I used the binary installers (Python 2.7 32bit) for numpy, scipy and matplotlib. The main reason were the recommendations to use the binary installer for matplotlib, also since the build instructions for Windows require Visual Studio compilers to be installed.
All three packages can be uninstalled from the Windows Control Panel/Programs and Features.
They install into the site-packages Directory so that virtualenv is not used. Since I usually always Need one of these packages this seemed acceptable for the time being.

But note that with this set-up there were still some errors when I tried to use matplotlib functions.
The six and pyparsing packages were missing.
Six is also needed for pandas and installed automatically as a dependency with it so I tried pip install pandas. The Installation failed again due to the missing c Compiler.
Since I had Microsoft Visual Studio 2013 installed I investigated why the compiler is not being found by pip.
I turns out that the different Python versions are built with specific versions of Visual Studio and the according packages need the very same compiler version to work (Visual Studio 2008 for Python 2.7 and Visual Studio 2010 for Python 3.3). This information can be found in this StackExchange question. In the answers some people had success by linked the location of newer Visual Studio versions to the expected location of the required version.
For me SET VS90COMNTOOLS=%VS120COMNTOOLS% did not work. pip found the compiler but the Installation aborted with a compile error.

Reading the error message carefully solves the problem: It gives a link to an official Microsoft download for Visual C++. Microsoft kindly provides a Compiler specifically for Python 2.7 packages.
With this compiler the install via pip worked for pandas and also for iPython:
pip install pandas
pip install ipython[notebook]

Installing iPython package under Windows 8.1
Installing iPython package under Windows 8.1

In the screenshot you can see iPython successfully installing and starting.

Python on OS X 10.9 Mavericks

I upgraded my older MacBook Pro to a new retina MacBook Pro.
For a couple of generations I used to migrate everything over to a new machine with the migration assistant or in target mode.

This time I used the chance to start over new since the system was clogged with a number of environments that became problematic (Ruby and Python installs, which weren’t always easy under rechten Mac OS versions).

Also wanted to replace legacy Macports installs with homebrew.

Pawel Lachowicz shares his how-to for installing Python on Mavericks:
www.quantatrisk.com
In this how-to I will generally follow Pawel’s list of packages to install. But doing so I will note errors, mitigations and changes I encountered during install.

I also installed R which for some functions relies on TCL/TK. Which is outdated on Mavericks, do R brings it’s own TCL/TK install package.
Since is installs to /usr/local it can potentially interfere with homebrew (according to brew doctor).

So I would rather recommend to remove the TCL/TK install that comes with package R and use the homebrew TCL/TK libs.
So far I have not tried homebrew R.

To be able to uninstall and to install different versions of Python I followed this recommendation on hackercodex.com.

First, install homebrew:
ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

brew install python --with-brewed-openssl
brew install python3 --with-brewed-openssl

virtualenv allows to install Python packages into different virtual environment. In turn this allows i.e. testing different versions of packages or different package sets.
Python packages can be installed with the older setuptools or the newer pip.
See here at hackercodex.com for advantages/disadvantages.
pip allows uninstall, so I chose pip.

pip install virtualenv

To make proper use of virtualenv one needs to create and activate an environmen, i.e.:
mkdir ~/source/virtualenvs
cd ~/source/virtualenvs
virtualenv quant
cd quant
. bin/activate

A virtualenv can be deactivated like this:
cd ~/source/virtualenvs/quant
. bin/deactivate

Then, go on installing the packages recommended by Pawel:

  • pip install mpmath
  • pip install numpy

The next package, scipy requires a Fortran compiler.
I did not have one installed. On my fresh OS X Mavericks install homebrew was the easiest solution. But is also possible to install Fortran from the Mac OS X 10.7 (Lion) installer, see Jonathan Goldsmith’s post at github.

brew install gfortran

Then continue:

  • pip install scipy
  • brew install pkg-config
  • brew install freetype
  • brew install libpng
  • brew install ffmpeg
  • pip install matplotlib

Trying to install matplotlib yields another error:

With the version change from matplotlib 2.5.0.1 to 2.5.1 the header directory structure was changed (see here). This is known, but seems to not have been fixed yet in the pip package.

This solution from David at stackoverflow solved the problem:

sudo ln -s /usr/X11/include/freetype2/freetype /usr/X11/include/

Again, continue installing packages:

  • brew install zeromq
  • pip install pyzmq
  • Already installed in site-packages with homebrew python 2.7.6: pip install Tornado
  • pip install readline
  • pip install azure
  • Skipped: pip install curses
  • pip install cython
  • pip install jinja2
  • pip install pexpect
  • pip install pygments
  • pip install pymongo
  • pip install sphinx
  • Skipped, since already installed by brew via dependency: pip install sqlite3
  • pip install wx
  • Skipped, since already installed by brew via dependency: pip install zmq
  • pip install sympy
  • pip install patsy
  • pip install scikit_learn
  • Note: statsmodels requires pandas, so install pandas first.
    pip install pandas
  • pip install statsmodels
  • pip install ipython