Development Environments

Development environments are something very personal. Each programmer has a unique recipe that works for them. This document covers some basics that can help get you started with the right structure and tools when working on python with different platforms. This document also has a focus on developing web applications, so the development environment presented also is focused on development of such platform with databases, web servers and more.

Apple Macintosh OSX

Where do I start with OSX? Great operating system that is modern, robust and used widely in the profesional computing scene. That said, somethings Apple seriously lacks at making great, especially things that really aren't Apple'ish. Python is one of those things.

For this reason, one of the first steps to get a proper Python environment in your Mac is going to be to install a separate package management system that installs latest Python and get's you on the right track. One package manager that is popular is called Homebrew.

Install Xcode

Install Mac OSX Xcode and it's command line tools. XCode is available in Apple's App Store and also you can download from:

Once you have installed Xcode you need to install the command line tools. The Xcode installation will ask for these. You can also utilize the command line:

user@learnpy.cisco.com:

xcode-select --install
xcodebuild -license

This will install the software and also do the Xcode license agreement that is required to be completed.

Install Home Brew

Every installation of OSX comes with a base set of tools including Ruby, Python, PHP and others. These are standard across the Unix world. Homebrew is installed in the system via ruby.

user@learnpy.cisco.com:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

BREW will change the environment variable PATH for you, but you need to insure that it stays set to this or OSX installed components will take precedence over BREW components. In OSX the profile file is located in ~/.profile and make sure that the PATH variable has the BREW path as the start of the string

user@learnpy.cisco.com:

export PATH=/usr/local/bin:/usr/local/sbin:$PATH

Install Python

Now you can install BREW's python that will supersede Apple's.

user@learnpy.cisco.com:

brew install python

The output of the installer should be similar to the following:

user@learnpy.cisco.com:

==> Installing python
==> Downloading https://homebrew.bintray.com/bottles/python-2.7.10_2.yosemite.bo
######################################################################## 100.0%
==> Pouring python-2.7.10_2.yosemite.bottle.tar.gz
==> /usr/local/Cellar/python/2.7.10_2/bin/python -s setup.py --no-user-cfg insta
==> /usr/local/Cellar/python/2.7.10_2/bin/python -s setup.py --no-user-cfg insta
==> /usr/local/Cellar/python/2.7.10_2/bin/python -s setup.py --no-user-cfg insta
==> Caveats
Pip and setuptools have been installed. To update them
  pip install --upgrade pip setuptools

You can install Python packages with
  pip install <package>

They will install into the site-package directory
  /usr/local/lib/python2.7/site-packages

See: https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/Homebrew-and-Python.md

.app bundles were installed.
Run `brew linkapps python` to symlink these to /Applications.
==> Summary
  /usr/local/Cellar/python/2.7.10_2: 4866 files, 76M

As you can see the installer also installed two important components of python, pip and also setuptools. PIP is a python tool to manage packaging from pypi ( you can check it out at https://pypi.python.org/pypi.

Setuptools is being depracted in favor of PIP, but stil has some uses as you will see soon.

At this point it would be a good idea to run the update for PIP.

user@learnpy.cisco.com:

pip install --upgrade pip setuptools

Stop!
You would think at this point you can install Python packages via PIP into the system. I highly suggest that you don't! The best plan of action is to utilize Python Virtual Environments that will localize all the packages to the projects that you are working avoiding scenarios that you have parent libraries in the system and virtual environment packages that can get confusing! Don't install any more Python packages until you get virtual environments working and your Mac will be better off.

Validate installed SSL packages

One of the reasons that we have installed a special Python is to get the proper SSL libraries working with OpenSSL that interact better with different Cisco components in a secure fashion. After the install you can issue the comand from the Python interpreter:

user@learnpy.cisco.com:
$ python
Python 2.7.10 (default, Jul 13 2015, 12:05:58)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> print ssl.OPENSSL_VERSION
OpenSSL 1.0.2d 9 Jul 2015

You can also see that with this version of Python we have all the different possible SSL protocol supported with this python build ( something that isn't the case with the Apple version ).

user@learnpy.cisco.com:
$ python
Python 2.7.10 (default, Jul 13 2015, 12:05:58)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> print ssl.PROTOCOL_SSLv23
2
>>> print ssl.PROTOCOL_SSLv3
1
>>> print ssl.PROTOCOL_TLSv1
3
>>> print ssl.PROTOCOL_TLSv1_1
4
>>> print ssl.PROTOCOL_TLSv1_2
5

Install/Validate GIT

If you installed XCODE git should be already installed in your system. Make sure that it is. GIT is a critical for revision control.

user@learnpy.cisco.com:

git --version

This should return back with the proper version of GIT if it is installed. If you don't have GIT installed and would like to install it, you could do so with HomeBrew on your Mac. I haven't had any problems with the version supplied by Apple XCODE.

If you want to use the HomeBrew version of GIT then do:

user@learnpy.cisco.com:

brew install -v git

MySQL

Installing MySQL on your Mac OSX computer is very easy with BREW. For Python development, installing MySQL isn't really a requirement, but could turn useful if you plan to do any DBA type application. Databases like SQLLite can be easier to implement or work with pragmatically from Python, but suffer from scalability concerns. Using tools like SQLAlchemy in Python make it possible to utilize different underlying DB engines by using it's abstraction layer.

To install MySQL with BREW you issue the command:

user@learnpy.cisco.com:

brew install -v mysql

Once the BREW has installed MySQL then you have to configure MySQL for operation. The first thing to do is to configure the MySQL CNF files. BREW provides a template mysql file that you then have to move to brew /etc.

user@learnpy.cisco.com:

cp -v $(brew --prefix mysql)/support-files/my-default.cnf $(brew --prefix)/etc/my.cnf

Once the file is copied you should make a couple of adjustements to the my.cnf configuration file to make it operational for your install. One of the important things to remember with MySQL is that if you don't specify an IP and listening port, it will only be active for localhost: traffic only ( meaning no external device would connect to it ). This is probably what you are really looking for in a development environment ( like a laptop computer ).

OptionDescription
innodb_buffer_pool_size = 128M Uncomment this line to increase/improve the performance of MySQL. If you are limited in memory constrains for any reason, you can leave this setting commented out.

To create the OSX Launch controls for MySQL, BREW has a toolset that makes this easier than it used to be when it had to be done by hand.

user@learnpy.cisco.com:

brew tap homebrew/services
brew services start mysql

Once mysql is operational you just should run the following command to secure the mysql installation.

user@learnpy.cisco.com:

$(brew --prefix mysql)/bin/mysql_secure_installation

This command will go through a series of questions. These include configuring the root password, confining root to localhost only, removing anonymous access and remove the test db.

Apache

There are various reasons to install Apache on your laptop, yet few have to do with Python. If you where to develop a web application in Python you could do much of the work directly using an IDE like PyCharm without the need of Apache running on your computer. That said there is much to learn when working on Python by having a Web Server running locally. Testing the configuration of a WSGI application running locally is possible. In fact you could have many applications developed installed running on your laptop for your use just like if they where installed applications.

The first step is to remove the launch files for the Apple apache that comes with OSX.

user@learnpy.cisco.com:

sudo launchctl unload /System/Library/LaunchDaemons/org.apache.httpd.plist 2>/dev/null

For Apache you need to install the homebrew/dupes. Since Apache is a duplicate of software that comes with OSX, the forumulae for this brew is placed in a separate repository. To make this accessible you have to tap that repository.

user@learnpy.cisco.com:

brew tap homebrew/dupes

Once that is completed, you can then install Apache.

user@learnpy.cisco.com:

brew install -v homebrew/apache/httpd24

You want to install the mod_wsgi module in conjunction with apache. WSGI stands for Web Server Gateway Interface and is a specification on web server communications with web applications. This will play a role when building WSGI Python apps and implementing them locally to see how they can be deployed in production servers.

Python WSGI

WSGI stands for Web Server Gateway Interace. It is described in detail on PEP-3333

user@learnpy.cisco.com:

brew install -v homebrew/apache/mod_wsgi

Once you have that installed you can fire up Apache and validate that it is working.

user@learnpy.cisco.com:

brew services start httpd24

If you point your browser to the address: http://localhost:8080 you will see that apache is now operational from the brew distribution. If you wish to make changes to how apache is configured, the files are located at:

  • /usr/local/etc/apache2/2.4
Apache as NON-root

You may have noticed that everything is running on port 8080 and 8443. The reason for this is because you are running apache in userland and not as root. This is something that goes with working with homebrew, as it does run outside of root. For this reason, you have the options of configuring a firewall rule to forward the traffic from port 80 to 8080. For more information on this you can reference the web as it goes beyond the scope of this document.

PHP

Although not related to Python, PHP is a very popular language to develop web based applications. Some of these applications can help you with Python development, like PHPMyAdmin that simplifies working with the DB in many ways.

user@learnpy.cisco.com:

brew tap homebrew/php
brew install -v php56 --homebrew-apxs --with-apache

Once this is completed you need to add the following to the HTTPD configuration file located in /usr/local/etc/apache2/2.4/httpd.conf

  • AddHandler php5-script .php
  • AddType text/html .php
  • DirectoryIndex index.php index.html

Then at this point you can tell brew to restart the service with the command:brew services restart httpd24

user@learnpy.cisco.com:

brew services restart httpd24

I would suggest that you also install a couple additional PHP modules.

user@learnpy.cisco.com:

brew install php56-mcrypt php56-imagick php56-tidy
brew services restart httpd24

You can now install phpmyadmin with brew.

user@learnpy.cisco.com:

brew install phpmyadmin

To complete the installation of phpmyadmin you have to create a virtual host definition in Apache. The best way to accomplish this is for each VHOST to have separate files. This is typically what happens in production environments. For a laptop development environment, putting the definition in the apache configuration file is also manageable. In a brew installation the location of the apache configuration is at /usr/local/etc/apache2/2.4/httpd.conf which we had modified previously to get PHP operational.

Simply add the following ( as long as the paths match what you have in your environment) to the apache configuration file.

user@learnpy.cisco.com:

  Alias /phpmyadmin /usr/local/share/phpmyadmin
  <Directory /usr/local/share/phpmyadmin/>
    Options Indexes FollowSymLinks MultiViews
    AllowOverride All
    <IfModule mod_authz_core.c>
      Require all granted
    </IfModule>
    <IfModule !mod_authz_core.c>
      Order allow,deny
      Allow from all
    </IfModule>
  </Directory>

Once you add that configuration you will have to restart apache.

user@learnpy.cisco.com:

brew services restart httpd24

At which point you should be able to reach phpmyadmin and log into it with the credentials that you had built for mysql root account ( or any other credentials you may have established ).

If you want to do development with PHP for any reason, you probably want to make available the USERDIR module for apache. This module will make it possible to run PHP modules from your home directory Sites folder.

In the apache configuration file in /usr/local/etc/apache2/2.4/httpd.conf locate a line that looks like:

#LoadModule userdir_module libexec/mod_userdir.so

And un-comment it to active the module. Then look for a line that is like:

# User home directories
#Include /usr/local/etc/apache2/2.4/extra/httpd-userdir.conf

and un-comment that include line and restart apache again.

user@learnpy.cisco.com:

brew services restart httpd24

Reference

URL Description
OS X 10.10 Yosemite Local Development Environment: Apache, PHP, and MySQL with Homebrew Really good document on homebrew that is focused on PHP development but provides exceptional details on homebrew.

Ubuntu

Under construction

Windows

Under Construction