Web Development on Windows Does Not Need To Suck
Posted byon under
Microsoft Windows has long been considered a subpar platform for web development not based on a Microsoft technology, or in more general terms, for any kind of open source work. If you are looking for tutorials online on any open source related topic and happen to be on a Windows PC, you are going to have a lot of trouble in finding material that does not assume you are on a Linux or Mac system with access to Unix-based tools.
I mostly work on a Mac computer these days, but I own and regularly use a Windows PC as well. In this article I'm going to introduce you to the open source project that I use to bridge the gap between Windows and the Unix world. This project allows me to work on my open source projects on the PC, using the same tools I use on the Mac, or on a Linux PC. The image above is a screenshot of my terminal, running on Windows 10 (click on it to see it in its actual dimensions). There you can see I have three independent very Unixy-looking console sessions. Not the type of terminal window you associate with Windows, right?
Unix on Windows: What Are Your Options
Fortunately, as a developer on a Windows machine, you have a few options to create a Unix friendly environment appropriate for open source work.
The most obvious is to create a virtual machine using VirtualBox and install your favorite Linux distribution in it. This is a great option if your have a PC with lots of RAM, because you can set up a decent VM for Linux and leave enough RAM for the Windows side to remain usable. Sadly, for many PCs this option may be too resource intensive and result in a poor user experience.
Another good option is to rent a cloud server and provision it with the Linux distribution of your choice. You can then use putty, an awesome remote shell client for Windows, to open a remote console and work on your cloud server from the comfort of your own PC. This is a very good alternative, but it costs money.
The goal of this article is to introduce you to a third option that does not have the disadvantages of the first two: the open source project Cygwin.
What is Cygwin?
Cygwin is an open source project that has been around for a very long time. It was already a well established project when I was introduced to it in 2005. According to their own description, Cygwin "is a large collection of GNU and Open Source tools which provide functionality similar to a Linux distribution on Windows", and "a DLL (cygwin1.dll) which provides substantial POSIX API functionality".
What does this mean? It basically means that you can install and use hundreds of Unix utilities on your Windows PC. The list of ported applications includes the most common Unix shells, bash and zsh, so you can replace your Windows prompt with a Unix-like console window. The gcc suite of compilers is also ported, so if you come across a Unix based application in source code form, you have a good chance of being able to compile it and run it on your PC, exactly like you would on Linux or OS X. If you want to see what packages are available, here is the complete list of ported applications.
Several scripting language interpreters have been ported and have a Cygwin version. Python 2.7 and 3.4 are available, and so are Ruby and Perl. These are different versions than the official Windows versions of these languages, as they have been compiled as Unix tools. For example, the Cygwin versions of these languages have access to POSIX subsystem calls not available on regular Windows, such as
fork(), which Cygwin implements. They are also free of the problems caused by the Windows backward slash separator for pathnames. Any tools compiled under Cygwin use the standard Unix forward slash as path separator.
It is important to note that Cygwin does not use virtualization. The utilities that come with the project are 100% native Windows executables, so there are no VMs to set up, and more importantly, no outrageous RAM requirements. This is made possible by the cygwin1.dll library, which translates POSIX calls into Windows native API calls on the fly.
Cygwin comes with a Windows interactive installer, which you must run when you do your first install, and also when you want to add or remove packages from your installation. Having to use an interactive installer to manage the installed packages is not ideal, but luckily there is a third party command-line package manager called apt-cyg that is inspired by the package managers of Linux distributions such as Debian and Ubuntu.
To make things more consistent with Unix, the Cygwin installer creates a proper Unix-like directory structure that includes the standard /bin, /dev, /etc, /usr and /home directories. And not only that, you also get a home directory under /home/username.
Is Cygwin perfect? No, it isn't. If you are used to working on Unix based systems, you'll find that Cygwin has some quirks. Most of the problems are minor though, and usually occur when you combine native Windows tools with Cygwin tools. A very common issue occurs when sharing text files, because native Windows applications use the CRLF sequence to end a line, while Cygwin uses the Unix standard of just a LF character. Another inconvenience I experience is that the Windows version of PyCharm is incompatible with the Cygwin version of Python, but in any case, using vim to edit Python code, and pudb to do interactive debugging work really well anyway.
I hope this gives you a good overview of Cygwin, which in my opinion provides a fairly decent environment for doing software development, or also to become familiar with the Unix way of doing things, something that I think is very useful in itself. If you are learning from online tutorials, using Cygwin you will be able to follow instructions for Linux or OS X with little or no modifications.
If you are comfortable working with an interactive installer, and don't mind having to select what packages to install from an overwhelmingly long list, then you are pretty much ready to go. There are two official installers available: setup-x86.exe for a 32-bit install, and setup-x86_64.exe for 64-bit. I strongly recommend that you use the 64-bit installer if you have the 64-bit version of Windows. Only use the 32-bit version if you can't run 64-bit code on your system.
I don't find the Cygwin installer very friendly myself, so I have created a little batch file that invokes the installer and automatically selects a small set of basic packages appropriate for an initial install. My batch script also installs the apt-cyg command line package manager, which you can then use to customize your packages.
The script is hosted on github. You can open the script in your browser by clicking the following link:
Click to open: install-cygwin.bat
Once you have it in a browser window, you can use your browser's Save As function to write it, for example, to your desktop.
To use this installer batch file, simply double click its icon, or open a command prompt and run it. The script does not take any options, it will install the 64-bit version of Cygwin in a directory called cygwin, inside your Windows profile directory (normally C:/Users/username). The first few lines in the script can be edited to change the installation directory, or to switch to the 32-bit version.
The batch file will ask for confirmation before starting the install. If you give it the okay, it will download and launch the official Cygwin installer, which will run for a few minutes without requiring any actions from you. Once the installation is complete the batch file will create a desktop icon with the label "Cygwin" that you can use to start a bash console. This installer does not require administrator rights, and does not touch anything in your C:\Program Files area. Everything is installed inside the root directory selected at the top of the file.
If everything goes well, when you double click the Cygwin icon you should see a bash console window like this one:
If for any reason you want to uninstall Cygwin, simply delete the root folder, which is C:\Users\your-username\cygwin, and then also remove the desktop icon. Cygwin does not come with an interactive uninstaller.
My batch file performs a very basic installation of Cygwin. You can now start a bash console, and that will give you access to a variety of unix tools, which include vim, Python 2.7 and the gcc compiler, among many others. Now comes the fun part of adding more packages. What packages to install and how to configure them is something that is key to making you productive in this environment. We are all different, so we like things configured in different ways. On Windows this level of openness is not common, you'll have to get used to it.
To add packages you can use the
apt-cyg installer. For example, if you wanted to install Python 3.4, you would run this command at the bash prompt:
$ apt-cyg install python3
The awesome thing is that now you can go out and research how people set up their development environments on OS X or Linux and do the same on your Windows PC!
I have decided not to include detailed instructions on how I set up my personal development environment as part of this article. There are literally hundreds of people that share their setups in blogs and YouTube videos, so you have plenty of material to research. If you need something to start with, I recommend that you look into tmux, the nifty utility that I show in the screenshot at the top of this article that allows you to split your terminal window in panels, among other things. You may also want to switch your shell from bash to zsh, which has extremely useful extensions such as oh-my-zsh, with support for command prompt themes and plugins. For code editors, vim is installed as part of the first install, but it is a plain install without any plugins. You can opt to learn vim and add plugins that make it more useful, or you can install emacs, nano or any other editor that you like. As far as languages, Perl and Python 2.7 are in the initial install, but you can add Python 3.4, Ruby and others.
If there is enough interest, I can write a follow up article in which I describe the setup pictured in the screenshot, which is based on zsh, oh-my-zsh, tmux, vim, and a number of vim plugins.
I hope you find Cygwin useful. Do you have questions? Write them below in the comments!
Become a Patron!
Hello, and thank you for visiting my blog! If you enjoyed this article, please consider supporting my work on this blog on Patreon!
#1 arthurz said 2015-08-11T01:33:25Z
I bought your Flask book and enjoyed it so much!
Please do write a follow up article.
From my experience, the # of packages is overwhelming https://cygwin.com/packages/package_list.html one can easily get lost trying to figure what to install.
Also Cygwin takes a lot of hard drive space and requires patience while being installed.
Thanks for the article!
#2 Miguel Grinberg said 2015-08-11T05:35:23Z
@arthurz: My cygwin install is about 850MB, I would not say it is huge, considering all the stuff that you get.
#3 Francisco Valladolid said 2015-08-11T05:42:44Z
Why use windows if you can use Linux or BSD with all goodies.
#4 Miguel Grinberg said 2015-08-13T05:17:05Z
@Francisco: For those that use a Windows computer (for whatever reason), this makes perfect sense in my opinion. If you never find yourself using a Windows PC, then this article is not for you.
#5 Jeremy Epstein said 2015-08-14T01:23:50Z
For a long time (about 5 years), I did most of my open-source coding (Python and PHP) on Windows with heavy use of Cygwin. It did the job, but Cygwin really is a poor man's *nix environment. It has so many quirks and limitations: near-impossible to work with symlinks; unix-style vs windows-style file permissions clash badly; very hard to launch any windowed app from the command-line (gksudo style).
In the end, I fully switched my dev work to Linux (I had already been dual-booting Win and Ubuntu for years), and I haven't looked back. Unfortunately I still have to boot in Win for various things (e.g. Photoshop, connecting to TV / projector (without losing my hair), 3D games), but coding isn't one of them!
#6 Miguel Grinberg said 2015-08-14T05:07:06Z
@Jeremy: yes, as I mention in the article, Cygwin isn't perfect.
#7 Newton Parrett said 2015-09-02T19:06:53Z
Why not just buy a computer that does all of these things already without having to hack a PC. Apple does most of these things right out of the box and have been doing it the right way for years.
#8 Miguel Grinberg said 2015-09-04T06:45:12Z
@Newton: I'm not trying to sell you Windows. If you like Mac computers and can get stuff done with them, then this article is not for you. I actually agree that Macs are better for development, I work mostly on a Mac, I openly said so at the beginning of the article. But I also own a PC, and when I'm using it I want to have the same familiar environment I have on the Mac or on Linux. What's wrong with that?
#9 Sergey said 2015-09-04T17:24:06Z
Hi. You can see your configuration file for vim?
#10 Miguel Grinberg said 2015-09-04T21:34:16Z
@Sergey: here is my current .vimrc file: http://pastebin.com/jQXCuK7s
#11 Kyle Lawlor said 2015-09-16T16:55:42Z
I am running into some strange problems uninstalling Cygwin. To make a long story short I was running into "command not recognized" issues for even simple commands like "ls" within the Cygwin terminal, which were working fine when I first installed with the batch file.
I was messing around with the .bashrc to add an Anaconda version of Python, and I was also adding some aliases. I think that the trouble started when I opened the .bashrc with windows notepad. Oddly it seemed that when I sourced the bash file with source .bashrc, the commented lines were no longer being recognized as commented lines (with the # prefix). I am currently blaming windows notepad for this behavior.
At this point I was hoping to get a fresh start, especially because I couldn't even use 'vi' to edit the bash file (vi was unrecognized as well). So I am trying to delete cygwin which is installed under C:\Users\user\cygwin. I am able to delete everything but a file called "id.exe". When I try to right click delete from Windows, I am getting "File Access Denied, You need permission to perform this action". I also tried running a command prompt as admin and then deleting the file using "rm id.exe" but I get "rm: cannot remove 'id.exe': Permission denied".
I am puzzled. Please help. Thanks!
I also just tried the following:
command: takeown /r /f cygwin
result: cygwin and cygwin/bin now owned by "computername\user", but again "Access is denied" for id.exe.
command: cacls cygwin /c /G "user":F /T
result: Fine for the two folders as above, but Access denied for id.exe
command: rmdir /s cygwin
result: Access denied id.exe
Very puzzled now. I often feel like a professional at breaking things :-P
Thanks ahead of time.
Hoping to have a nice development environment on my windows machine soon :-)
#12 Miguel Grinberg said 2015-09-16T23:10:30Z
@Kyle: so one thing you need to keep in mind when you work with Cygwin is that things don't work too well when you mix native Windows apps with Cygwin apps. I assume the Anaconda version you were trying to install is the Windows version, correct? That's not a good idea, Cygwin has its own builds of Python, which are compiled with Unix settings. Also, to edit Cygwin files it's best to use an editor from Cygwin, such as vim, nano, etc.
I suspect this mixing of Windows and Cygwin got you in trouble. My recommendation is that you start over, just delete the cygwin folder and then run the batch file again to get a fresh install.
#13 Karl said 2015-09-29T08:12:25Z
a while back you recommended babun in some of your blog posts. Why have you now written your own installation script? Thanks, Karl
#14 Miguel Grinberg said 2015-09-30T00:33:13Z
@Karl: babun only runs the 32-bit version of Cygwin, which has some problems when used on 64-bit systems. The babun maintainers are not interested in the burden of supporting 64-bit, unfortunately.
#15 Aris Green said 2015-10-07T17:29:05Z
Cygwin always an option on Windows, but I do a fair amount of Python on Windows and I install using the installers from Python.org and develop using Notepad++ and running in a Windows CMD (without using Cygwin). One point of frustration is that you only get 1 version of Python 2 and 1 version of Python 3 ever to be able to coexist. With pyenv and pyenv-virtualenv you get python versions and virtual environments in rainbow colors on OS X and Linux. I'd like to at least have multiple Python 3's (3.4, 3.5 ) on Windows. I've head you can take the MSI installers from Python.org and to an admin install into a directory. You then manually setup the environment in a batch file.
By extracting Python 3.4.x and 3.5.x in sibling directories, you can then work by stitching together your environment using batch files, Powershell, etc to switch between versions I am hoping. Have you heard of this or tried this? Anaconda or Miniconda seems a little massive and a lot of people are using the Python.org msi installers on Windows but seem to get stuck on one version for the server and dev environments.
I guess another route is to build Python from sources on Windows (which I guess pyenv essentially does your OS X or Linux). Building packages like lxml seems to be too painful though. I'd love to hear your thoughts on this. A lot of people might like Cygwin in the dev environments, but maybe not on their corporate servers. Maybe a title for a follow up post would be "Does Development for Python Application On Windows Servers Without Cygwin Have To Suck?"
Basically the goal is multiple version of Python with isolated virtualenv's on Windows, like nvm for Node, chruby + ruby-install + gem_home for Ruby, pyenv + pyenv-virtualenv for Python.
#16 Miguel Grinberg said 2015-10-08T17:08:17Z
@Aris: I have the official Python 2.7, 3.3 and 3.4 installed on my Windows machine. I have 3.4 in my PATH variable, but whenever I want to use one of the others I just use the full path. And once you create a virtualenv the PATH is configured for you. So I guess I'm not sure I understand what is the problem you are having.
In any case, the point of this article was not to propose Cygwin's Python as a better Python, there is much more to development than the Python interpreter. The Windows cmd line, even powershell are not as powerful as bash, zsh on Unix, and even if you manage to be productive with those, very few people use them for open source development, so you won't find tutorials or information about them.
#17 Aris said 2015-10-08T17:21:30Z
@Miguel: Did you install Python 3.3 and 3.4 in Cygwin, or did you use the Python MSI installers. The MSI installers seem to prevent you from being able to install 2 flavors of Python 3 side by side. That is what I am after. Did you use the MSI with the admin install (msiexec -a <pythondownload.msi>)? But now, as of 3.5, it seems that there is not .MSI, there is a web install, exe installer, and embedabble zip. I'll take a look.
#18 Miguel Grinberg said 2015-10-08T18:43:11Z
@Aris: sorry, I wasn't very clear. I have the official Windows Python 2.7, 3.3 and 3.4 all installed. They are on folders C:\Python27, C:\Python33 and C:\Python34. I do not remember having any issues installing these. I have not installed 3.5 yet, so I don't know if anything changed on that release.
#19 Andreia Melo said 2016-01-10T12:15:27Z
Thanks! Great article! I'm going to give it a try, as you said my pc doesn't handle the virtual machine option very well :(
#20 Badri said 2016-01-10T13:25:11Z
Coming from Linux, one thing that I really miss in Windows is the middle-click-to-paste functionality! I found that the Windows version of Geany supports this: I was wondering if it works for Cygwin-installed programs as well?
#21 Miguel Grinberg said 2016-01-14T15:30:17Z
@Badri: Actually yes. Cygwin uses mintty for the console window, which supports copy by dragging the mouse over the text, and paste with middle or right click button.
#22 elmer cacaldo said 2016-01-14T17:27:48Z
NOw .... how to login in heroku??? still cannot login there using this Cygwin...give me some tutorials
#23 Miguel Grinberg said 2016-01-16T22:58:01Z
@elmer: you need to install the heroku client. See https://toolbelt.heroku.com/standalone.
#24 Purusharth Saxena said 2016-03-07T20:24:09Z
PS: From the list of mirrors use http://mirrors.kernel.org for faster download.
Also, when I had a basic machine with low ram, and a need of a pure Linux environment, I used to install VM and give it just enough ram to boot up and load all the important drivers and stuff.(~128MB).
And then I used Cygwin to ssh into the VM. I used a barebone Linux distro like puppy linux or lubuntu, but now ubuntu snappy core is out it should be even less resource intensive. In case, anyone is goin' to do the same.
#25 Raj Mitra said 2016-06-19T14:16:08Z
I have a couple of comments to make:
1. First of all, your book on "Flask web Development" is just great, especially because the git code is updated reularly. But I noticed a "deprecation" warning in many of the import lines, eg: the "flask.ext.wtf" should be replaced with "flask_wtf".
2. Regarding cygwin, I am a frequent user, because I find that some utilities of windows dont exist in cygwin or linux. eg: linux still lacks a decent pdf viewer+editor, which I use heavily in windows to read books/reports and make notes/comments on them. Due to this limitation of linux, I have not yet been able to do a full migration to linux.