Monday, July 27, 2009

Prep the PEP

The CRLF Windows issues is the only thing holding me back from producing a patch and PEP and submitting it for inclusion into the Python Core. I posted on comp.lang.python and until I get a response, I will be gathering sources, references and planning out my PEP as well as looking over my code and checking to see where optimizations can be made.

Friday, July 24, 2009


While chatting with an Argentinean buddy, he used an amusing portmanteau. He referred to "Windows" as "Guindous," a combination of the word Windows and the Spanish verb "guindar" or "to freeze." I was rather amused and felt like sharing the term with people.


I discovered that Wing IDE has beta releases with Python 3.1 support and have been using them to debug my code. My unit tests for my subprocess.Popen changes now work flawlessly on both Linux and Windows. The only things holding me back now are issues with stdout being opened in text mode by libc. See the following message from Amaury on Python-Dev:

> Ah so any streams opened in text mode on Windows will read '\n' newlines as
> '\r\n'?

No, it is the libc stdout which is opened in text mode. This simple program:
int main() {
when run like this:
program > out
will create a file ending with \r\n.

ReadFile and WriteFile (and other functions from the win32 API) are
unaware of this, and faithfully transmit the bytes without
This is causing my unit test for my file wrapper to fail; all of my '\n' newlines are converted to '\r\n.' I would greatly appreciate suggestions on how to deal with it as well as using ctypes or adding to _subprocess.c. If you are not subscribed to Python-Dev, you can view the community's discussion on whether ctypes or C code would be a better solution at this link;

Thursday, July 16, 2009

C You Later

I have abandoned modifying _subprocess.c in favor of a path that will be much easier than attempting to pull out the parts of Mark Hammond's C++ code and converting it to C: Python ctypes. After importing ctypes, I can call "ctypes.windll.kernel32.PeekNamedPipe" to access the PeekNamedPipe function which is used in the modifications to In C/C++ multiple return values are generally handled by passing variable by reference. In ctypes, this functionality is duplicated so I must pass the variables that I will need to get data from using the ctypes.byref() function and then parse said data into a tuple.

Tuesday, July 14, 2009

MSVC++ Runtime Library

When examining the full build output from Microsoft Visual C++ Express, it was discovered there was an error that was along the lines of "MSC_DLL_LIB" and the program make_versioninfo.exe. Executed alone, make_versioninfo.exe complained of missing MSVC++ DLL runtime libraries and after re-install Microsoft Visual C++ Express, Python now builds successfully.

I adjusted Mark Hammond's code to be integrated into the _subprocess.c file and now that I can compile Python on Windows, I will be testing it to make sure it functions as expected and integrating the other two routines from PyWin32 that I need for my modification to


I have modified one of the three functions from Mark Hammond's PyWin32 library to be integrated into the _subprocess.c file of the Python 3.1 source code. When I first attempted to compile the Python source code in Windows yesterday, I encountered a linking error: "LINK : fatal error LNK1181: cannot open input file '.\python31_d.lib'". Unable to compile the code, I went to the Python-Dev list for assistance and a few suggestions were made but none seemed to work. I disabled Windows Data Execution Protection and restarted my machine and downloaded the Python 3.1 source code from the SVN library. That code compiled fine and I considered the issue resolved.

As the old saying goes "correlation does not imply causation." I have, again, encountered the same linking error and can no longer compile the Pytohn source. I downloaded both the Python 3.2 code from the Subversion repository again; did not compile. I downloaded the official Python 3.1 code Gzipped and Bzipped tarbills; neither compiled.

Luckily, I thought ahead when I woke up and prepared to compile the _subprocess.c file with Mark Hammond's code integrated into it. I downloaded the Python 3.1 tar ball and attempted to compile taking screenshots at each step.

I downloaded the Python 3.1 source tar ball from

Using WinRAR, I extracted the source code.

I tried double clicking the pcbuild.sln file and right clicking and selecting open but neither of those actions did anything. The cursor changed to an hour glass briefly but Visual C++ did not open.

I then opened Visual C++ and went to "File," then "Open."

I browsed for the pcbuild.sln file and selected it to open.
A modal error dialog about a limitation in my version of Visual C++ popped up.

The project opens after hitting "OK" so I right click on the "python" portion of the project and hit "Build."

It proceeds to build but eventually errors at the end of the process.
Build log output:

Friday, July 10, 2009

Mid Term Summary

Asynchronous I/O for the Python subprocess module has been successfully implemented in both the Python 2k and Python3k branches. Building off of that functionality, I also implemented a wrapper class for the asynchronous I/O so that child processes could act as stand-ins for file objects. The unit tests I produced for the asynchronous I/O and file-like object run flawlessly.

In the last half of Google Summer of Code, I will be be perfecting my file-like object for subprocess.Popen, porting Mark Hammond's win32 modules to C and integrating them with the _subprocess.c file as the asynchronous I/O depends on his code to run. When all is said and done, I will produce a PEP for my changes and attempt to get them integrating into future Python releases.

Thursday, July 2, 2009

Core Conversion Complete

My file wrapper and Josiah's code now work in Python 3.1 and the changes have been committed to the Google Code project. The next thing to tackle is converting Mark Hammond's code from C++ to C and integrating it into the _subprocess.c file.