Archive for the ‘Python’ Category

In this post, we will explore how to set different TAB settings while editing C/C++ and Python code. For C/C++, we want 2 spaces for indendation, while for Python, we will stick to PEP 8 recommendations (4 spaces for indendation, 8 spaces for TAB, 4 spaces for expandtab). Vim being such a powerful and versatile editor, there are many ways to achieve this. We will explore the most portable way – with changes just to the .vimrc.

" Python options
let python_highlight_all = 1
function! s:set_python_settings()
  set tabstop=8
  set softtabstop=4
  set shiftwidth=4
function! s:unset_python_settings()
  set tabstop=2
  set softtabstop=2
  set shiftwidth=2
autocmd BufNewFile,BufEnter *.{py} call set_python_settings()
autocmd BufLeave *.{py} call unset_python_settings()

What we are doing here is define two functions, set_python_settings() which will change tab settings as per PEP 8 recommendations and unset_python_settings() which will default to our C/C++ settings. Autocmd upon entering a new buffer (BufEnter) or a new file is opened (BufNewFile) of type *py, we call set_python_settings() to set tab settings for Python. When we leave a buffer (BufLeave) of type *.py, we get back to original settings by calling unset_python_settings().

Exuberant ctags is a pretty nifty utility for source code browsing. Especially since it integrates so well with vim. Also exuberant ctags can understand many languages (41 as per the official website). So this is relevant not only for C/C++ or Python but for all 41 languages supported (and future ones too). In this post, we explore yet another popular plugin using exuberant ctags – TagList.

TagList is capable of showing a list of functions/global variables/class/struct towards one side of the vim. This is similar to some IDEs or event editors like Notepad++. This makes browsing and navigation pretty easy. Our aim will be to be able to navigate current file using TagList window and to always display current function name in the status line.

Screenshot of TagList plugin showing current function name in status line

For this, we will need:

TagList will work only with exuberant ctags and not with any other ctags (specifically GNU ctags). Install exuberant ctags if required. Also ensure that ctags command should actually invoke exuberant ctags.

Install TagList plugin (see elaborate steps in the link). Typically vim plugins are installed by simply copying to $HOME/.vim/plugin directory. If any documentation is there, copy that to $HOME/.vim/doc and re-index vim help by giving “:helptags $HOME/.vim/doc” command in vim.

Now we need to customize the TagList plugin options to get what we want.

" TagList options
let Tlist_Close_On_Select = 1 "close taglist window once we selected something
let Tlist_Exit_OnlyWindow = 1 "if taglist window is the only window left, exit vim
let Tlist_Show_Menu = 1 "show Tags menu in gvim
let Tlist_Show_One_File = 1 "show tags of only one file
let Tlist_GainFocus_On_ToggleOpen = 1 "automatically switch to taglist window
let Tlist_Highlight_Tag_On_BufEnter = 1 "highlight current tag in taglist window
let Tlist_Process_File_Always = 1 "even without taglist window, create tags file, required for displaying tag in statusline
let Tlist_Use_Right_Window = 1 "display taglist window on the right
let Tlist_Display_Prototype = 1 "display full prototype instead of just function name
"let Tlist_Ctags_Cmd = /path/to/exuberant/ctags

nnoremap <F5> :TlistToggle
nnoremap <F6> :TlistShowPrototype

set statusline=[%n]\ %<%f\ %([%1*%M%*%R%Y]%)\ \ \ [%{Tlist_Get_Tagname_By_Line()}]\ %=%-19(\LINE\ [%l/%L]\ COL\ [%02c%03V]%)\ %P

Notice the call to Tlist_Get_Tagname_By_Line() within the statuline. This is what displays the current function name (actually tag which can be class name or struct name or any tag) in the status line. With the above settings, pressing F6 will show the prototype of current function at the bottom.

Pressing F5 will open up the TagList window which will show the list of functions/classes/structs/define etc. The focus is automatically switched to the list window and we can quickly jump to the function/class/struct definition we want to. Upon selecting an entry, the list window is automatically closed.shrink the window back to default size.

Screenshot of TagList window

Now with the full prototype display, it is not very easy to read the actual function name. TagList has a zoom feature to overcome this. Press “x” and the TagList window will enlarge, occupying almost the complete vim screen. Selection of an entry or pressing F5 again will close the window. Pressing x again will shrink the window back to default size.

Screenshot of TagList window zoomed using "x" key

TagList generates its own tags file and does not require the user to provide a tags file. This file is generated every time we switch to a buffer. The tags file is created somewhere in /tmp/ folder. TagList cannot work with a user generated tags file.

In my previous post, I came up with an enhanced pygenie, which could count total complexity of a Python project as a whole. As explained earlier, my motivation is purely for fun. I took a test run of the enhanced pygenie on some of the popular open-source Python projects. And the results are out ūüôā

Django 1.2.1

time ./pygenie/ Django-1.2.1/ -r
Total Cumulative Statistics
Type         Count      Complexity
X            1431             1629
C            2393             2408
M            6299            13000
F            1144             3600
T           11267            20637

real    0m11.840s
user    0m11.489s
sys     0m0.056s

Zope 2.11.4

time ./pygenie/ Zope-2.11.4-final/ -r
Total Cumulative Statistics
Type         Count      Complexity
X            3005             4549
C            6453             6500
M           23137            45392
F            4985            12960
T           37580            69401

Totally 5 files failed
FAILED to process file: /home/aufather/Downloads/Zope-2.11.4-final/lib/python/ZEO/scripts/
FAILED to process file: /home/aufather/Downloads/Zope-2.11.4-final/lib/python/zope/app/testing/
FAILED to process file: /home/aufather/Downloads/Zope-2.11.4-final/lib/python/zope/app/component/
FAILED to process file: /home/aufather/Downloads/Zope-2.11.4-final/lib/python/zope/app/component/
FAILED to process file: /home/aufather/Downloads/Zope-2.11.4-final/lib/python/zope/rdb/gadfly/

real    0m37.042s
user    0m36.302s
sys     0m0.140s

Twisted 10.1.0

time ./pygenie/ Twisted-10.1.0/ -r
Total Cumulative Statistics
Type         Count      Complexity
X             935             1282
C            3736             3922
M           16782            26142
F            1323             3268
T           22776            34614

Totally 6 files failed
FAILED to process file: /home/aufather/Downloads/Twisted-10.1.0/doc/core/howto/listings/udp/
FAILED to process file: /home/aufather/Downloads/Twisted-10.1.0/doc/core/howto/listings/udp/
FAILED to process file: /home/aufather/Downloads/Twisted-10.1.0/doc/historic/2003/pycon/deferex/
FAILED to process file: /home/aufather/Downloads/Twisted-10.1.0/doc/historic/2003/pycon/deferex/
FAILED to process file: /home/aufather/Downloads/Twisted-10.1.0/doc/historic/2003/pycon/deferex/
FAILED to process file: /home/aufather/Downloads/Twisted-10.1.0/twisted/python/test/

real    0m20.794s
user    0m20.525s
sys     0m0.036s

Even though I started this activity for fun, I was surprised by the speed and utility of pygenie. Some of the uses I can think of

  • Identify syntax errors within a project. Since pygenie uses the same python compiler, if pygenie could not process a file, the file cannot be processed by Python byte code compiler. Zope has 5 files which will not run while Twisted has 6. Django is clean in this front. When we take a closer look, it is test cases, demo code, tutorials, historical code etc that are broken.
  • Object orientation used within a project. The ratio between methods to functions is a fair indicator. The scores for the three projects are Django(5.5), Zope (4.6) and Twisted (12.7). This is a good indicator of existing style of code.
  • Overall size of a project. Pygenie report indicates that Zope is 3.4 times and Twisted is 1.7 times as big as Django.

The original pygenie is available here. Enhanced pygenie is available from my dropbox here.

Recently I measured some code metrics of a C/C++ project at work. I used C and C++ Code Counter(CCCC) and it is pretty good. There was a similar project at work which was almost completely in Python. I wanted to compare the two and searched for a suitable tool to measure McCabe’s Cyclomatic Complexity(CC) for Python code and found pygenie.

Original pygenie, by default, prints Files(X), Classes(C), Methods(M), Functions(F) only if their CC exceeds 7. If –verbose or -v is given, it prints CC irrespective of whether it crosses the threshold of 7 or not. But I wanted overall CC. My main motivation is purely for fun or for interesting numbers. Kind of like the calories burnt counter on a treadmill or a cycle computer for a bicycle. But overall CC can be useful in estimating testing effort.

Here is a sample run of original pygenie on its own source code.

aufather@simstudio:~/Downloads/pygenie_orig$ ./ complexity *.py
File: /home/aufather/Downloads/pygenie_orig/
This code looks all good!

File: /home/aufather/Downloads/pygenie_orig/
Type Name Complexity
F    main 8          

File: /home/aufather/Downloads/pygenie_orig/
This code looks all good!

And here is a sample run of modified pygenie on its own source code.

aufather@simstudio:~/Downloads/pygenie$ ./ * -c
File: /home/aufather/Downloads/pygenie/
This code looks all good!

File: /home/aufather/Downloads/pygenie/
This code looks all good!

File: /home/aufather/Downloads/pygenie/
This code looks all good!

Total Cumulative Statistics
Type         Count      Complexity
X               3                4
C               8                8
M              31               70
F              13               26
T              55              108

I added the total cumulative statistics and a summary for each file. Total is represented by ‘T’. This gives a fair idea of how complex the overall project is. ‘F’ stands for functions, ‘M’ stands for methods, ‘C’ stands for classes and ‘X’ stands for files. Also while I was at it, added a whole bunch of command line options to pygenie.

Here is the help for original pygenie. Valid commands are “complexity” or “all”.

aufather@simstudio:~/Downloads/pygenie_orig$ ./ all -h
Usage: ./ command [options] *.py

 -h, --help     show this help message and exit
 -v, --verbose  print detailed statistics to stdout

And the new options added.

aufather@simstudio:~/Downloads/pygenie$ ./ -h
Usage: [options] *.py

 --version             show program's version number and exit
 -h, --help            show this help message and exit
 -c, --complexity      print complexity details for each file/module
 -t THRESHOLD, --threshold=THRESHOLD
 threshold of complexity to be ignored (default=7)
 -a, --all             print all metrics
 -s, --summary         print cumulative summary for each file/module
 -r, --recurs          process files recursively in a folder
 -d, --debug           print debugging info like file being processed
 -o OUTFILE, --outfile=OUTFILE
 output to OUTFILE (default=stdout)

You can get the full zipped source code from my dropbox here

PS: I based my modification on revision 44 of pygenie. I got the original from

PPS: I could not find a better way of sharing the code in wordpress. I have updated the too so that unit test do not fail. If any one knows a better way of sharing code, please let me know. Dropbox works well for me.