Monday, April 14

Add memory info to python logging: several options?

Our program is using a lot of memory. Too much of memory, maybe. So I want to write memory of current process in logging information.

Getting the memory size of current process is the easy part (in Linux):
resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
I can think of several ways to add this into logging system: Adding a handler (derive the handler from either StreamHandler or FileHandler), or change the Formatter. Finally I decided to add a filter:
class MemFilter(logging.Filter):
    def filter(self, record):
        record.mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
        return True
logging.basicConfig(format='[%(levelname)s][MEM:%(mem)s] [%(asctime)s]:%(message)s')
logging.root.addFilter(MemFilter())

Case closed.

Labels:

Wednesday, April 2

Display current time for each command, in Python Interactive Interface

I like making the prompt sign (ps) into showing current time, so that I know exactly what time I executed a command, and what time it is completed. I made that for DOS command, Linux terminal, and MySQL terminal. Here I will show you how to do that in Python Interactive Interface (Python Interpreter).

Here is what it is after setting:



The first thing, setup a $PYTHONSTARTUP environment variable to point to a python file location. Exactly how to do that depends on your operating system. Now matter how it is done, the file it points to will be executed when "python" (Python Interactive Interface) is called.
In my Ubuntu, execute
export PYTHONSTARTUP=~/.pythonstartup.py
Next step, in my home folder "~", I created the file .pythonstartup.py as:
# filename: .pythonstartup.py
# after setting $PYTHONSTARTUP=~/.pythonstartup.py, this file will be executed before python interactive interface is open
import sys, time
sys.ps1 = time.strftime("%A %H:%M:%S") + ">>>"
Pretty simple, then the prompt sign (I guess that is the full name of "ps) will be current time, as "Wednesday 16:49:51" showing in the above screenshot.

PS: the way for showing the same information in linux command line (Ubuntu) is to modify the PS1:
PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\] [\D{%a} \t] \n\$ '
(add this in .bashrc)

For mysql client tool, add this line under [mysql] of the ~/.my.cnf:
prompt=\\u@\\d [\\w \\R:\\m:\\s]>

For MacBook, the operation is similar with Linux. Add these 2 lines into ~/.bash_profile (create this file if not existed):

export CLICOLOR=1
export PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\] [\D{%a} \t] \n\$ '

For "DOS" command line in Windows system, go to the Control Panel->System Properties->Advanced->Environment Variables, where you can modify path information. Check for "PROMPT". Modify it or add it if you don't have it yet. Set the value as "$P [$T] $_$G ", that will let your command line have path, time, and start from the second line.


If you are using Git Bash in Windows, it already have most of what I want, except the computer time. Open C:\Program Files\Git\etc\profile.d\git-prompt.sh to add one line 
PS1="$PS1"'\[\033[00m\] [\D{%a} \t]'                 # current time
after the other $PS1 lines. That will do the trick.


Labels: