Application lock file implementation

Its always a good practice to put a check in the code to see if another instance of the same app is running.

This is helpful in case you have a cron job that is called every hour and single execution of an app can take more then a hour, in this case if we dont have a check in place then that means we can have more then one instance of the same at one time.

Recently I had to add the same kind of check for one of my Python scripts, sharing that if it might help someone trying to do the same:

if os.access(os.path.expanduser("~/.application.lock"), os.F_OK):
       lck_fl = open(os.path.expanduser("~/.application.lock"), "r")
       lck_fl.seek(0)
       oldpid = lck_fl.readline()
       if os.path.exists("/proc/%s" % oldpid):
           print "lck_fl exist! process is already running"
           print "Process is already running as pid %s," % oldpid
           sys.exit(1)
       else:
           print "lck_fl exists but no PID found as current process"
           print "Removing lck_fl of application process: %s" % oldpid
           os.remove(os.path.expanduser("~/.application.lock"))
lck_fl = open(os.path.expanduser("~/.lck_fl.vestibular.lock"), "w")
lck_fl.write("%s" % os.getpid())
lck_fl.close

–

Shoaib Mir
shoaibmir[@]gmail.com

Printing PDF file with images

A few days back I was having a terrible time when I tried to print a few tickets that I had in a PDF document. It all started since I upgraded my desktop Ubuntu to 9.1, didnt have time at that moment so asked a colleague to do the print for me.

Once I got time, after a little bit of googling found out that it was a package that I was missing which is needed for the rendering of images and that needs to be installed before I go on printing PDF docs with images in them.

It was:

poppler-data


Hope it helps someone else who is going through the same pain of not able to print out PDF docs

–
Shoaib Mir
shoaibmir[@]gmail.com

Using pg_hba.d folder for HBA rules

Its been quite a while since I did post something and that was due to a very busy schedule at work but got lucky today to get a few mins and sharing something I always wanted to write about.

We got an environment at work where we have a quite a huge number of databases on each PostgreSQL development database server and it is because of the static images of the production DBs we need to create for QA and Dev most of the time. In such a scenario if we keep on adding rules to the pg_hba.conf file it can be a big mess with around 200-300 lines in there and thats a nightmare to manage for a DBA.

Solution – What we ended up doing for this was by having a folder pg_hba.d like this:


pg_hba.conf
pg_hba.d
postgresql.conf
postmaster.opts


and under pg_hba.d folder:


host1-dbuser1-trust.db
host2-dbuser2-md5.db


The above two files then contain the list of databases that need to given access for those hosts.


host1-dbuser1-trust.db:
db1
db2
db3


This way you can have host files for each of your staging servers that are used by QA and then have a list of databases that need to be given access from those.

Last step is adding details for using these pg_hba.d files, this can be done in pg_hba.conf file the following way:


host @pg_hba.d/host1-dbuser1-trust.db dbuser1 192.168.2.5/32 trust
host @pg_hba.d/host2-dbuser2-trust.db dbuser2 192.168.2.6/32 md5


This way instead of having the whole list of databases in pg_hba.conf file we now just have two entries and making it very easy to manage. When ever you need to give a new database access to those hosts just add it to the specific file for that host under pg_hba.d folder and reload the DB server.

–
Shoaib Mir
shoaibmir[@]gmail.com

PID lock file in python

I have a python script which when running on large disks can (rarely) run on for like more then a hour, and then I have got this cron setup which runs this script and the cron is set to run every hour. This means I can have several instances of the same Python script at one time. In order to get rid of that kind of scenario I did basic style of pid lock file implementation today:

#This is to check if there is already a lock file existing#
if os.access(os.path.expanduser("~/.lockfile.vestibular.lock"), os.F_OK):
        #if the lockfile is already there then check the PID number
        #in the lock file
        pidfile = open(os.path.expanduser("~/.lockfile.vestibular.lock"), "r")
        pidfile.seek(0)
        old_pd = pidfile.readline()
        # Now we check the PID from lock file matches to the current
        # process PID
        if os.path.exists("/proc/%s" % old_pd):
                print "You already have an instance of the program running"
                print "It is running as process %s," % old_pd
                sys.exit(1)
        else:
                print "File is there but the program is not running"
                print "Removing lock file for the: %s as it can be there
                because of the program last time it was run" % oldpid
                os.remove(os.path.expanduser("~/.lockfile.vestibular.lock"))

#This is part of code where we put a PID file in the lock file
pidfile = open(os.path.expanduser("~/.lockfile.vestibular.lock"), "w")
pidfile.write("%s" % os.getpid())
pidfile.close

Hope this can help someone doing the same.

–
Shoaib Mir
shoaibmir[@]gmail.com

Implementing Yes/No to continue

Here is a quick how-to for doing something like “Do you wish to continue Yes/No” within shell scripting:

 echo "Will you like to delete $dir under $PWD for a fresh extract? (y/n)"
 read opt
        case $opt in
                Y|y) cleanfolder $dir ;;
                [Yy][Ee][Ss]) cleanfolder $dir ;;
                N|n) exit ;;
                [Nn][Oo]) exit ;;
                *) exit;;
        esac

This will look for almost all kinds of combination of Yes/No options and do the appropriate action depending on the user input.

--
Shoaib Mir
shoaibmir[@]gmail.com

Option parser in Python

I have just started playing around with Python like a few weeks back and having lots of fun scripting in it. Today at work got to know of a very nice feature with Python which lets you easily configure the options taken from command line, which is called option parser.

Firstly, you need to include the library with ‘import’:

from optparse import OptionParser

Now you can use it like this:

usage = "usage: %prog -H hostname -p port -d diskpath -t threshold"
parser = OptionParser(usage=usage, version="%prog 1.0")
parser.add_option("-H", "--host", action="store", type="string", dest="hostname",
help="input host name for database server")
parser.add_option("-d", "--disk", action="store",
type="string", dest="diskpath", help="input disk path")
parser.add_option("-p", "--port", action="store", type="string",
dest="dbport", help="input port for database server")
parser.add_option("-t", "--thresh", action="store", type="int",
dest="thresh", help="input threshold for disk size ")

Where every add_option means adding a new argument on command line to check for and the best thing is you just put in the help section with the add_option function call and it automagically builds the full help section for the program to be use with “-h”, so that means you cant have an option with “-h” as that clashes with the default help option for option parser, and that was the reason I had to go with “-H” for hostname

In order to get the values into variables just simply use:

(options, args) = parser.parse_args()
if options.hostname:
   hostname=options.hostname
if options.diskpath:
   diskpath=options.diskpath
if options.dbport:
   dbport=options.dbport
if options.thresh:
  thresh=options.thresh

Now when you run the program with “-h” option you get the following:

shoaib@shoaib-desktop:~/Desktop/Scripts$ ./mgmnt -h
Usage: mgmnt -H hostname -p port -d diskpath -t threshold
Options:
--version             show program's version number and exit
-h, --help            show this help message and exit
-H HOSTNAME, --host=HOSTNAME
input host name for database server
-d DISKPATH, --disk=DISKPATH
input disk path
-p DBPORT, --port=DBPORT
input port for database server
-t THRESH, --thresh=THRESH
input threshold for disk size

–
Shoaib Mir
shoaibmir[@]gmail.com

GCC tricks for the Linux Kernel

While debugging some GCC specific linking problems, came across this very handy resource for gcc tips and tricks… very helpful indeed!

Check it out at –> http://www.ibm.com/developerworks/linux/library/l-gcc-hacks/index.html

–
Shoaib Mir
shoaibmir[@]gmail.com

Disable notification system in Ubuntu

I dont know about everyone else but for some reason I just found the notification system really annoying with my Ubuntu setup, after struggling with a few files and settings finally was able to get it all disabled.

Here is a quick how-to in case someone wants to disable/enable it….

For disabling use the following:

sudo mv /usr/share/dbus-1/services/org.freedesktop.Notifications.service
/usr/share/dbus-1/services/org.freedesktop.Notifications.service.disabled

To re-enable it:

sudo mv /usr/share/dbus-1/services/org.freedesktop.Notifications.service.
disabled /usr/share/dbus-1/services/org.freedesktop.Notifications.service

–
Shoaib Mir
shoaibmir[@]gmail.com

Scripting within psql

In one of my previous posts I was told of using PSQL instead of shell scripting to make things much simpler. That turned out to a good tip and with some help from my colleague at work I am now so used to PSQL in these kind of scenarios.

Just today I had to get the dead tuple percentages (using pgstattuple contrib) for tables within all databases on the server and here is how I was to get it done using PSQL and a simple shell script….

#!/bin/bash
PSQL="psql -A -t"
DBLIST=$($PSQL postgres -c "select datname from pg_database
where not datistemplate and datname not in ('postgres')")
for DB in $DBLIST; do
 echo $DB;
 psql $DB  < gettupd.sql > $DB.out
done
exit 0

gettupd.sql

\a
\t
\o /tmp/tupd.sql
select 'SELECT \'' || relname || '\', * from pgstattuple('''||schemaname||
'.'||relname||''');' from pg_stat_user_tables ;
\o
\i /tmp/tupd.sql
\! rm /tmp/tupd.sql
\a
\t

PSQL is just the way to go… 

–
Shoaib Mir
shoaibmir[@]gmail.com

A quick way to change index tablespace

On one of the database servers, I had to quickly move all indexes from all the databases on to a different disk in order to free up some space from the main disk. I thought of sharing it so that someone else wanting to do the same thing can get an idea for doing the same….

A quick how to,

First made a query to get all the indexes within a database using the following:

select pg_namespace.nspname||'.'||pg_class.relname  from pg_class, pg_namespace
where pg_class.relnamespace  = pg_namespace.oid and pg_class.relkind = 'i' and pg_class.relname
not like 'pg_%'

and then using a simple script, plugging in the the above mentioned query to do an ALTER INDEX for each index by setting the new table space:

#!/bin/bash
DNAME_LIST=$(psql postgres -A -t -c "select datname from pg_database where
not datistemplate and datname != 'postgres'")
for DATABASE in $DNAME_LIST; do
INAME_LIST=$($PSQL $DATABASE -A -t -c "select pg_namespace.nspname||'.'||pg_class.relname
from pg_class, pg_namespace where
pg_class.relnamespace  = pg_namespace.oid and pg_class.relkind = 'i' and pg_class.relname
not like 'pg_%'")
    ( for INDEX in $INAME_LIST; do echo "ALTER INDEX $INDEX SET TABLESPACE
new_disk_index;"; done ) | psql $DATABASE
done

–
Shoaib Mir
shoaibmir[@]gmail.com