Move table to another tablespace in Postgresql

dbtest# \t
dbtest# \o dbtest_tablespace.txt
dbtest# select 'alter table '||tablename||' set tablespace temporary_tablespace;' from pg_tables where tablename like 'example%';
dbtest# select 'alter index '||indexname||'  set tablespace temporary_tablespace;' from pg_indexes where tablename like 'example%';
dbtest# \i dbtest_tablespace.txt

FusionIO kernel linux 3.2 patch

Recently I've got problems with fusionIO and linux new kernel because fusionIO doesn't support new kernels (> 2.6). Below is patch for kernel 3.2 ( I think it should work on 3.x too). FusionIO kernel patch

Log coloring with bash

Below is link to short script I've made to see different logs with highlight different patterns.
How to use it ?

shamrock@alucard:~/dev/shell$ ./cgrep.sh
Usage : ./cgrep.sh <-g|--grep,-l|--less,-m|--more,-t|--tail> file [line|pattern]
-g, --grep - use grep with highlight
-l, --less - use less with highlight
-m, --more - use more with highlight
-t, --tail - use tail with highlight
line - highlights whole line
pattern - highlights whole pattern
Available colours: black,red,green,yellow,blue,magenta,cyan,white
Available bold colours: blackb,redb,greenb,yellowb,blueb,magentab,cyanb,whiteb

It is possible to use sed different patterns inside quotation marks i.e.:

cgrep sample.log pattern redb "Process target" blueb [Cc]urrent magentab "[0-9]\{3,10\}"

How doe it work in practice?

shamrock@alucard:~/dev/shell$ cat test
Lorem ipsum
Test lorem ipsum
test test lorem ipsum
shamrock@alucard:~/dev/shell$ ./cgrep.sh --grep test redb [Ll]orem

Lorem ipsum
Test lorem ipsum
test test lorem ipsum

cgrep.sh

Recover mysql/postgresql root password

To recover MySQL password you simply need to 

1) shutdown mysql database if it's possible or just kill with TERM 

In Linux :

# service mysqld shutdown

For killing process

# kill -TERM `ps axxwww | grep mysqld`

2) Run database with "skip grants" option

# /bin/sh /usr/bin/mysqld_safe --defaults-file=/etc/my.cnf \

--pid-file=/var/run/mysqld/mysqld.pid --skip-grant-tables

3) Log in into database and change password

# mysql 

mysql> use mysql;
mysql> update user set password=PASSWORD("newpassword") where user="root";
mysql> quit;

4) Restart database and log in with new password

To recover Postgresql password

1) Edit postgresql config file pg_hba.conf  and change line from :

 local   all postgres  peer

to 

local all postgres trust

2) Restart postgresql server :

# service postgresql restart 

3) Log in into database and change password

# psql -U postgres

postgres=# use template1
postgres-# alter user postgres with password 'secret';

postgres-# \q

4) Restart postgresql server

Vim tips

Converting TAB to SPACE

:set et
:ret!

or convert all SPACE to TAB

:set et!
:ret! 

Joining lines based on pattern:

global join:

 :g/pattern/j 

single join:

 :/pattern/j 

Some useful things to .vimrc

set cursorline                  " highlight current line
hi cursorline guibg=#333333     " highlight bg color of current line
"hi tab guibg=#000333     " highlight bg color of current line
hi CursorColumn guibg=#333333   " highlight cursor

syntax on

MySQL - connection class in python

Simple class allowing connect to mysql databse and gather/insert data into it.

Import class system and MySQLdb

import sys
import MySQLdb

Connect to database - if an error occures display message and exit.

def __init__(self,host,user,password,database):
self.database_connect(host,user,password,database)
return

def database_connect(self,host,user,password,database):
try:
self.conn = MySQLdb.connect(host=host, user=user, passwd=password, db=database)
except MySQLdb.Error, e:
print "Connection error\n%d: %s\n" % (e.args[0],e.args[1])
sys.exit(1)
return

Try to connect to database and return cursor.

def cursor_return(self):
try:
self.conn.ping()
except:
self.database_connect()
return self.conn.cursor()

Get one result from database.

def result_return(self,query):
kursor = self.cursor_return()
try:
kursor.execute(query)
except MySQLdb.Error, e:
print "Query error\n%s\n%d: %s\n" % (query,e.args[0],e.args[1])
sys.exit(1)

result = kursor.fetchone()
kursor.close()
return result

Get results from database.

def results_return(self,query):
kursor = self.cursor_return()
try:
kursor.execute(query)
except MySQLdb.Error, e:
print "Query error\n%s\n%d: %s\n" % (query,e.args[0],e.args[1])
sys.exit(1)

results = kursor.fetchall()
kursor.close()
return results

Execute query.

def execute_query(self,query):
kursor = self.cursor_return()
try:
kursor.execute(query)
except MySQLdb.Error, e:
print "Query error\n%s\n%d: %s\n" % (query,e.args[0],e.args[1])
sys.exit(1)
kursor.close()
return

How to us it?

import DBMySQL

c=DBMySQL.DataBase("hostname","username","password","database_example")

t = c.results_return("SELECT id,title FROM films;")

for (i,j) in t:
print "%s %s " % (i,j)

1 Transporter
2 Taxi 1
3 Taxi 2
4 Taxi 3
5 Alien 2
6 Alien 1
7 Alien 3
8 Alien 4
9 The Thing
10 Reservoir dogs

Source code

Apache in jail

Sometimes you need to run few secure services on the same server.
You can use chrooted environment or use jail.

What is jail in FreeBSD ?
It's implementation of operating system-level virtualization that allows administrators to partition a FreeBSD box into several independent mini-systems.
It means that if someone breaks into the jail, he can only move inside jail (of course it depends on system's security).

In my example I've installed apache with php,mysql,mhash,libxml,curl  and some other features.

Step one
Create fresh jail.
You can use my script below:

root@alucard# cat > create_jail.sh

#!/bin/sh

data=`date +%Y-%m-%d-%H:%M`
echo $data
D=/usr/JAIL_$data
cd /usr/src
mkdir -p $D
make -j 24 world DESTDIR=$D
make -j 24 distribution DESTDIR=$D
mount -t devfs devfs $D/dev
cp /etc/resolv.conf /usr/$data/etc/
#EOF

Step two
Enter into the jail and install reqired packages.

root@alucard# chroot /usr/{$date}/ /bin/tcsh

root@alucard_jail# pkg_add -r perl
root@alucard_jail# pkg_add -r cmake
root@alucard_jail# pkg_add -r iconv

After installation make rehash and load libraries:

root@alucard_jail# rehash
root@alucard_jail# ldconfig -m /usr/local/lib

Step three
Download and install from source: apache,php, mysql and rest required software., i.e.:

root@alucard_jail# cd libtool-2.4.2
root@alucard_jail# ./configure
root@alucard_jail# make && make install
root@alucard_jail# ldconfig -m /usr/local/lib

root@alucard_jail# cd mhash-0.9.9.9
root@alucard_jail# ./configure --enable-static --with-gnu-ld
root@alucard_jail# make && make install
root@alucard_jail# ldconfig -m /usr/local/lib

root@alucard_jail# cd libxml2-2.7.1
root@alucard_jail# ./configure --with-iconv --with-html-dir --with-html-subdir --without-python
root@alucard_jail# make && make install
root@alucard_jail# ldconfig -m /usr/local/lib

root@alucard_jail# cd ../libxslt-1.1.19
root@alucard_jail# make && make install
root@alucard_jail# ldconfig -m /usr/local/lib

root@alucard_jail# cd ../curl-7.23.1
root@alucard_jail# ./configure
root@alucard_jail# make && make install
root@alucard_jail# ldconfig -m /usr/local/lib

root@alucard_jail# cd ../mysql-5.5.19
root@alucard_jail# cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql
root@alucard_jail# make && make install
root@alucard_jail# ldconfig -m /usr/local/mysql/lib/

root@alucard_jail# cd ../httpd-2.2.21
root@alucard_jail# ./configure --prefix=/usr/local/apache2 --enable-so --with-mpm=prefork \
--with-port=80 --with-ssl --enable-ssl --enable-modules-all \
--enable-rewrite --with-devrandom --with-egd --with-included-apr \
root@alucard_jail# make && make install
root@alucard_jail# ldconfig -m /usr/local/apache2/lib/

root@alucard_jail# cd ../php-5.3.8
root@alucard_jail# ./configure --prefix=/usr/local/php5 --with-layout=GNU \
--with-libxml-dir --enable-dba=shared --enable-safe-mode \
--with-mysql=/usr/local/mysql --enable-bcmath --with-pear --with-mhash \
--enable-soap --with-openssl --with-apxs2=/usr/local/apache2/bin/apxs \
--with-regex=system --with-bz2 --with-curl --with-xsl \
--with-pdo-mysql=/usr/local/mysql --without-iconv \
root@alucard_jail# make && make install
root@alucard_jail# ldconfig -m /usr/local/php5/lib/

Step four
Create required directories in temporary directory i.e.: /tmp/APACHE_JAIL
and copy required binaries and libraries (it is good moment to reduce apache installation
from unused binaries, manuals and help docs).
You can use script below:

root@alucard_jail# cat >  copy_libraries.sh

#!/bin/sh

PWD=`pwd`
PWD_ORIG=`pwd`
LDD_BIN="/usr/local/apache2/bin"
LDD_SO="/usr/local/apache2/modules"

if [ -d /tmp/APACHE_JAIL ]; then
 rm -rf /tmp/APACHE_JAIL
 fi

 mkdir /tmp/APACHE_JAIL
 cd /tmp/APACHE_JAIL/
 PWD_TEMP=`pwd`
 mkdir dev
 mkdir etc
 mkdir lib
 mkdir libexec
 mkdir bin
 mkdir tmp
 mkdir -p "var/run/"
 mkdir -p "usr/local"
 mkdir -p "var/log"

 for zm_tmp in `ls $LDD_BIN`
 do
 for zmienna in `ldd $LDD_BIN/$zm_tmp | awk '{print $3}'`
 do
   cp -n $zmienna "$PWD_TEMP/lib/"
   echo $zmienna
 done
 done

 for zm_tmp1 in `ls $LDD_SO/*.so`
 do
 for zmienna in `ldd $zm_tmp1 | awk '{print $3}'`
 do
   cp -n $zmienna "$PWD_TEMP/lib/"
   echo $zmienna
 done
 done


LDD_BIN="/usr/local/php5/bin"



 for zm_tmp in `ls $LDD_BIN`
 do
 for zmienna in `ldd $LDD_BIN/$zm_tmp | awk '{print $3}'`
 do
   cp -n $zmienna "$PWD_TEMP/lib/"
   echo $zmienna
 done
 done

 for zmienna in `ldd /usr/local/apache2/modules/libphp5.so | awk '{print $3}'`
   do
       cp -n $zmienna "$PWD_TEMP/lib/"
       echo $zmienna
 done

   chmod a+rx $PWD_TEMP/lib/*

   cp /bin/sh $PWD_TEMP/bin/

 for zm_tmp in `ls /tmp/APACHE_INSTALLS/bin/`
 do
   for zmienna in `ldd /tmp/APACHE_INSTALLS/bin/$zm_tmp | awk '{print $3}'`
   do
       cp -n $zmienna "$PWD_TEMP/lib/"
       echo $zmienna
   done
 done

cp -rp /usr/local/apache2 $PWD_TEMP/usr/local/

cp -rp /usr/local/php5 $PWD_TEMP/usr/local/

cp /libexec/ld-elf.so.1 $PWD_TEMP/libexec/

cp -rp /usr/share/zoneinfo/Europe/Warsaw $PWD_TEMP/etc/localtime

chown -R root:wheel $PWD_TEMP
find $PWD_TEMP -type d -exec chmod a+rx {} ";"
find $PWD_TEMP -type f -exec chmod a+r {} ";"
find $PWD_TEMP/lib -type f -exec chmod a-wx {} ";"

#### EOF

Step five
Copy files into your temporary "etc" directory 
(in my case /tmp/APACHE_JAIL/etc )

auth.conf
host.conf
login.conf
master.passwd
pwd.db
spwd.db
group
localtime
login.conf.db
passwd
resolv.conf

You can edit files and delete unnecessary things.
For example group file may look like this

wheel:*:0:root
www:*:80:
nogroup:*:65533:
nobody:*:65534:

Step six
Copy from temporary jail your temporary directory with apache installation into final destination for jail.

Step seven
Configure apache and php in jail.

Step eight
Add into /etc/rc.conf jail settings i.e.:

syslogd_flags="-m 5 -l /usr/jail/APACHE_JAIL/var/run/log -n -s"
# Jail
jail_enable="YES"
jail_list="apache"
jail_set_hostname_allow="YES"
jail_socket_unixiproute_only="YES"
jail_sysvipc_allow="NO"

jail_apache_exec="/usr/local/apache2/bin/httpd -k start -D SSL"
jail_apache_rootdir="/usr/jail/APACHE_JAIL"
jail_apache_hostname="apache.aster.pl"
jail_apache_ip="192.168.10.7"
jail_apache_devfs_enable="YES"
jail_apache_fdescfs_enable="NO"
jail_apache_procfs_enable="NO"
jail_apache_mount_enable="NO"

Step nine
Reload syslog and run jail.