Friday, July 6, 2012

Sort by more than one column Bash

Problem:

Sort this data using first and second column. 
chr10   10010946
chr1    10010947
chr1    10010948
chr1    10010949
chr1    10010950
chr1    10110951
chr10   10010952
chr10   10010953
chr10   10010954
chr10   10010955
chr10   10010956
chr10   10010957
chr10   10010941
chr10   1001942
chr10   1001943
chr10   1001944
chr10   1001945
chr10   1001958
chr10   1001959
chr10   10010960
chr10   10010961
chr10   10010962
chr10   10010963
chr10   10010964
chr10   10010965
chr10   10010966
chr10   10010967
chr10   10010968

Introduction:
If we use only #sort command (sort [file]):
chr10   10010941
chr10   10010946
chr10   10010952
chr10   10010953
chr10   10010954
chr10   10010955
chr10   10010956
chr10   10010957
chr10   10010960
chr10   10010961
chr10   10010962
chr10   10010963
chr10   10010964
chr10   10010965
chr10   10010966
chr10   10010967
chr10   10010968
chr10   1001942
chr10   1001943
chr10   1001944
chr10   1001945
chr10   1001958
chr10   1001959
chr1    10010947
chr1    10010948
chr1    10010949
chr1    10010950
chr1    10110951
If we use #sort with the second column as a number (sort -k2n [file]):
chr10   1001942
chr10   1001943
chr10   1001944
chr10   1001945
chr10   1001958
chr10   1001959
chr10   10010941
chr10   10010946
chr1    10010947
chr1    10010948
chr1    10010949
chr1    10010950
chr10   10010952
chr10   10010953
chr10   10010954
chr10   10010955
chr10   10010956
chr10   10010957
chr10   10010960
chr10   10010961
chr10   10010962
chr10   10010963
chr10   10010964
chr10   10010965
chr10   10010966
chr10   10010967
chr10   10010968
chr1    10110951
 Solution:
We need to sort with both columns (sort -k1,1 -k2,2n [file])
"Sort from first column to first column with default method and then, sort from second column to second columns with numeric order"
chr1    10010947
chr1    10010948
chr1    10010949
chr1    10010950
chr1    10110951
chr10   1001942
chr10   1001943
chr10   1001944
chr10   1001945
chr10   1001958
chr10   1001959
chr10   10010941
chr10   10010946
chr10   10010952
chr10   10010953
chr10   10010954
chr10   10010955
chr10   10010956
chr10   10010957
chr10   10010960
chr10   10010961
chr10   10010962
chr10   10010963
chr10   10010964
chr10   10010965
chr10   10010966
chr10   10010967
chr10   10010968

Monday, June 25, 2012

Install UCSC Genome Browser

Introduction:

How to install a minimal installation of UCSC Genome Browser on Centos6 x86_64 with in a custom directory.

Steps:

- Install apache and mysql:
 yum install httpd mysql-server mysql
- Start services:
service httpd start
service mysqld start
chkconfig httpd on
chkconfig mysqld oni
- Configure Apache:
cd /var/www
mkdir html
mkdir cgi-bin
Create a config file like: /etc/httpd/conf.d/ucsc_genomebrowser.conf:
#
# UCSC GENOME BROWSER
#

<Directory "/var/www/ucsc-genomebrowser/html">
     Order Allow,Deny
     Allow from all
</Directory>

<Directory "/var/www/ucsc-genomebrowser/cgi-bin">
     AllowOverride None
     Options None
     Order allow,deny
     Allow from all
</Directory>

XBitHack on
<Directory "/var/www/ucsc-genomebrowser/html">
     Options +Includes
</Directory>

ScriptAlias /genomebrowser/cgi-bin /var/www/ucsc-genomebrowser/cgi-bin
Alias /genomebrowser /var/www/ucsc-genomebrowser/html
Alias /style /var/www/ucsc-genomebrowser/html/style
Reload Apache: service httpd restart
- Preparing the system:
Execute and add to /etc/profile:
    export WEBROOT=/var/www/ucsc-genomebrowser/html
    export CGI_BIN=/var/www/ucsc-genomebrowser/cgi-bin
    export MYSQLDATA=/var/lib/mysql
cd /usr/local/
mkdir apache
cd apache/
ln -s /var/www/ucsc-genomebrowser/html/ htdocs
yum install openssl
cd /usr/lib64
ln -s libcrypto.so.10 libcrypto.so.6
ln -s libssl.so.10 libssl.so.6
- Getting web files:
rsync -avzP rsync://hgdownload.cse.ucsc.edu/cgi-bin/ $CGI_BIN
rsync -avzP rsync://hgdownload.cse.ucsc.edu/htdocs/ $WEBROOT/
mkdir  /gbdb
cd /gbdb
mkdir -p hg19/nib
cd hg19/nib
rsync -avzP --delete --max-delete=20 rsync://hgdownload.cse.ucsc.edu/gbdb/hg19/hg19.2bit .
cd  /gbdb/hg19/
ln -s nib/hg19.2bit
chcon -R unconfined_u:object_r:httpd_sys_script_exec_t:s0 /gbdb
chown -R apache.apache /gbdb/

- Set up the Database:
  - hgcentral DB:
wget http://hgdownload.cse.ucsc.edu/admin/hgcentral.sql
mysql> create database hgcentral
mysql --[mysqlaccessoptions] hgcentral < hgcentral.sql
   - hgFixed DB:
mysql --[mysqlaccessoptions] -e "create database hgFixed"
  - hg19 DB:
mysql> CREATE DATABASE hg19
cd /var/lib/mysql/hg19
rsync -avP rsync://hgdownload.cse.ucsc.edu/mysql/hg19/grp* .
rsync -avP rsync://hgdownload.cse.ucsc.edu/mysql/hg19/trackDb* .
rsync -avP rsync://hgdownload.cse.ucsc.edu/mysql/hg19/hgFindSpec* .
rsync -avP rsync://hgdownload.cse.ucsc.edu/mysql/hg19/chromInfo* .
rsync -avP rsync://hgdownload.cse.ucsc.edu/mysql/hg19/gold* .
rsync -avP rsync://hgdownload.cse.ucsc.edu/mysql/hg19/gap* .

- BD Permisions:

GRANT SELECT, CREATE TEMPORARY TABLES
  ON hgcentral.*
  TO readonly@localhost
  IDENTIFIED BY 'access';

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER
  ON hgcentral.*
  TO readwrite@localhost
  IDENTIFIED BY 'update';
 
GRANT SELECT, CREATE TEMPORARY TABLES
  ON hg19.*
  TO readonly@localhost
  IDENTIFIED BY 'access';

GRANT SELECT, CREATE TEMPORARY TABLES
  ON hgFixed.*
  TO readonly@localhost
  IDENTIFIED BY 'access';

 - Getting config file:
wget http://genome-test.cse.ucsc.edu/~kent/src/unzipped/product/ex.hg.conf
mv ex.hg.conf /var/www/ucsc-genomebrowser/cgi-bin/hg.conf
chmod 600 /var/www/ucsc-genomebrowser/cgi-bin/hg.conf
chown apache.apache /var/www/ucsc-genomebrowser/cgi-bin/hg.conf
chcon unconfined_u:object_r:httpd_sys_script_exec_t:s0 /var/www/ucsc-genomebrowser/cgi-bin/hg.conf

- Configuring:
Change users and passwords (readonly/access and readwrite/update)
[...]
defaultGenome=hg19
[...]

-Trash:
mkdir /var/www/ucsc-genomebrowser/trash
chown apache:apache /var/www/ucsc-genomebrowser/trash


REFERENCES:

http://genome.ucsc.edu/admin/mirror.html
http://bergmanlab.smith.man.ac.uk/?p=1111
http://genomewiki.ucsc.edu/index.php/Browser_Installation
http://genomewiki.ucsc.edu/index.php/Minimal_Browser_Installation

Thursday, June 21, 2012

Permission denied Apache in Centos 6

When you have a "Permission denied" error but the permissions of the files are correct:

File: /var/log/httpd/error_log
Couldn't open ./hg.conf , Permission denied, referer: http://127.0.0.1/
# ll hg.conf
-rw-------. 1 apache root 13659 Jun 21 19:12 hg.conf

Try to see more information with ls -laZ
# ls -laZ hg.conf
-rw-------. apache root unconfined_u:object_r:admin_home_t:s0 hg.conf
And try to change with:
# chcon unconfined_u:object_r:httpd_sys_script_exec_t:s0 hg.conf

And...... IT WORKS!


Monday, June 11, 2012

Exchange mail + contacts + calendar Thunderbird

Introduction:

Configure Thunderbird as a Exchange client using DavMail.

Steps:

Common:
- Install DavMail (http://davmail.sourceforge.net/)
    - DavMail is a gateway mailClient <-> MS Exchange
    - Configure it with:
        * Main tab -> OWA (Exchange) URL: https://YOUR_WEBMAIL_URL/
        * Advanced tab -> Default windows domain: YOUR_DOMAIN

Mail:
- Configure a new email account in thunderbird with:
    - Protocol: IMAP
    - Server: localhost
    - Port: 1143
    - User: YOUR USER WITHOUT DOMAIN
    - Pass: YOUR PASS
    - SMTP server: localhost
    - SMTP port: 1025

Contacts:
- Go to your Thunderbird Address Book
    - File -> New -> LDAP Directory...
        - General Tab:
            * Name: EXCHANGE
            * Hostname: localhost
            * Base DN: ou=people
            * Port number: 1389
            * Bind DN: YOUR USER WITHOUT DOMAIN
        - Advanced Tab:
            * Search filter: (objectclass=*)

Calendar:
- Install Lightning Extension (from tools -> Add-ons menu)
- Go to Calendar Tab (Events and Tasks -> Calendar)
- Right button on left column (down of the last calendar) -> New Calendar
    - Select On the Network
        * Format: CalDav
        * Location: http://localhost:1080/users/MAIL_ACCOUNT/calendar
    - Next
        * Calendar Name: EXCHANGE
    - Next
        * User: YOUR USER WITHOUT DOMAIN
        * Pass: YOUT PASSWORD

Conclusions:

With this configuration, you can use all capabilities of a MS Exchange server in Linux an OSX systems.

REFERENCES:

http://davmail.sourceforge.net/
http://davmail.sourceforge.net/thunderbirdcalendarsetup.html

Wednesday, June 6, 2012

Apace mod_python with CentOS 6

Introduction:

How to install mod_python.so CentOS 6 

We gonna use the svn trunk source code because the last stable version at http://archive.apache.org/dist/httpd/modpython/ doesn't compile (see ref. 3).

Steps:

Install dependences:
yum install gcc
yum install httpd-devel
yum install python-devel
yum install flex
yum install svn
Download source code with svn:
svn co https://svn.apache.org/repos/asf/quetzalcoatl/mod_python/trunk
cd trunk

Tuesday, June 5, 2012

Install Raid Monitor/Tools DELL PowerEdge 1950

Introduction:


System: CentOS release 6.2 (Final) x86_64

Install tools to monitor and admin DELL PowerEdge 1950 RAID ( RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 1078 )

Tool: MegaRAID Storage Manager (http://www.lsi.com/products/storagecomponents/Pages/MegaRAIDSAS8708EM2.aspx)

Steps:


Installing dependences:
yum install libstdc++.i686
Downloading and installing MSM tool:
wget http://www.lsi.com/downloads/Public/MegaRAID%20Common%20Files/11.06.00-05_Linux_MSM.zip
unzip 11.06.00-05_Linux_MSM.zip
tar -xzvf MSM_linux_installer-11.06.00-05.tar.gz
cd disk/
./RunRPM.sh
Creating symbolic link:
cd /sbin
ln -s /opt/MegaRAID/MegaCli/MegaCli64 MegaCli

Conclusions:

Now you can see the log of the RAID controller at messages log (/var/log/messages) like:

MR_MONITOR[11520]: <MRMON248> Controller ID: 0 Device removed Device Type: Disk Device Id: --:--:0
MR_MONITOR[11520]: <MRMON081> Controller ID: 0 State change on VD: 0 Previous = Optimal Current = Degraded
MR_MONITOR[11520]: <MRMON251> Controller ID: 0 VD is now DEGRADED VD 0
MR_MONITOR[11520]: <MRMON114> Controller ID: 0 State change: PD = --:--:0 Previous = Online Current = Failed
MR_MONITOR[11520]: <MRMON114> Controller ID: 0 State change: PD = --:--:0 Previous = Failed Current = Unconfigured Bad
MR_MONITOR[11520]: <MRMON091> Controller ID: 0 PD inserted: --:--:0
MR_MONITOR[11520]: <MRMON247> Controller ID: 0 Device inserted Device Type: Disk Device Id: --:--:0
MR_MONITOR[11520]: <MRMON114> Controller ID: 0 State change: PD = --:--:0 Previous = Unconfigured Bad Current = Unconfigured Good
MR_MONITOR[11520]: <MRMON114> Controller ID: 0 State change: PD = --:--:0 Previous = Unconfigured Good Current = Offline
MR_MONITOR[11520]: <MRMON106> Controller ID: 0 Rebuild automatically started: PD --:--:0
MR_MONITOR[11520]: <MRMON114> Controller ID: 0 State change: PD = --:--:0 Previous = Offline Current = Rebuild
And use MegaCli command to administer the RAID configuration.

REFERENCES:

https://twiki.cern.ch/twiki/bin/view/FIOgroup/DiskRefPerc