Monday, November 29, 2010

Troubleshooting Gaussian Calculation

For installing Gaussian TCP Linda, you may want to take a look  UNIX Binary Gaussian 09 Revision A.02 Installation instructions which has been updated recently.

"Troubleshooting Gaussian calculations" is a good resource that helps explain some of the errors and issues faced when running Gaussian.

Friday, November 26, 2010

Optimizing OpenFOAM Productivity in Clusters - Article from Desktop Engineering

This is an interesting article from Desktop Engineering "Optimizing OpenFOAM Productivity in Clusters". This article explore the performance gained when the HPC system is configured properly, there is a lot of productivity and gain in performance.

Thursday, November 25, 2010

Encountering Unauthorised Request for Qmgr command

Taken from the Torque Users Group

If you encounter "qmgr obj=que svr=default: Unauthorized Request" whenever you type the qmgr command, it is due to /var/spool/pbs/server_name not in sync with the name associated with eth0. To mitigate the problem,
  1. server_name must match the first no-localhost name in /etc/hosts
  2. You can check your hostname on your Linux Box, by typing hostname

Wednesday, November 24, 2010

Quick way to update user information for fingering

You can use the usermod command to update comments field or "Name" field

# usermod -c "User1 Name" user1

Alternatively, you can edit directly at the /etc/passwd 5th column
user1:x:524:100:User1 Name:/home/user1:/bin/bash

You can verify by fingering
# finger user1

Tuesday, November 23, 2010

Default ulimit setting in torque overide ulimit setting

Even though you may have setup the ulimit correctly as suggested by Blog entry "Checking that the ulimit is correct when login as ssh", your job may still encounter the error is you are using torque resource manager.
This is because the default setting in /etc/init.d/pbs_mom is set too low and overide the ulimit setting, the default setting at /etc/init.d/pbs_mom should be changed to

ulimit -l unlimited
ulimit -c unlimited
ulimit -n 32768
#ulimit -n 20000
#ulimit -i 20000

Checking that the ulimit is correct when login as ssh

To check that whether your ulimit work properly when you ssh, you should use the below command. To set ulimit, you may want to check on blog entry Encountering Segmentation Fault, Bus Error or No output
$ ssh node00 ulimit -l

To check that ulimits is set correctly for your mpirun

$ mpirun -np 8 bash -c ulimit -l

The 2 commands output should be similar.

Monday, November 22, 2010

Green Revolution Cooling (

AT SC10, I saw quite a unique and disruptive solution for cooling HPC stuff. It won SC 2009 disruptive technology of the year.

Green Revolution (GR) Cooling method of cooling datacentre is very revolutionary and innovative. According to Green Revolution Cooling, datacentre are cooled by submerging servers in dielectric fluid coolant (GreenDEF coolant).

According to its website, GR Cooling is the first to develop a complete rack and server rack solution using this method at a feasible up-front costs and high performance.

For more information, read the GR Cooling website.

Monday, November 15, 2010

Undergraduate Petascale Education Program

Petascale Education Program seeks to promote understanding and petascale computing among undergraduate and faculty. On the site, you can fund useful materials, undergraduate petascale modules etc....

Bootable Cluster CD

Bootable Cluster CD......Cluster in your pocket....
The BCCD was created to facilitate instruction of parallel computing aspects and paradigms......The BCCD image addresses this problem by providing a non-destructive overlay way to run a full-fledged parallel computing environment on just about any workstation-class system.....
Take a look

Sunday, November 14, 2010

Defunct but useful ClusterWorld Magazine

ClusterWorld Magazine, now defunct as it merge with Linux Magazine. The last edition is June 2005. But nonetheless, it has interesting information on MPI which is still applicable today.

Wednesday, November 10, 2010

Open CL Programming Webinar Series

This is a series of webinars where AMD experts discuss and answer questions about data parallel computing on GPU leveraging on the OpenCL(tm) architecture. These webinars will include beginning and advanced tracks offered at varying times

  1. OpenCL Programming Webinar Series ( OR
  2. OpenCL Programming Webinar Series (

Saturday, November 6, 2010

Graph Editor for Open-Source Computer Algebra System Sage

Graph editor part of the open-source computer algebra system Sage. Will need HTML 5

Friday, November 5, 2010

Installing lammps using Intel Compilers, OpenMPI and FFTW (Update)

This is an update of  Blog Entry Installing lammps using Intel Compilers, OpenMPI and FFTW

The steps are still relevant and workable, but if you are doing a lot of customisation pathing or you just simply wish to tweak the parameters in the configuration file, here is my sample script. I got my information from the Message threads of LAMMPS

Here are my settings, amending the $SOURCE/lammps-30Mar10/src/MAKE/Makefile.Linux

# linux = RedHat Linux box, Intel icc, MPICH2, FFTW

SHELL = /bin/sh
# ---------------------------------------------------------------------
# compiler/linker settings
# specify flags and libraries needed for your compiler

CC = mpicc
CCFLAGS = -g -O -I/usr/local/fftw/include -DFFT_FFTW
LINK = mpicc
USRLIB = -lfftw -lmpi
LIB = -lstdc++ -lpthread
SIZE = size

# ---------------------------------------------------------------------

# LAMMPS-specific settings
# specify settings for LAMMPS features you will use
# LAMMPS ifdef options, see doc/Section_start.html

# MPI library, can be src/STUBS dummy lib
# INC = path for mpi.h, MPI compiler settings
# PATH = path for MPI library
# LIB = name of MPI library:wq

MPI_PATH = -L/usr/mpi/intel/lib # where I store my openmpi binary and lib
MPI_LIB = -lmpi -lpthread

# FFT library, can be -DFFT_NONE if not using PPPM from KSPACE package
# INC = -DFFT_FFTW, -DFFT_INTEL, -DFFT_NONE, etc, FFT compiler settings
# PATH = path for FFT library
# LIB = name of FFT library

FFT_PATH = -L/usr/local/fftw/lib # I've put my fftw at /usr/local/fftw
FFT_LIB = -lfftw


You can add further flags at CCFLAGS. See my other blog entry
  1. Understanding -DOMPI_SKIP_MPICXX Flag for compiler/linker settings
  2. Understanding -DOMPI_IGNORE_CXX_SEEK Flag for compiler/linker settings

Thursday, November 4, 2010

Copy to remote machines with symbolic links intact

If you are copying directories with hard links and symbolic links to another remote servers intact. Once of the best method is to use rsync
# rsync -lH -rva gromacs remote_server:/usr/local
where l - copy symlinks as symlinks, H - preserve hard links, r - recursive, v - verbose, a - archive

Wednesday, November 3, 2010

Compiling Infiniband or OpenIB with OpenMPI and Intel Compilers on CentOS

Building on the Blog Entry "Useful Information to Compile Infinifband with OpenMPI", here is a more detailed writeup

I'm assuming you have compiled the Intel Compilers and Unzip the OpenMPI Package. Some of these information can be found on simple Ethernet-based Building OpenMPI with Intel Compiler (Ver 2) .
Don't configure and make yet for the OpenMPI Package.

Firstly to compile with OpenIB support,
#./configure --prefix=/usr/mpi/intel/ \ 
CC=icc CXX=icpc F77=ifort FC=ifort \
--with-openib \ 

# make all install 

To test whether you have compiled and installed correctly, you can run the "ompi_info" command and look for components for your networks.
# ompi_info | grep openib
You should get something like this depending on your component
MCA btl: openib (MCA v2.0, API v2.0, Component v1.4.1)

You are in good shape.

For more information, see OpenMPI, see
  1. 26. How do I build Open MPI with support for Open IB (Infiniband), mVAPI (Infiniband), GM (Myrinet), and/or MX (Myrinet)?

Tuesday, November 2, 2010

Understanding -DOMPI_SKIP_MPICXX Flag for compiler/linker settings

When I was compiling LAMMPS, I come across this Flag when preparing the compiler/linker settings
CCFLAGS =       -g -O -I/usr/local/fftw/include -DFFT_FFTW -DOMPI_SKIP_MPICXX -DOMPI_IGNORE_CXX_SEEK

What do -DFFT_FFTW -DOMPI_SKIP_MPICXX really mean? The answer is nicely given Rolf Vandevaart "Speeding Up C++ Compiles"

In a summary, from the blog
The issue is that if you are compiling C++ code, but you are not using the C++ bindings, the code can be slow to compile.  The workaround is to use the -DOMPI_SKIP_MPICXX flag which tells MPI not to include the C++ header files
And here are the results.  Not quite a 50% speedup, but close

Monday, November 1, 2010

Understanding -DOMPI_IGNORE_CXX_SEEK Flag for compiler/linker settings

When I was compiling LAMMPS, I come across this Flag when preparing the compiler/linker settings

What do -DOMPI_IGNORE_CXX_SEEK really seek to solve. It happens on openmpi-1.2 with Intel. This is due to BOTH stdio.h and the MPI C++ interface use SEEK_SET, SEEK_CUR, SEEK_END.
For more inforamtion. see this FAQs from NCSA
 I got an error message: SEEK_SET is #defined but must not be for the C++ binding of MPI. What does that mean?