Saturday, February 3, 2018

Prevent Process Termination over SSH Connection

When running heavy long-time running processes on a remote server through ssh, it is quite annoying that your processes are terminated when you are disconnected. Here is a tip just for this.

Let's assume you ssh into your server
$ ssh USERNAME@SERVER_IP

You want to run some program, which will run for quite some time.
$ ./a.out arg1 arg2 ...

What happens if your ssh connection is lost, perhaps due to unstable network connection, etc. Well, your program a.out will terminate itself, so you will have to re-run it. In fact, you will need to keep your ssh connection until the program completes, which is quite annoying.

One solution is using nohup
$ nohup ./a.out arg1 arg2 ... &

This will redirect the output to nohup.out file. With this, even if you disconnect from ssh connection, your program will keep running in the background.

Another solution, which is probably much more sophisticated, is to use screen. It is your work session where you can run processes, and you can easily detach from it to do other things while your processes running inside the session are intact and not terminated. You can always re-attach to the screen session and resume your processes still running.

To use this, you need to first create a new screen session
$ ssh USERNAME@SERVER_IP
$ screen -S SESSION_NAME

This will create a new screen. Here, you can run your program
$ ./a.out arg1 arg2 ...

Next, you can always detach from this current session with <CTRL + a> d. That is, press <CTRL> and <a> keys together, release the keys and then press <d> key and release. This will detach from the session. At this point, your program will keep running regardless of whether you disconnect from your ssh connection to the server.

To list running screen sessions, run
$ screen -list

To re-attach to the session, you simply run
$ screen -r SESSION_NAME

To exit the screen, you run
$ exit

By the way, you may notice that within screen session, your scroll will register as up/down arrow keys. If you want your scroll to work as scroll terminal output, you can run the following (credit to pistos)
$ echo 'termcapinfo xterm* ti@:te@' >> ~/.screenrc

Happy hacking!

Friday, February 2, 2018

Three Minutes Daily Vim Tip: Display Indentation Levels

It could be just me, but it is difficult for me to sometimes tell the indentation level of multiple indented lines. So here is a tip for someone like me:

IndentLine is an excellent vim plugin that does it for you. For installation, simply use Vundle; insert the following line in your .vimrc
Plugin 'Yggdroot/indentLine'

If you are not too familiar with Vundle, refer to this post where I explained how to use Vundle to install a different plugin.

Thursday, February 1, 2018

Debugging Bash Scripts

Bash scripts are extremely handy when mostly dealing with Linux commands, such as find, grep, sed, and so on. One can write functions, just like in any other languages. However, I have to admit that I haven't really dealt with Bash scripting much so far. One of the reasons, I suppose, is that it is difficult to debug Bash scripts. I didn't think there was any debugging tool for Bash. Well, I just realized that I was wrong! There is in fact a very nice debugger for Bash: bashdb

To install, you can run the following on macOS
$ brew install bashdb

or the following for Ubuntu
$ sudo apt-get install bashdb

To start debugging, run
$ bashdb bash_script.sh argument1 argument2 ...

Many of the commands are similar to that of gdb, but for more info, type in help. Also, this documentation can be of great reference.

Happy bashing!

Wednesday, January 17, 2018

Replace Lines with Given Strings in Shell

Let's learn how to replace given lines by given string. Consider, for example,
$ cat some_file
abcdefg
hijklmn
opqrstu
vwxyz
12345
67890

Let's replace line 2 with string THIS_IS_NEW_LINE2
$ sed '2s/.*/THIS_IS_NEW_LINE2/g' some_file
abcdefg
THIS_IS_NEW_LINE2
opqrstu
vwxyz
12345
67890

You can replace multiple lines with multiple -e options
$ sed -e '2s/.*/THIS_IS_NEW_LINE2/g' -e '3s/.*/THIS_IS_NEW_LINE3/g' some_file
abcdefg
THIS_IS_NEW_LINE2
THIS_IS_NEW_LINE3
vwxyz
12345
67890

Of course, you can always use -i option to modify file on the fly
$ sed -i '2s/.*/THIS_IS_NEW_LINE2/g' some_file
$ cat some_file
abcdefg
THIS_IS_NEW_LINE2
opqrstu
vwxyz
12345
67890

Print Selected Lines in Shell

Let's assume you want to print certain lines in given text. For example, consider
$ cat some_file
1
2
3
4
5
6
7
8
9

Let's say you want to print lines 3-5. This is very easy using sed:
$ sed -n '3,5p' some_file
3
4
5

Let's say you want to print all lines, except 3-5. This can be done with d option:
$ sed '3,5d' some_file
1
2
6
7
8
9

Note that by default sed prints all lines, so -n option asks sed to print only 3 to 5 lines and suppresses the default behavior. On the other hand, the d letter in the single quotes asks sed to delete the lines 3-5 from printing by default, so it prints only the rest of the lines instead. 

Find Line Number Matching Given Expression in Shell

Say you want to find the line number that matches a given search string. For instance,
$ cat some_file.txt
this is some file line 1
it was written in Vim line 2
let's learn unix commands line 3

To find an expression, say unix, we can use grep
$ grep unix some_file
let's learn unix commands line 3

To show the line number of the expression, use -n option
$ grep -n unix some_file
3:let's learn unix commands line 3

To output the line number only, pipe with cut command
$ grep -n unix some_file | cut -d : -f 1
3

Here, -d option specifies delimiter, and -f option specifies the position separated by the delimiter.

If there are multiple lines matching the given expression, you can always use -m option of grep to print just first N of them
$ grep -n -m 2 unix some_file | cut -d : -f 1
1
2

Happy hacking!

Monday, January 15, 2018

Build HTK on macOS with Minimum Effort

In the last post, I presented instructions to build HTK on Ubuntu. In this post, I will show you how to compile HTK on macOS.

As before, download HTK latest stable version source code, 3.4.1 at the time of writing, from here. Extract the files as usual.
$ tar xfz HTK-3.4.1.tar.gz
$ cd htk

Next, you need some required software tools from Apple.
$ xcode-select --install

Now, let's configure and make
$ ./configure --prefix=$(pwd)
$ make

You will probably encounter an error complaining
(cd HTKLib && /Applications/Xcode.app/Contents/Developer/usr/bin/make HTKLib.a) \

 || case "" in *k*) fail=yes;; *) exit 1;; esac;

gcc  -ansi -g -O2 -DNO_AUDIO -D'ARCH="darwin"' -I/usr/include/malloc -Wall -Wno-switch -g -O2 -I. -DPHNALG   -c -o HGraf.o HGraf.c
HGraf.c:73:10: fatal error: 'X11/Xlib.h' file not found

There are two options here. The first is to simply skip HSLab package that requires X11, thereby building without X11.
$ ./configure --prefix=$(pwd) --disable-hslab
$ make
$ make install

You will see all binary files in the bin folder.


The other option is to install X11 library. Download XQuartz from here and install it. This will create /opt/X11/ folder with necessary library and header files. We can now manually compile HTKLib package by specifying the header files
$ cd HTKLib
$ gcc  -ansi -g -O2 -DNO_AUDIO -D'ARCH="darwin"' -I/usr/include/malloc -Wall -Wno-switch -g -O2 -I. -DPHNALG   -c -o HGraf.o HGraf.c -I /opt/X11/include
$ cd ..

We also need to set library path:
$ export LIBRARY_PATH=/opt/X11/lib

Now, we are ready to resume the make process
$ make
$ make install

Viola! You should see bin directory with all HTK binary files!

Saturday, January 13, 2018

Build HTK in Ubuntu 16.04 with Minimum Effort

This post will walk through compilation of HTK on Ubuntu 16.04 64-bit. For macOS, take a look at this post.

First, you will need to download HTK from here. In this post, I will assume you download the latest stable version, 3.4.1. Extract the source files and change the directory
$ tar xfz HTK-3.4.1.tar.gz && cd htk

Because HTK was designed for 32-bit OS, we need to set the environment as such for 64-bit OS. Enter 32-bit environment by running
$ linux32 bash

Now, we do the usual things, starting off with configure
$ ./configure --prefix=$(pwd) && make

You will see an error:
HGraf.c:73:77: fatal error: X11/Xlib.h: No such file or directory
compilation terminated.

To find out which library you need, you can run the following
$ sudo apt-get install apt-file -y && apt-file update
$ apt-file search Xlib.h

You will see some lines of search result, one of which should read
libx11-dev: /usr/include/X11/Xlib.h

From this, you know that you need to install libx11-dev package.
$ sudo apt-get install libx11-dev -y

Let's resume make
$ make

You will encounter yet another error, complaining
Makefile:77: *** missing separator (did you mean TAB instead of 8 spaces?).  Stop.

It turns out that there is an error in HLMTools/Makefile line 77. In the very beginning of the line, there are a number of spaces, which should be replaced by a single tab.
<INSERT TAB HERE>if [ ! -d $(bindir) -a X_ = X_yes ] ; then mkdir -p $(bindir) ; fi

OK, let's continue.
$ make

You should see the compilation succeeds without any problem. Let's install this and you should find HTK binary files inside bin folder in the current directory!
$ make install
$ bin/HCopy

How to SSH into VirtualBox Guest from Host with NAT

Suppose you want to install Linux on your Macbook with VirtualBox. Since GUI is so heavy and resources-consuming, you may want to run Linux guest system in the headless mode and ssh into the guest system from your Macbook. The following shall let you do so.

1. Select Guest system in VirtualBox menu open up settings.

2. In the Network tab, make sure you have enabled an adapter with NAT.

3. Open up Advanced menu and click Port Forwarding.

4. Create a rule where Protocol = TCP, Host IP = 127.0.0.1, Host Port = 2222, Guest IP = 10.0.2.15, and Guest Port = 22.

5. Now, install openssh-server on your guest system, and you should be good to go by running the command from your host:
$ ssh -p 2222 guest_user@127.0.0.1