Replace Lines with Given Strings in Shell

Let's learn how to replace given lines by given string. Consider, for example,

$ cat some_file

abcdefg

hijklmn

opqrstu

vwxyz

12345

67890

Let's replace line 2 with string THIS_IS_NEW_LINE2

$ sed '2s/.*/THIS_IS_NEW_LINE2/g' some_file

abcdefg

THIS_IS_NEW_LINE2

opqrstu

vwxyz

12345

67890

You can replace multiple lines with multiple -e options

$ sed -e '2s/.*/THIS_IS_NEW_LINE2/g' -e '3s/.*/THIS_IS_NEW_LINE3/g' some_file

abcdefg

THIS_IS_NEW_LINE2

THIS_IS_NEW_LINE3

vwxyz

12345

67890

Of course, you can always use -i option to modify file on the fly

$ sed -i '2s/.*/THIS_IS_NEW_LINE2/g' some_file

$ cat some_file

abcdefg

THIS_IS_NEW_LINE2

opqrstu

vwxyz

12345

67890

Print Selected Lines in Shell

Let's assume you want to print certain lines in given text. For example, consider

$ cat some_file

1

2

3

4

5

6

7

8

9

Let's say you want to print lines 3-5. This is very easy using sed:

$ sed -n '3,5p' some_file

3

4

5

Let's say you want to print all lines, except 3-5. This can be done with d option:

$ sed '3,5d' some_file

1

2

6

7

8

9

Note that by default sed prints all lines, so -n option asks sed to print only 3 to 5 lines and suppresses the default behavior. On the other hand, the d letter in the single quotes asks sed to delete the lines 3-5 from printing by default, so it prints only the rest of the lines instead. 

Find Line Number Matching Given Expression in Shell

Say you want to find the line number that matches a given search string. For instance,
$ cat some_file.txt
this is some file line 1
it was written in Vim line 2
let's learn unix commands line 3

To find an expression, say unix, we can use grep
$ grep unix some_file
let's learn unix commands line 3

To show the line number of the expression, use -n option
$ grep -n unix some_file
3:let's learn unix commands line 3

To output the line number only, pipe with cut command
$ grep -n unix some_file | cut -d : -f 1
3

Here, -d option specifies delimiter, and -f option specifies the position separated by the delimiter.

If there are multiple lines matching the given expression, you can always use -m option of grep to print just first N of them
$ grep -n -m 2 unix some_file | cut -d : -f 1
1
2

Happy hacking!

Build HTK on macOS with Minimum Effort

In the last post, I presented instructions to build HTK on Ubuntu. In this post, I will show you how to compile HTK on macOS.

As before, download HTK latest stable version source code, 3.4.1 at the time of writing, from here. Extract the files as usual.

$ tar xfz HTK-3.4.1.tar.gz

$ cd htk

Next, you need some required software tools from Apple.

$ xcode-select --install

Now, let's configure and make

$ ./configure --prefix=$(pwd)

$ make

You will probably encounter an error complaining

(cd HTKLib && /Applications/Xcode.app/Contents/Developer/usr/bin/make HTKLib.a) \

 || case "" in *k*) fail=yes;; *) exit 1;; esac;

gcc  -ansi -g -O2 -DNO_AUDIO -D'ARCH="darwin"' -I/usr/include/malloc -Wall -Wno-switch -g -O2 -I. -DPHNALG   -c -o HGraf.o HGraf.c

HGraf.c:73:10: fatal error: 'X11/Xlib.h' file not found

There are two options here. The first is to simply skip HSLab package that requires X11, thereby building without X11.

$ ./configure --prefix=$(pwd) --disable-hslab

$ make

$ make install

You will see all binary files in the bin folder.

The other option is to install X11 library. Download XQuartz from here and install it. This will create /opt/X11/ folder with necessary library and header files. We can now manually compile HTKLib package by specifying the header files
$ cd HTKLib
$ gcc  -ansi -g -O2 -DNO_AUDIO -D'ARCH="darwin"' -I/usr/include/malloc -Wall -Wno-switch -g -O2 -I. -DPHNALG   -c -o HGraf.o HGraf.c -I /opt/X11/include
$ cd ..

We also need to set library path:
$ export LIBRARY_PATH=/opt/X11/lib

Now, we are ready to resume the make process

$ make

$ make install

Viola! You should see bin directory with all HTK binary files!

Build HTK in Ubuntu 16.04 with Minimum Effort

This post will walk through compilation of HTK on Ubuntu 16.04 64-bit. For macOS, take a look at this post.

First, you will need to download HTK from here. In this post, I will assume you download the latest stable version, 3.4.1. Extract the source files and change the directory
$ tar xfz HTK-3.4.1.tar.gz && cd htk

Because HTK was designed for 32-bit OS, we need to set the environment as such for 64-bit OS. Enter 32-bit environment by running
$ linux32 bash

Now, we do the usual things, starting off with configure
$ ./configure --prefix=$(pwd) && make

You will see an error:
HGraf.c:73:77: fatal error: X11/Xlib.h: No such file or directory
compilation terminated.

To find out which library you need, you can run the following
$ sudo apt-get install apt-file -y && apt-file update
$ apt-file search Xlib.h

You will see some lines of search result, one of which should read
libx11-dev: /usr/include/X11/Xlib.h

From this, you know that you need to install libx11-dev package.
$ sudo apt-get install libx11-dev -y

Let's resume make
$ make

You will encounter yet another error, complaining
Makefile:77: *** missing separator (did you mean TAB instead of 8 spaces?).  Stop.

It turns out that there is an error in HLMTools/Makefile line 77. In the very beginning of the line, there are a number of spaces, which should be replaced by a single tab.
<INSERT TAB HERE>if [ ! -d $(bindir) -a X_ = X_yes ] ; then mkdir -p $(bindir) ; fi

OK, let's continue.
$ make

You should see the compilation succeeds without any problem. Let's install this and you should find HTK binary files inside bin folder in the current directory!
$ make install
$ bin/HCopy

How to SSH into VirtualBox Guest from Host with NAT

Suppose you want to install Linux on your Macbook with VirtualBox. Since GUI is so heavy and resources-consuming, you may want to run Linux guest system in the headless mode and ssh into the guest system from your Macbook. The following shall let you do so.

1. Select Guest system in VirtualBox menu open up settings.

2. In the Network tab, make sure you have enabled an adapter with NAT.

3. Open up Advanced menu and click Port Forwarding.

4. Create a rule where Protocol = TCP, Host IP = 127.0.0.1, Host Port = 2222, Guest IP = 10.0.2.15, and Guest Port = 22.

5. Now, install openssh-server on your guest system, and you should be good to go by running the command from your host:
$ ssh -p 2222 guest_user@127.0.0.1