Getting the correct filename from the server with wget

If you’ve used the GNU wget utility, you know that often when wget downloads files, it saves files with odd file names. Sometimes these are long garbled filenames or the name ‘download’.

To get wget to honour the correct filename from the server, the method is to use wget with the –content-disposition flag.

Eg:

wget https://github.com/i-MSCP/imscp/archive/1.1.11.tar.gz --content-disposition

Unfortunately, –content-disposition does not have an alias, and it becomes easily a chore, typing that repeatedly. Some people create an alias for wget and use:

alias wget='wget --content-disposition'

The more proper method is to set the option in the configuration file in your home directory:

#cat ~/.wgetrc
content-disposition = on

Now wget will download files to the correct name.


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.
Midnight commander (mc) shows garbled text characters instead of graphics

Midnight commander (mc) shows garbled text characters instead of graphics

On a Debian ssh shell running via Putty on Windows 8, I started getting this weird display on midnight commander:

MWSnap009 2013-05-06, 21_29_08

 

To fix this,

In Putty>Load Session

Putty>Window>Translation>Remote character set>UTF8

MWSnap010 2013-05-06, 21_29_55

Also export the following variable in your .bashrc file:

NCURSES_NO_UTF8_ACS=1

Save the session. Exit and restart Putty. mc is fixed now.

MWSnap011 2013-05-06, 21_31_41

Credits: 
http://www.andremiller.net/content/getting-midnight-commander-line-drawing-work-putty

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Pipe grep to less with color

You need to use the grep –color=always option with less -R

Eg:

#grep -inr 'robots' * --color=always | less -R

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.
Using ‘less’ command with special character interpretation

Using ‘less’ command with special character interpretation

It is possible to use perl or bash for ascii colorization of output and send them to a log file. The advantage is that you can highlight errors, info messages etc in a log and view them later. The obvious disadvantage is that if your log file runs to a few thousand Kilobytes or more, you have to use the less command and the results are less than awesome.

For example,

A command like this:

tail -n 50 /root/wordpress_update.log

Normally results in:

MWSnap004 2013-04-02, 09_24_36

 

If less was used for piping:

tail -n 50 /root/wordpress_update.log | less

It now looks like this:

MWSnap005 2013-04-02, 09_27_21

Which is very ugly.

The solution is to use the -R switch in less:

tail -n 50 /root/wordpress_update.log | less -R

Which results in what you needed:

MWSnap006 2013-04-02, 09_29_23


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Adding auto complete for apt-get install

Question:

I have noticed that while on Ubuntu, if I type the following:

mc

and it isnt installed, I get the message below:

The program 'mc' is currently not installed. You can install it by typing: sudo apt-get install mc

However in Debian, that is not available. It just gives a “-bash: /usr/bin/mc: No such file or directory” message. How can I implement the same functionality in bash command line on Debian? Yes, I know that if it is package suggestion that I want, I can simply do a regex search using *apt-cache search*. However I was hoping for the simpler suggestion immediately on typing the name of the program.

As per discussions, the functionality is provided by the package command-not-found. However even after installing it, and also installing bash-completion package, this isnt available on the Debian bash shell.

Answer:

The suggestions are definitely provided by the package command-not-found.

However after installing the package, one needs to run the following command once to initialize the database:

update-command-not-found

After doing this, bash starts providing suggestions to install the correct package.

$htop
Could not find the database of available applications, run update-command-not-found as root to fix this
htop: command not found

$update-command-not-found
Downloading complete file http://ftp.de.debian.org/debian/dists/squeeze/Contents-i386.gz
I: Writing data for ftp.de.debian.org_debian_dists_squeeze_Contents-i386.gz ... . done

$htop
The program 'htop' is currently not installed. You can install it by typing:
apt-get install htop
htop: command not found

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Add bash autocompletion for your own script

Let your script be named utstart, and accepts any of the following arguments: start, stop, install, uninstall.

As an example, you would perhaps run your script like this from bash:

./utstart start

You would like to autocomplete start, by pressing TAB key like this:

./utstart s[TAB]

This is easy.

Prerequisite:

Install bash-completion package.

apt-get install bash-completion

Now create a new script like this:

#!/bin/bash
_utstart()
{
    local cur prev opts
    COMPREPLY=()
    cur="${COMP_WORDS[COMP_CWORD]}"
    prev="${COMP_WORDS[COMP_CWORD-1]}"
    opts="start stop install uninstall"

    if [[ ${cur} == * && ${COMP_CWORD} -eq 1 ]] ; then
        COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
        return 0
    fi
}
complete -F _utstart -o filenames utstart

Note that we have added the options “start stop install uninstall” in the script. Note also the last line contains the name of the function in our auto completion script, and also the name of our main bash script, which accepts autocompletion entries.

Once done, copy this file to /etc/bash_completion.d/utstart-auto.

Now you may change permissions on it to make it executable.

chmod +x /etc/bash_completion.d/utstart-auto

Source it with:

. /etc/bash_completion.d/utstart-auto

Now try the following:

./utstart s[TAB}

 

You will now be presented with start and stop as options.


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Get the time difference between two times, in hours, minutes and seconds

This one’s a no brainer.

date2=$(date +"%s")
#Wait for some time
date2=$(date +"%s")
diff=$(($date2-$date1))
echo "Time difference is $(($diff / 3600)) hours $(($diff / 60)) mins $(($diff % 60)) secs."

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Parse text files with assignments, and get the value – Bash

Suppose you have a text file containing the following:

strings=hello world

and you wanted to use the value of strings, which is “hello world” in your script

You can parse it with the following:

grep '^strings=' test | sed 's/strings=//g'

So the generic format is:

grep '^variablename=' filename | sed 's/variablename=//g'

The grep searches for a string beginning with ‘strings=’, in the file ‘test’, and outputs the following:

$grep '^strings=' test
strings=hello world

Next we have to remove the initial assignment part upto and including the ‘=’ sign, this is done by piping the output of grep, to sed.

Now the sed line substitutes the string “strings=” with an empty string. The substitution is done globally. Finally sed outputs just the value of the assigned variable.

Practical example:

I wanted to use the main User directory on the server, to make my backups more generic. I know that the file /etc/imscp/imscp.conf contains the following line:

USER_HOME_DIR = /var/www/virtual

So what I want is to get /var/www/virtual from this file, removing the assignment and spaces.

The solution is:

grep -i 'USER_HOME_DIR' imscp/imscp.conf | sed 's/USER_HOME_DIR[ ]=[ ]//'

which outputs:

/var/www/virtual

without the initial spaces. A space within a square brackets matches any number of spaces in the expression.

However, if you want to use these values in variables, you need to make minor adjustments to take care of variable escaping:

apache_home_var="USER_HOME_DIR"
apache_conf_file=/etc/imscp/imscp.conf
grep -i "$apache_home_var" "$apache_conf_file" | sed 's/'$apache_home_var'[ ]=[ ]//'

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Send mail from bash with attachment

Install mutt.

apt-get install mutt

 

Run mutt with the following syntax:

echo "This is the message body" | mutt -a /somewhere/file -s "subject of message" -- [email protected]

Note that when you use variables, there is a slight modification necessary:

echo $BODY | mutt -a /pathto/file -s "$SUB" -- [email protected]

Note that the subject coming after -s needs to be enclosed in quotes. In fact whenever you need to use variables as parts of the mutt command, you need to enclose the respective variable in double quotes. Single quotes of course escape the variables and would use the variable name as literals.

The sender unless specified is taken as your [email protected], where username is your root shell username, which is usually root.

You can use a file as the BODY field of the email. Can you guess the syntax?

cat /pathto/bodyfile | mutt -a /somewhere/attachmentfile -s "subject of message" -- [email protected]

cat /pathto/bodyfile | mutt -a /somewhere/attachmentfile -s “subject of message” — [email protected]

If you need to specify the sender:

[[email protected]] ~ #cat ~/.muttrc
set from="BackupServer <[email protected]>"

You can change the sender header at runtime, without relying on .muttrc, by building up a variable for the header using quotes for your literals, and concatenate (join) them with variables.

Eg:

Say you need your email to be coming from [email protected], with the name John B. Doe, you’d have to do something like this:

[email protected]
fname="John B. Doe"
hdrst='my_hdr From:'$fname' <'$email'>'

which will build the string:

my_hdr From:John B. Doe<[email protected]>

and send it to the mutt command as a custom header for your email.

echo $BODY | mutt -e "$hdrst" -a fullbackup -s "$SUB" -- [email protected]

Once again, note that the command will fail unless the variables $hdrst and $SUB are enclosed in double quotes.

References:

  1. Official mutt FAQ
  2. Mutt manual
  3. Cybergav blog post

You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Batch Remove spaces from filenames, and convert upper case to lowercase via bash

Use the following script:

#!/bin/bash

ls | while read -r FILE
do
    mv -v "$FILE" `echo $FILE | tr ' ' '_' | tr -d '[{}(),\!]' | tr -d "\'" | tr '[A-Z]' '[a-z]' | sed 's/_-_/_/g'`
done

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.