How to dump your Whatsapp chat messages to a csv file

Pre-requisites:
Linux pc with java jdk installed (Does not work with oracle jdk).
adb in the path

First get your whatsapp database:
adb pull /sdcard/WhatsApp/Databases/msgstore.db.crypt12
Decrypt the file:
git clone https://gitlab.com/digitalinternals/whatsapp-crypt12.git
javac -classpath “lib/whatsapp_spongycastle.jar:.” crypt12.java
Now pull the key for whatsapp from your phone (Requires root):
adb shell
su
cp /data/data/com.whatsapp/files/key /sdcard
exit
adb pull /sdcard/key
java -cp “lib/whatsapp_spongycastle.jar:.” crypt12

sqlite3
sqlite> .open msgstore.db
sqlite> .tables
chat_list messages_fts_content
frequents messages_fts_segdir
group_participants messages_fts_segments
group_participants_history messages_links
media_refs messages_quotes
media_streaming_sidecar messages_vcards
message_thumbnails messages_vcards_jids
messages props
messages_edits receipts
messages_fts status_list
sqlite> .schema messages
sqlite> .mode csv
sqlite> .headers on


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Decompiling and recompiling an apk in Linux using apktool (Tested on Ubuntu 16.04 and Android L on One Plus One)

Install apktool first:
Get it from http://ibotpeaches.github.io/Apktool/
Download latest apktool from here.

cd ~/
mkdir apktool
wget https://raw.githubusercontent.com/iBotPeaches/Apktool/master/scripts/linux/apktool
wget https://bitbucket.org/iBotPeaches/apktool/downloads/apktool_2.1.1.jar
mv apktool_2.1.1.jar apktool.jar
chmod +x apktool*
sudo mv apktool* /usr/local/bin/

You’ve now installed apktool.

Create a temp directory for apktool, and get your apk file.
Get your apk file via adb/from the connected phone via a file manager:

adb shell
cp /data/app/com.poke.media.pokemon/base.apk /sdcard/
adb pull /sdcard/base.apk 

Now, decompile it:

apktool d base.apk

The directory base will contain the decompiled files.

After working on it and making the necessary modifications, you need to install it after recompiling it:
apktool b base
adb install ./base/dist/base.apk

There will be an error because you havent signed it yet.

$ adb install ./new/dist/base.apk
5527 KB/s (18972821 bytes in 3.352s)
	pkg: /data/local/tmp/base.apk
Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]

Signing recompiled apk files:
You need either the signtool that comes with java development kit (jdk)
Install jdk:

sudo apt-get install openjdk-9-jdk

If it has failed dependencies, like it did when I installed it on Ubuntu 16.04, try:

sudo aptitude install openjdk-9-jdk

jarsigner and keytool reqiured for signing will be installed by java jdk to the PATH. So, just do:

keytool -genkey -v -keystore my-release-key.keystore -alias mykey1 -keyalg RSA -keysize 2048 -validity 10000
Enter keystore password:  
Re-enter new password: 
What is your first and last name?
  [Unknown]:  Joel
What is the name of your organizational unit?
  [Unknown]:  None
What is the name of your organization?
  [Unknown]:  Somewhere
What is the name of your City or Locality?
  [Unknown]:  Trivandrum
What is the name of your State or Province?
  [Unknown]:  Kerala
What is the two-letter country code for this unit?
  [Unknown]:  IN
Is CN=Joel, OU=None, O=Somewhere, L=Trivandrum, ST=Kerala, C=IN correct?
  [no]:  yes

Generating 2,048 bit RSA key pair and self-signed certificate (SHA256withRSA) with a validity of 10,000 days
	for: CN=Joel, OU=None, O=Somewhere, L=Trivandrum, ST=Kerala, C=IN
Enter key password for <alias_name>
	(RETURN if same as keystore password):  
[Storing my-release-key.keystore]
jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore my-release-key.keystore ./new/dist/base.apk mykey1

Now install the apk:

adb install ./new/dist/base.apk
3863 KB/s (19077789 bytes in 4.821s)
	pkg: /data/local/tmp/base.apk
Success

You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Adding datestamp to output apk files in Gradle

In build.gradle, start off with:

import java.text.SimpleDateFormat
def buildTime() {
    def df = new SimpleDateFormat("yyyy-MM-dd'-'HH:mm") //you can change it
    df.setTimeZone(TimeZone.getTimeZone("IST"))
    return df.format(new Date())
}

Now add the following to modify the APK name:

applicationVariants.all { variant ->

        def apk = variant.outputFile;
        def newName;

        def time = buildTime()
        def versionNameSuffix = "-beta-build-${time}"

        newName = apk.name.replace(".apk", "-v" + defaultConfig.versionName + "-" + variant.buildType.name.toUpperCase() + versionNameSuffix + ".apk");

        newName = newName
                .replace("-" + variant.buildType.name, "")
                .replace(project.name, "droidzone");

        variant.outputFile = new File(apk.parentFile, newName);
        if (variant.zipAlign) {
            variant.outputFile = new File(apk.parentFile, newName.replace("-unaligned", ""));
        }

        logger.info('INFO: Set outputFile to ' + variant.outputFile + " for [" + variant.name + "]");
    }

You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

No javac found: Eclipse

Just link the location of the bin dir containing javac binary, to a dir /home/joel/android/adt-bundle-linux-x86/eclipse/jre:

cd /home/joel/android/adt-bundle-linux-x86/eclipse/jre
ln -s /home/joel/android/java/jdk1.7.0_09/bin bin

Here my copy of Oracle JDK is extracted to /home/joel/android/java/jdk1.7.0_09


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

.bashrc file

My customized .bashrc file follows:

alias l='ls -lh --color'
alias la='ls -lah --color'
alias download='curl -c - -O'
#PS1='\[\e[0;32m\]\u\[\e[m\] \[\e[1;34m\]\w\[\e[m\] \[\e[1;32m\]\$\[\e[m\] \[\e[1;37m\]'

#Blue prompt
PS1='\[\e[1;34m\][\u@\h \W]\$\[\e[0m\]'

#Green prompt
#PS1='\[\e[1;32m\][\u@\h \W]\$\[\e[0m\] '
PATH=$PATH:~/bash-advanced-scripts:~/toolchains/linux-x86/bin:/home/droidzone/android/adb

#Path to SDK
PATH=$PATH:~/android/envt/SoftwareDevelopmentKit/sdk/tools:~/android/envt/SoftwareDevelopmentKit/sdk/platform-tools

#Path to javac and java
PATH=$PATH:~/android/java/jdk1.7.0_09/bin

#apktool
PATH=$PATH:~/android/Apktool

#Java envt
JAVA_HOME=~/android/java/jdk1.7.0_09
export GREP_OPTIONS="--color -i -n"

d=.dircolors
test -r $d &amp;&amp; eval "$(dircolors $d)"

My root bashrc /root/.bashrc is:

alias l='ls -l --color'                                                                                                                
alias download='curl -c - -O'                                                                                                          
#PS1='\[\e[0;32m\]\u\[\e[m\] \[\e[1;34m\]\w\[\e[m\] \[\e[1;32m\]\$\[\e[m\] \[\e[1;37m\]'                                               
PS1='\[\e[1;31m\][\u@\h \W]\$\[\e[0m\]'                                                                                                
#'\[\e[1;32m\][\u@\h \W]\$\[\e[0m\] '                                                                                                  
PATH=$PATH:~/bash-advanced-scripts:~/toolchains/linux-x86/bin:/home/droidzone/android/adb                                              

#Path to SDK                                                                                                                           
PATH=$PATH:~/android/envt/SoftwareDevelopmentKit/sdk/tools:~/android/envt/SoftwareDevelopmentKit/sdk/platform-tools                    

#Path to javac and java                                                                                                                
PATH=$PATH:~/android/java/jdk1.7.0_09/bin                                                                                              

#apktool                                                                                                                               
PATH=$PATH:~/android/Apktool                                                                                                           

#Java envt                                                                                                                             
JAVA_HOME=~/android/java/jdk1.7.0_09

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Install Oracle Java JDK on Debian

Get the source file from Oracle.Since I used JDK 7, I visited this page and downloaded jdk-7u10-linux-i586.tar.gz (for Linux x86).

Extract it somewhere

curl -J -O -C - http://download.oracle.com/otn-pub/java/jdk/7u10-b18/jdk-7u10-linux-i586.tar.gz?AuthParam=1355492655_c33bc95a6d13e7345ba318a0b46bcc4a
mv jdk-7u10-linux-i586.tar.gz?AuthParam=1355492655_c33bc95a6d13e7345ba318a0b46bcc4a jdk-7u10-linux-i586.tar.gz
tar -xvvf jdk-7u10-linux-i586.tar.gz

Perform the installation:

sudo mv ./jdk1.7.0_10 /usr/lib/jvm/jdk1.7.0
sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0/bin/javac" 1
sudo update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0/bin/javaws" 1

Now choose the correct version of the programs if you have more than installed (as is most likely the case as the OS often comes with a version, and you install another). This is to avoid the error message that states that the version of java and javac clash (or something similiar).

sudo update-alternatives --config java
sudo update-alternatives --config javac
sudo update-alternatives --config javaws

 

[[email protected] droidzone]# sudo update-alternatives --config java
There are 2 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                            Priority   Status
------------------------------------------------------------
  0            /usr/bin/gij-4.4                 1044      auto mode
  1            /usr/bin/gij-4.4                 1044      manual mode
* 2            /usr/lib/jvm/jdk1.7.0/bin/java   1         manual mode

Press enter to keep the current choice[*], or type selection number:

As noted, I’ve chosen the version I just installed, which is jdk1.7.0.

Depending on the versions you have installed, you may need to repeat similiarly for:

[[email protected] droidzone]# sudo update-alternatives --config javac
There is only one alternative in link group javac: /usr/lib/jvm/jdk1.7.0/bin/javac
Nothing to configure.

And:

[[email protected] droidzone]# sudo update-alternatives --config javaws
There is only one alternative in link group javaws: /usr/lib/jvm/jdk1.7.0/bin/javaws
Nothing to configure.

You will notice that since I had only single versions of javac and javaws, I didnt need to choose another one. It may just be worthwhile to try these commands anyway.

Confirm that the version of java and javac correspond:

[[email protected] droidzone]# `which java` -version
java version "1.7.0_10"
Java(TM) SE Runtime Environment (build 1.7.0_10-b18)
Java HotSpot(TM) Server VM (build 23.6-b04, mixed mode)

[[email protected] droidzone]# `which javac` -version
javac 1.7.0_10
[[email protected] droidzone]#

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Install Libre office in Debian Squeeze

Remarks: Note the following procedure seemed to corrupt dependencies for my kernel to a point where it was easier to reinstall the OS than remove conflicting packages and install correct ones. 

Add a new repository to apt sources:

Create a new file in sources.list.d

kdesudo kate /etc/apt/sources.list.d/libre-office.list

Add this line:

deb http://backports.debian.org/debian-backports squeeze-backports main

Running command:

[[email protected] c]$ sudo apt-get install libreoffice

 

Reading package lists… Done
Building dependency tree
Reading state information… Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libreoffice : Depends: libreoffice-core (= 1:3.5.4+dfsg-3~bpo60+2) but it is not going to be installed
Depends: libreoffice-writer but it is not going to be installed
Depends: libreoffice-calc but it is not going to be installed
Depends: libreoffice-impress but it is not going to be installed
Depends: libreoffice-draw but it is not going to be installed
Depends: libreoffice-math but it is not going to be installed
Depends: libreoffice-base but it is not going to be installed
Depends: libreoffice-report-builder-bin but it is not going to be installed
Depends: libreoffice-filter-mobiledev but it is not going to be installed
Depends: libreoffice-java-common (>= 1:3.5.4+dfsg~) but it is not going to be installed
E: Broken packages

 

The dependency problems seemed difficult to fix, so I tried:

[[email protected] c]$ sudo aptitude -t squeeze-backports install libreoffice

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Installing javac for Apktool on linux

The build steps of the latest Apktool is detailed below

http://code.google.com/p/android-apktool/wiki/BuildApktool?tm=4

 

It uses something called gradlew to build the latest version of the binary.

The instructions are as follows:

We use gradle to build. Its pretty easy. First clone the REPO.

git clone git://github.com/iBotPeaches/Apktool.git
Move into the directory.

cd Apktool
Issue the build.

./gradlew build
Then look in

/brut.apktool/apktool-cli/build/libs/apktool-xxxxx.jar
Other commands such as clean are

./gradlew clean

If you attempt that and dont have JDK installed, it will error out.

You need to download the latest version of the JDK available as a .tar.gz file at http://www.oracle.com/technetwork/java/javase/downloads/index.html

I used jdk-7u9-linux-i586.tar.gz

Once extracted, set the path as the path of the folder containing the bin folder (which contains javac). Note that the path should be the parent of the bin folder, and not include the bin folder itself.

export JAVA_HOME=/home/droidzone/android/java/jdk1.7.0_09

Here, jdk1.7.0_09/bin contains javac.

The final file will be produced in a location brut.apktool/apktool-cli/build/libs/ relative to the parent and named of the form apktool-cli-1.5.1-SNAPSHOT.jar

So

cp brut.apktool/apktool-cli/build/libs/*jar ./

should get you the latest file.

Note that you need the wrapper too.

wget http://android-apktool.googlecode.com/files/apktool-install-linux-r04-brut1.tar.bz2
tar -jxvvf apktool-install-linux-r04-brut1.tar.bz2

Finally since the apktool refers to a file named apktool.jar, create a symlink.

 

Possible errors:

Exception in thread "main" java.lang.UnsupportedClassVersionError: brut/apktool/Main : Unsupported major.minor version 51.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:634)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:334)
Could not find the main class: brut.apktool.Main. Program will exit.

That happens when your main system version java is older than version 7.

You can solve it either by:

1. Upgrading your main java version

2. Setting the PATH of your local java to override your system java.

Eg: If PATH is :/usr/bin:~/bin, make it ~/bin:/usr/bin

3. Creating a symlink in /usr/bin, point it to your version

cd /usr/bin

su

mv java java.old

ln -s /home/droidzone/android/java/jdk1.7.0_09/bin/java java

 

 


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Downloading files in linux with Curl and Wget – Some simple examples

How to download files with Curl

The simplest format for curl is:

curl -O [URL]

That’s a capital “O” (Oh)

Eg:

curl -O http://droidzone.in/roms/supernova.zip

 

will download the file supernova.zip from remote and store it in the same name in the current location.

 

Resuming a Broken download with curl:

curl -C - -O android-ndk-r8c-linux-x86.tar.bz2 http://dl.google.com/android/ndk/android-ndk-r8c-linux-x86.tar.bz2

will resume downloading from http://dl.google.com/android/ndk/android-ndk-r8c-linux-x86.tar.bz2 to the local file android-ndk-r8c-linux-x86.tar.bz2

Note that that is a capital C and a small O (Oh)

Now here’s an interesting observation. Wget and curl can resume download of a download started by the other too!

How to download files with Wget

The basic syntax for downloading a file with wget is:

wget [URL]

 

Eg:

wget http://droidzone.in/roms/supernova.zip

Resuming a Broken download with wget:

wget -c http://droidzone.in/roms/supernova.zip[/code]
will resume the download into the same filename received from server, i.e supernova.zip, and will not overwrite it.

Specify a destination directory with wget:

This is done with the -P option. Eg: Getting a specific patch from Github
[email protected]:~/Note2Core-GT-N710x_Kernel$ wget https://github.com/glewarne/Note2Core-GT-N710x_Kernel/commit/2c305d77b1ebd8a45242312ae693764de11630a9.patch -P ../patches/
--2012-11-27 22:21:17--  https://github.com/glewarne/Note2Core-GT-N710x_Kernel/commit/2c305d77b1ebd8a45242312ae693764de11630a9.patch
Resolving github.com... 207.97.227.239
Connecting to github.com|207.97.227.239|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1177 (1.1K) 
Saving to: `../patches/2c305d77b1ebd8a45242312ae693764de11630a9.patch'

100%[============================================================================================================================&gt;] 1,177       --.-K/s   in 0s      

2012-11-27 22:21:18 (461 MB/s) - `../patches/2c305d77b1ebd8a45242312ae693764de11630a9.patch' saved [1177/1177]

Download using proper file name from server

Often, while using curl or wget to download files with urls like http://download.oracle.com/otn-pub/java/jdk/7u9-b05/jdk-7u9-linux-i586.tar.gz?AuthParam=1354451808_536b13c3c7c8eb652e674f6f7c23467b, you will find that it actually downloads file with a filename like 7u9-b05/jdk-7u9-linux-i586.tar.gz?AuthParam=1354451808_536b13c3c7c8eb652e674f6f7c23467b. Frankly it's a chore to rename it back to the correct filename jdk-7u9-linux-i586.tar.gz and wreaks havoc with any sort of automation you had in mind.

So it turns out that both wget and curl have experimental options to resolve the correct filename from server.

If we want to use curl to get a file with proper filename and also resume it if the file already exists, and get the proper filename from server we might do something like:

curl -C - -O -J http://download.oracle.com/otn-pub/java/jdk/7u9-b05/jdk-7u9-linux-i586.tar.gz?AuthParam=1354451808_536b13c3c7c8eb652e674f6f7c23467b

Here, -C is the flag to "Continue (resume) a file download if a partial file already exists"

-O specifies the output is to a file rather than stdout

-J makes sure that filename is resolved properly

 

The corresponding option for wget is to use the --content-disposition flag.

So our command would be:

wget --content-disposition -c http://download.oracle.com/otn-pub/java/jdk/7u9-b05/jdk-7u9-linux-i586.tar.gz?AuthParam=1354451808_536b13c3c7c8eb652e674f6f7c23467b

 

r


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.

Modifying USE flags in Gentoo

To modify the global USE flags, set the USE variable in /etc/make.conf

You can set USE flags for individual packages by specifying the package name and USE flag, in /etc/portage/package.use (the file or directory may not exist).

Eg of /etc/portage/package.use

dev-db/mysql berkdb -java[/code]

The above example disables java use flag and enables berkdb for package dev-db/mysql.

If you want only a temporary USE flag during emerge of just the current package, you can specify the flag on the same command line where you specify the emerge:

USE="-java" emerge seamonkey[/code]

To view the current USE flags, run:

emerge --info

Once some USE flags have been set, one must make Portage see the new flags, with:

emerge --update --deep --newuse world[/code]


You are reading this post on Joel G Mathew’s tech blog. Joel's personal blog is the Eyrie, hosted here.